Cloud Giant

It has been years since Amazon began innovating in the hosting market with their cloud hosting, but I can still remember the day I spun up an EC2 instance and cried "WHAT?" over my coffee, staring at a Linux machine that cost me US$ 0.10. (By the time I had stopped staring and fiddling around, it probably cost me much more.) Nor can I forget the hours configuring machines to do fairly simple things. All along I wished for a tool just to spin up a server and all the stuff that goes with it and just make it work.

Well, for some time there has been: Elastic Beanstalk [1]. This tool isn't new, but if you're wandering around the cloud hosting landscape, I find this a much better starting point than the usual configure-your-cloud-from-scratch approaches.

Elastic Beanstalk is similar in its approach to Heroku, which is now just one of many platforms on which you git push your project, then a back-end system – Heroku, Elastic Beanstalk, etc. – deals with getting that project onto a server and behind a URL for you.

The big difference between Beanstalk and many of the other platforms is that, with AWS, the underlying architecture is exposed, so you can dig into it. If your app is successful, or just becomes more complex than the platform can manage, you can take the Beanstalk configuration of servers, load balancers, and the like and extend it.

But, you're not there yet. First, you'll want to play with Elastic Beanstalk.

Launching

To launch an application, you start by selecting the platform – Tomcat, IIS, Node, PHP, Python, Ruby – or you can customize the platform yourself, which makes it a nice fit if you're building the platform for, say, massive online coding schools, and you just want people to be able to spin up the right environment, or any situation in which you want people to be able to spin up from your configuration.

To keep things simple, select PHP; that's it, it's already launching. The UI whirls while your environment boots.

While this is happening, have a look around at what you have, and you'll notice the individual services of AWS being used to host the application. Notice Beanstalk calls it the "application," not instance. This is deliberately a level up from the normal nuts and bolts.

Again, if you've looked at Heroku, the core concepts are similar. The environment is wrapped up for you. You have a CLI tool to push to. Scaling happens out of the box. Like EC2 and other AWS services, monitoring and events show up in their respective tabs; however, unlike Heroku, Elastic Beanstalk (EBS) exposes the underlying technology used to power it, which gives them license to bill it as being "impossible to outgrow."

EBS is free because it's simply a wrapper for the existing AWS services. Although this method might make it very attractive for prototyping and rapid deployment, costing it becomes a little more fiddly than with other PaaS offerings.

Setting Up the Environment

You to unzip the eb CLI tool [2] somewhere (e.g., ~/bin/eb/), then include the path to eb:

export PATH=$PATH:~/bin/eb/eb/macosx/python2.7

First, you must create a Git repo,

git init .
git add .
git commit -m "Starting..."

then init the eb tool with:

eb init

Enter your AWS access key ID and secret key, then choose a region and name the app. I named mine DanApp for the purposes of this exercise. Accept the environment name as given. The next option is much like the GUI: Choose 64bit Amazon Linux running PHP 5.4, which for me was option 2:

    1) 32bit Amazon Linux running PHP 5.4
    2) 64bit Amazon Linux running PHP 5.4
    3) 32bit Amazon Linux running PHP 5.3
    4) 64bit Amazon Linux running PHP 5.3
    5) 32bit Amazon Linux running Node.js
    6) 64bit Amazon Linux running Node.js
    ...

Don't create an RDS instance this time, so type n for that option; however, notice how easy launching a whole DB-backed application would have been. That's some fun for you to have on your own later.

In a moment, you'll launch this killer empty app, but before then, take a look around the eb CLI tool:

$ eb --help
...
COMMAND  init, branch, start, status, update, stop, delete, logs, events, push
...

The status gives a quick summary of what's going on with your environment. At the end of this tutorial, you should shut things down so you can check that everything's dead using:

$ eb status
Environment "DanApp-env" is not running.

To do something interesting, start the environment:

$ eb start
Starting application "DanApp".
Would you like to deploy the latest Git commit to your environment? [y/n]: y

Booting this is much like booting anything with Cloud Formation – it can take a while for the instances to come to life and get IPs and for everything to complete. While this is booting, you can poke around in the web UI and look at the progress, just as when you launched through the web UI. When done booting, you'll see output like that in Listing 1.

ElasticBeanstalk git:(master) eb start
 Starting application "DanApp".
 Would you like to deploy the latest Git commit to your environment? [y/n]: y
 remote:
 remote: error: Unable to deploy application version: No
Environment found for EnvironmentName = 'DanApp-env'.
 remote:
 To
https://AKIAJ2XDUJ2NAFXXXXXXXX:
XXXXXXa9a9fe7e0fd7XXXXXXXXa94d94942@git.elasticbeanstalk.eu-west
-1.amazonaws.com/v1/repos/44616e417070/commitid/
6335366636313033613837366239366230396534333832386564323637313461
6533383837653130/environment/44616e4170702d656e76
 * [new branch] HEAD -> master
 Waiting for environment "DanApp-env" to launch.
 2013-05-08 20:02:27 INFO createEnvironment is starting.
 2013-05-08 20:02:32 INFO Using
elasticbeanstalk-eu-west-1-894012917938 as Amazon S3 storage
bucket for environment data.
 2013-05-08 20:03:03 INFO Created load balancer named:
awseb-e-m-AWSEBLoa-OXVMML7XC6PR
 2013-05-08 20:03:27 INFO Created security group named:
awseb-e-mnpsy5bpzk-stack-AWSEBSecurityGroup-18JDKTE9QITLU
 2013-05-08 20:03:32 INFO Created Auto Scaling launch
configuration named:
awseb-e-mnpsy5bpzk-stack-AWSEBAutoScalingLaunchConfiguration-
1QR56F972DMG2
 2013-05-08 20:04:57 INFO Created Auto Scaling group
named:
awseb-e-mnpsy5bpzk-stack-AWSEBAutoScalingGroup-12BAR59E5FUDM
 2013-05-08 20:04:57 INFO Waiting for EC2 instances to
launch. This may take a few minutes.
 2013-05-08 20:07:45 INFO Created Auto Scaling group
policy named:
arn:aws:autoscaling:eu-west-1:894012917938:scalingPolicy:
927c9769-d96e-46ba-b08f-099650ae7a3d:autoScalingGroupName/awseb-
e-mnpsy5bpzk-stack-AWSEBAutoScalingGroup-12BAR59E5FUDM:
policyName/awseb-e-mnpsy5bpzk-stack-
AWSEBAutoScalingScaleDownPolicy-KW4NGGQ0LULU
 2013-05-08 20:07:48 INFO Created CloudWatch alarm named:
awseb-e-mnpsy5bpzk-stack-AWSEBCloudwatchAlarmLow-14SOWYQGJ2F5X
 2013-05-08 20:07:51 INFO Created Auto Scaling group
policy named:
arn:aws:autoscaling:eu-west-1:894012917938:scalingPolicy:
22724a85-a99c-4a05-a66a-cb56a70f54e7:autoScalingGroupName/awseb-
e-mnpsy5bpzk-stack-AWSEBAutoScalingGroup-12BAR59E5FUDM:
policyName/awseb-e-mnpsy5bpzk-stack-
AWSEBAutoScalingScaleUpPolicy-1ETLUW450U2V1
 2013-05-08 20:07:53 INFO Created CloudWatch alarm named:
awseb-e-mnpsy5bpzk-stack-AWSEBCloudwatchAlarmHigh-198IBXQXHBDPH
 2013-05-08 20:09:25 INFO Application available at
DanApp-env-ceqwdhduep.elasticbeanstalk.com.
 2013-05-08 20:09:25 INFO Successfully launched
environment: DanApp-env
 Application is available at
"DanApp-env-ceqwdhduep.elasticbeanstalk.com".

What just happened here? Using the AWS keys, the code is pushed to a Git server running on Elastic Beanstalk. Beanstalk then starts the launch, which involves creating an S3 bucket, launching the EC2 instance, creating the autoscaling groups, and initiating some monitoring. All this is done for you, so you don't really have to worry about what's going on.

When the environment is running, you'll have a URL like http://DanApp-env-ceqwdhduep.elasticbeanstalk.com/, which delivers a basic page – whatever you put into your app.

Updates Are More Interesting

To update the app, edit anything – create some files, add some HTML, whatever you like – then git add, commit it, and then push using git aws.push (Listing 2).

git aws.push
 ElasticBeanstalk git:(master) git aws.push
 Counting objects: 5, done.
 Delta compression using up to 4 threads.
 Compressing objects: 100% (3/3), done.
 Writing objects: 100% (3/3), 331 bytes, done.
 Total 3 (delta 0), reused 0 (delta 0)
 remote:
 To https://XXXXX:XXXXX@git.elasticbeanstalk.eu-west-1.amazonaws.com/v1/repos/44616e417070/commitid/37346533646166656662386330633434363938363361616635383863363963356164326535346466/environment/44616e4170702d656e76
 c56f610..74e3daf HEAD -> master

Your app is updated. For hacking something together quickly, this can be a dream. Bash it out locally, spin up a Beanstalk application and git aws.push a few times until it's done.

After pushing each time, you visit http://<your app URL>/ in your browser again and there it is. Sometimes, updating took a few minutes until I eventually added http://danapp-env-ceqwdhduep.elasticbeanstalk.com/?junk=123 to jog whatever cache was in the way into giving me an updated version.

Give It a Poke

You can poke around a bit and have some fun by uploading a script like:

<?php
print_r($_SERVER);
?>

(Adjust this to whatever language you're using – just throw out the environment variables to see what is happening on that server.) To see the environment, use $_SERVER['/var/www/html/index.php'].

Git and Branches

One server is cool, but what about an application server for each branch, so you can push to one for development and to another for stable? Start by checking out a new branch,

$ git checkout -b stable

Then, ask eb what's going on (Listing 3).

$ eb status
The current branch "stable" is not associated with an Elastic Beanstalk environment. Call "eb branch" to set up a new environment for this branch.
Proceeding with default settings.
URL : DanApp-env-ceqwdhduep.elasticbeanstalk.com
Status : Ready
Health : Green

The output suggests you call eb branch; to do so, enter:

$ eb branch
The current branch is "stable".
Enter an AWS Elastic Beanstalk environment name (auto-generated value is "DanApp-stable-env"):
Do you want to copy the settings from environment "DanApp-env" for the new branch? [y/n]: y

So, where are you now? To find out, enter:

$ eb status
Environment "DanApp-stable-env" is not running.

You know what to do about that: Start it up:

$ eb start
...the usual stuff

Now pause to realize that you've just changed branch, copied the *entire stack config*, and launched a clone of the live environment in a few lines. That is neat and means that when you try something, much as with a new way of caching on the homepage or logging in with Twitter, you can spend an extra couple of minutes sending it to your team to see what they think. When they say – "but will it work on the live servers?" – you can say, "yes," because you know it will.

While you're waiting, you can see two environments in the AWS web console. The startup is the same set of steps, but it's interesting to see that this is the stable version (Listing 4).

Starting application "DanApp".
 Waiting for environment "DanApp-stable-env" to launch.
 2013-05-08 20:20:43 INFO createEnvironment is starting.
 2013-05-08 20:20:48 INFO Using elasticbeanstalk-eu-west-1-894012917938 as Amazon S3 storage bucket for environment data.
 2013-05-08 20:21:04 INFO Created load balancer named: awseb-e-h-AWSEBLoa-1118ULBZLSFGH
 2013-05-08 20:21:34 INFO Created security group named: awseb-e-hxaxpp3bsa-stack-AWSEBSecurityGroup-ZNUGTMANJ7WO
 2013-05-08 20:21:40 INFO Created Auto Scaling launch configuration named: awseb-e-hxaxpp3bsa-stack-AWSEBAutoScalingLaunchConfiguration-12RI7463WQWE3
 2013-05-08 20:23:25 INFO Waiting for EC2 instances to launch. This may take a few minutes.
 2013-05-08 20:23:25 INFO Created Auto Scaling group named: awseb-e-hxaxpp3bsa-stack-AWSEBAutoScalingGroup-1NMEE2RF9GMBP
 2013-05-08 20:25:11 INFO Created Auto Scaling group policy named: arn:aws:autoscaling:eu-west-1:894012917938:scalingPolicy:8bc35c42-856d-4848-8111-9ec4e4df0fcf:autoScalingGroupName/awseb-e-hxaxpp3bsa-stack-AWSEBAutoScalingGroup-1NMEE2RF9GMBP:policyName/awseb-e-hxaxpp3bsa-stack-AWSEBAutoScalingScaleDownPolicy-179VH5P3Y0VH9
 2013-05-08 20:25:13 INFO Created CloudWatch alarm named: awseb-e-hxaxpp3bsa-stack-AWSEBCloudwatchAlarmLow-1Q23UL8HF2B6K
 2013-05-08 20:25:16 INFO Created Auto Scaling group policy named: arn:aws:autoscaling:eu-west-1:894012917938:scalingPolicy:732bdc0c-4734-4d27-b02e-99d716c66dc6:autoScalingGroupName/awseb-e-hxaxpp3bsa-stack-AWSEBAutoScalingGroup-1NMEE2RF9GMBP:policyName/awseb-e-hxaxpp3bsa-stack-AWSEBAutoScalingScaleUpPolicy-1Q75QYV2QAY6X
 2013-05-08 20:25:18 INFO Created CloudWatch alarm named: awseb-e-hxaxpp3bsa-stack-AWSEBCloudwatchAlarmHigh-1AD9DJ2JI6IPZ
 2013-05-08 20:26:56 INFO Application available at DanApp-stable-env-3jhebnynie.elasticbeanstalk.com.
 2013-05-08 20:26:56 INFO Successfully launched environment: DanApp-stable-env
 Application is available at "DanApp-stable-env-3jhebnynie.elasticbeanstalk.com".

To get your app to appear at DanApp-stable-env-3jhebnynie.elasticbeanstalk.com, you need to push to it with git aws.push (Listing 5) and wait a couple of minutes. You now have two environments running, each running its own code, and each running its own virtualized hardware – and you haven't configured a single thing.

$ git aws.push
Pushing to environment: DanApp-stable-env
Counting objects: 7, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (7/7), 589 bytes, done.
Total 7 (delta 0), reused 0 (delta 0)
remote:
 To https://AKIAJ2XDUJ2NAFFPK5SA:20130508T193045Z74f087d7af6327f3447173d8dba5de13b9c695da67033b955885c5cb4a7226b1@git.elasticbeanstalk.eu-west-1.amazonaws.com/v1/repos/44616e417070/commitid/37346533646166656662386330633434363938363361616635383863363963356164326535346466/environment/44616e4170702d737461626c652d656e76 + 04156e4...74e3daf HEAD -> master (forced update)

Clean Up, Kill It All, and Rest

To finish, ruthlessly go through, using eb delete until everything is dead; then, sit back and enjoy how little work it took to launch so much stuff.

It's kind of mad that AWS throws out stuff that feels too bare, when they could ship things like this that are wrapped up a little more. Elastic Beanstalk has been the easiest and most sensibly integrated system in AWS I've used, yet it feels tucked away. While this example launched, I had time to think about how this method is a much better way to get into AWS. Don't start with EC2. Don't start with Cloud Formation. Start with EB, because you get to experience the performance of AWS yet save yourself days spent getting things to work.