Bootstrapping custom Windows EC2 instances from stock AMIs

Lately I've been playing around with building out EC2 Windows and Linux instances from just plain stock AMIs.    It's no fun managing a separate custom AMI for each of the seven EC2 regions, and when you have two or three custom AMIs to build it's tedious and there are really better ways to spend our time.  After all, it's a pain to keep AMIs up to date, and that goes double for Windows AMIs

Windows AMIs have always been a headache though, with packaging and sysprep, and a lot of it is just tedious voodoo that I feel I shouldn't be wasting my time on.

What I'd like to do is just start with the Amazon-supplied AMIs and bootstrap a full Windows server with all my bits installed into it--and do that from issuing a single command line.   After a bit of tinkering with it last week, I finally got it going, and realized that yep, it can be done, and can be done reliably.

The Goal

A little while ago (April 2012 or so), AWS added the capability to supply userdata to a Windows instance at creation time (more here: https://forums.aws.amazon.com/thread.jspa?messageID=340466)   This is wonderful; Linux distros have had this for a long while, and it makes creating  the exact same system real easy.   Right now, I have some Puppet stuff that can do that for Linux--and Puppet supports Windows now too--so why not try and piggy back on the bit of Puppet infrastructure we have now?

I just want to be automatically create a new Windows EC2 instance, download the most current copy of Puppet installer to it, and then run the installer.  At that point Puppet can manage installation and provisioning of everything else.

The Secret Sauce

Looking around, there have been a few forum postings about attempting to use the new userdata capability towards Windows; but they've mostly been orientated towards getting PowerShell commandlets up and running.  Pass in a <powershell>[script data]</powershell> in userdata and it'll get run!  Sounds exciting.  Well... perhaps... but I've not been a fan of PowerShell: it's syntactically odd, it doesn't follow other familiar language patterns, and it has restrictive execution policies.  I never know if a particular PowerShell script will work on a given box, and to me it's just overcomplicated.

But there's another trick: AWS also allows you to pass in just plain old DOS batch commands which get executed as the local administrator.  That sounds a lot more promising. Pass in userdata like <script>ipconfig > C:\ipconfig-boot.txt</script> and it gets run.    No need to learn PowerShell for the nth time.  So maybe I can just grab our Puppet installer package from some network location and run it!

There's one big drawback though: the DOS command line doesn't have a way to download stuff from a regular HTTP location via curl or wget.   That's a small hurdle, as there's an olded technology that can overcome that: VBScript.   VBScript can do quite a bit, and you can easily script downloads in VBScript by creating a good old XMLHTTP object.  And as an added bonus, that XMLHTTP support exists on all versions of Windows back to XP, and probably back quite a bit more.   VBScripts just run.  It's the way to go.

Now the attack plan is clear:

- Via the command line create a new Windows EC2 instance, with userdata to send some DOS commands

- The new instance runs DOS commands in a Windows batch file.  This batch file writes the VBScript file.

- The last DOS command executes the newly written VBScript file

- The VBScript downloads the installer via HTTP and executes it

The result of last step, when the installer's all finished, has the effect of adding the Windows instance to our managed infrastructure.  At that point Puppet can take over the system building. 

The Script

All of this needs to be put into a script, and surrounded by <script> tags too for EC2 to see it as a script.    Here's what it looks like.

It's nothing too fancy, but it gets the job done--it starts up, connects to an HTTP server, downloads the Puppet Enterprise installer (pe.msi), and then executes it silently, pointing the Puppet agent on this new Windows instance to our Puppetmaster server.

Save this script into a file and call it win-userdata.txt. That will get used from the command line.



The Command Line


Now let's pass that userdata script when we launch a new instance from the command line.

On a machine that has the EC2 API tools installed and configured, issue a command like this:

ec2-run-instances ami-e7ce94a2 -n 1 -g default -k <your-key-name> -t m1.small -z us-west-1a -f win-userdata.txt

That AMI referenced above is the Windows 2008 default AMI for the US-West-1 region in Northern California.

After the command line is executed, you'll see the Windows EC2 instance being spun up.  The installer will be executed very quickly after boot--no need to wait 5 to 15 minutes for the Administrator password.  After about four minutes the new instance will check in with the Puppetmaster for certificate signing (which also can be automated and secured, since you know a lot of info already about the instance you created.)

Now you don't have to download that Puppet installer--that installer that's downloaded can be anything you want--but I find that Puppet makes it easier to keep things tidy and orderly and known.

With this technique, new Windows EC2 instances can be brought online and customized without any more manual intervention than issuing a single command line .  That's a huge win for speeding up deployments and testing, and for getting reliable and repeatable automation going.