Share with Your Network
An old colleague of mine likened managing engineers to herding cats. I’m not so sure about herding engineers, but managing development environments quickly becomes an exercise in organizing chaos. Every engineer has different needs and a different way of doing things.
During a recent data-center migration we moved all of our development environments to the AWS Cloud. Since it turned out to be one of the more novel mini-projects for us, we thought we’d share the outcomes.
Why did we do it?
Before we moved development to the cloud we had a pretty reasonable setup using vagrant and chef-solo to manage virtual machines locally. This worked really well for us when our application was relatively small, and we had fewer back-end processes. But, we felt we’d outgrown that solution.
- Running the virtual machine with a large amount of data, ie what we have in our demo account for production, required a LOT of RAM on our poor laptops.
- One of our front-end engineers literally lives and works from his camper. Getting large data changes or updated software into his virtual machine over a tin-can and string internet connection could be an exercise in madness.
- Local development is great when you share an office and can huddle around a laptop to show changes to your product folks. But, with a semi-distributed team that can be a challenge, so we wanted to fix that.
How we did it
We already used chef to manage all our production infrastructure, so that made it relatively easy to create miniature production environments running on individual cloud servers. There are definitely different requirements between the two environments, so we had to:
- Create a chef role to run all our production processes on a self-contained host.
- Configure the application to look to localhost for everything.
- Make everything smaller. Way smaller. Lower the memory and other requirements as development should have less data and fewer requests than production.
- Sprinkle on some software and scripting to make development easier. Editors, debugging tools, a way to sync code to local laptops for the folks on GUI Editors (You know who you are).
What’s awesome about it?
Lower maintenance and ease of collaboration are definitely the big overall wins. But here’s a list:
- Sharing our current work is really, really easy. Anyone with VPN access (two factor naturally!) can see what’s going on with any development environment with their normal web browser.
- Helping someone fix a problem in their local VM can be a challenge if you can’t see what’s going on. With our instances in the cloud it’s just a matter of hopping on their instance and poking around.
- Multiple development instances! We don’t do it often, but if someone is working on something really esoteric, or long lived we can launch another development instance for them and have it running concurrently with their normal instance.
- Work from anywhere. This is especially true for our Emacs and Vim geeks, but it takes very little to connect and start working from a new machine.
- On-boarding. It still takes a while to initially build a development instance, but if we a new hire starting she’ll have a development environment before she’s opened a browser on their new laptop. Find out for yourself!
Possible pitfalls
Things have worked out remarkably well for us so far, but for anyone else interested there are definitely some things to watch out for.
- We’re really, really network dependent. Our old setup was pretty reliant on network access so this wasn’t a huge leap for us. Thankfully if all else fails we have tethering.
- GUI-based editor access requires some magic. It took some trial and error to determine which syncing software worked best for our folks.
- Cost management requires some thought. We make use of reserved instances to keep our costs low, but there are probably other valid strategies as well.
- This hasn’t come up yet, but we’ve traded downtime from broken laptops to downtime from our hosting provider.
Would we do it again?
Overall, I think everyone is happy with how things turned out. We had a great deal invested in our chef infrastructure that made this a relatively easy addition. Now, our development environments operate more similarly to production than ever, we’re more able to share new work and issues internally, and we can run as many instances as we need at any time. The flying felines seem to have been adequately tamed and enjoying life in their cumulus castle. For now.
If we were to tackle something similar today, we might look at some interesting musings coming out around Docker for inspiration.
