In this article, development environments refer to sandboxes where you can test your code changes before deploying, and should not be confused with integrated development environments (IDEs) like Eclipse or Microsoft Visual Studio.
Dev environments have always been a mystery to me. Despite learning about them on my first day at Slack, and using them almost every day for the last three years, I have never understood how they truly worked.
Half a year ago, I set a goal to understand dev environments inside and out. I interviewed some of the most senior engineers at Slack, studied countless pages of documentation, and sifted through years of Slack conversations. What I discovered was a fascinating journey of how our dev environments evolved over time.
Why do we need dev environments?
Dev environments are copies of our application that we can modify freely. Their main value is allowing us to test changes to our application without impacting real users or putting real data at risk.
They enable us to iterate rapidly because it’s quick and easy to test changes on them. They also make it possible for us to easily share our changes with others for review.
Altogether, dev environments drastically increase development speed and safety.
What lies under the hood?
Slack’s dev environments are copies of our application that live on remote servers — Amazon EC2 instances to be exact. These instances are configured to run the Slack application and the many services it depends on.
Each dev environment hosts its own Slack subdomain, which we can navigate to on our browser to see the changes we make.
No changes in dev environments can impact real users because they use their own set of infrastructure (e.g. databases) isolated from production.
Developing remotely vs. locally
At Slack, we develop remotely, meaning our dev environments live on servers. Another option is developing locally on our personal computers. Local development is great for smaller projects because it’s fast and doesn’t require an internet connection. However, for larger projects, remote dev environments offer significant advantages.
First, we don’t have to set up the Slack application locally. Given that Slack has a very complex architecture that depends on many different services, not having to set things up locally is immensely valuable.
Second, if a change works in dev, it’ll most likely work in production, because our dev environments are configured to mirror production. Some level of drift may still happen with especially long-lived dev environments, but the likelihood and magnitude are much smaller than when developing locally with unique machines that often end up with inconsistent configurations.
Third, remote dev environments don’t rely on a personal computer, which may crash or lag. Cloud hardware is much more affordable, resilient, and scalable. Further, they allow us to easily develop on multiple machines and share our work with teammates for review.
As the internet becomes increasingly faster and more reliable, it makes more sense to develop remotely.
How we use dev environments at Slack
The best way to illustrate Slack’s dev environment workflow is with an example.
Let’s say for some reason we wanted to test a version of Slack’s homepage with all-caps, purple, Comic Sans text.
We first create a feature branch, and then attach it to a dev environment using a command line tool called
slack sync-dev. It reserves a random dev environment then syncs our local changes to it, so whatever local edits we save automatically transfer to the dev environment.
At its core,
slack sync-dev is simply a wrapper around two well-known utilities —
fswatch (detects changes) and
rsync (transfers changes).
If we make any frontend changes, we have to build them locally using webpack, an open source tool we adopted as our build system. The command
slack run buildy:watch builds our frontend assets and serves them to our dev environment over localhost.
When we’re done, we can navigate to dev575’s subdomain, and voila! Behold our violet masterpiece.