Open-Sourcing ThoughtWorks Go

Photo of Martin Fowler

Martin Fowler is an author, speaker, and general loud-mouth on software development. He is the Chief Scientist of ThoughtWorks.

25 March 2014

Chad Wathington: Managing Director of ThoughtWorks Studios

Go is a product that ThoughtWorks Studios (the product group of ThoughtWorks) has built over the last 6 years to support continuous delivery. It helps create and manage automated deployment pipelines. It supports automating the entire build-test-release process from check-in to deployment.

ThoughtWorks just announced that we are open sourcing Go — it is now freely available under a BSD-style license, and the source will be available soon as well. To explain the background to this, and answer some common questions, I interviewed Chad Wathington, who is a Managing Director of ThoughtWorks Studios.

Martin: Why does the world need another continuous integration server?

Chad: Go isn’t a continuous integration server. It’s meant for continuous delivery (CD), of which continuous integration is a part. Go supports easy modeling of sequential and parallel execution, fan-in and fan-out dependency management, and other activities unique to deployment pipelines.

Isn't this something that can be done with scripting CI servers or using plugins?

You can cobble together anything, but sometimes you shouldn’t. Deployment pipelines should be a first class concept in your CD tools to avoid headaches. I’ve seen many teams, including some at ThoughtWorks, wasting substantial time and resources implementing pipelines with CI tools. There are basic things lacking, like clear traceability back to source if you have more than a simple workflow.

Does open sourcing Go mean it failed as a commercial product?

No, but we weren't meeting our goals with it as a commercial product. ThoughtWorks is a three pillar organization. One of those pillars is to champion software excellence and revolutionize IT. Another is to run a sustainable business. It’s hard to advocate a big switch in behavior, charge a lot, and compete against free CI cobbled together solutions. (We’re a big fan of OSS CI servers too; we’ve OSS’ed three successful ones.) In other words, we weren’t making enough financially to justify no revolution. Think of it as balancing sliders that have clear minimums.

We also have come to believe that infrastructure software needs to be OSS. Almost everything in the ecosystem that Go spans is OSS, from source control and build, to deployment and configuration management. These kinds of tools need community. While OSS business models are tougher in some ways, we’re willing to forgo the revenue if we can help make continuous delivery mainstream. Roy, our founder, is a big believer in creating public goods.

Why is the source not available yet?

There’s an interesting domino effect that happens once you decide to make something freely available. You can’t keep charging for it the same way - that’s unethical. As a response, you offer prospects and customers heavy discounts or tell them not to pay right now. That breeds suspicion, and surprisingly people insist on paying you. As you calm fears, you eventually break down and tell individuals you’re open sourcing it. Once you’ve gone through that scenario many times, you have to move ahead with a broad announcement. You’re announcing it anyway. Google pre-announced Android as OSS and HP did the same for WebOS. My guess is that they both were trying to avoid leaked non-announcements in a similar way.

The code isn't ready yet. We’re working hard to make the developer experience great. We have stuff to do that you’d expect, like cleaning up language in the code comments and removing the old licensing. But, there are other complexities. We use Go to build, test, and release Go on a large grid of VMs across two data centers. We need to create a public build infrastructure that’s simpler. We’re doing that on OpenStack. Our large suite of automated functional tests for Go are written in Twist, which isn’t open source, so we have to work around it. There’s a lot to do.

Why call it Go when there is golang?

Go was originally named Cruise, as homage to CruiseControl. But, the similar name choice created confusion, so in July 2010 we renamed it Go. That’s a little after Google publicly unveiled golang, but the language was by no means popular then.

Isn't this a good opportunity to change the name?

Yes. Given all the work to get the source available, we decided not to accept further delay by bothering with the name at first. However, the name will be up to the community. Once the code is out, if the community wants to change the name, we’re all for it.

What's the connection between Go and Cruise Control?

ThoughtWorks created the first Continuous Integration server, CruiseControl as an OSS project in 2001. In 2007, we started an effort to modernize it, but we realized that CruiseControl’s architecture wouldn’t allow for the grid we wanted and first class deployment pipelines. Currently, CruiseControl and Go don't share anything in common.

What support are you putting into Go in the future?

The current Go team isn’t going anywhere. In fact, if the project has a thriving community, we’ll add more people to it. ThoughtWorks is also going to provide commercial add-ins and support.

You’ve recently released another tool called “Snap”, what’s the difference between them?

Snap is our hosted CD as a service. It’s meant for applications with simpler deployment needs. We built Go to solve hairy complex CD setups. We designed Snap to make CD itself simple for teams and organizations deploying to the usual cloud suspects. In other words, whereas Go‘s feature breadth centers on complex scenarios, Snap’s feature depth focuses on using convention to make CD easier.

What features do you expect to add?

There’s quite a bit we’d like to do, the order of which will be determined in large part by the community. We made the conscious decision to not have a “product manager” role on the team in favor of a more community driven process.

Adding more extension points is very high on the list so that people can extend Go more easily. There are some really cool ideas around making the value stream map (vsm) have more value stream related features in addition to the visualization. Many users have asked for features that make it easier to get started. We’d like to move the authentication for Go out to a plugin so that people could create their own.

There will be a much better list of these in the very near future on GitHub issues where everyone can comment and help them land in an order.