How a Linux migration led to the creation of Amazon Web Services

Dan Rose, chairman of Coatue Ventures and Coatue Growth, posted a thread on Twitter the other day, 280 characters or less at a time, in which he chronicled how it came about that AWS infrastructure is built on Linux.

Rose was at Amazon from 1999 to 2006, where he managed retail divisions and helped incubate the Kindle reader before moving to Facebook. So he was at Amazon in 2000 when the internet bubble popped,and one high-flying dot-com after another was shriveling up and dying, having burned through ridiculous amounts of capital on luxurious offices while often having nothing by way of a product to show for it.

Rose said Amazon’s biggest expense was the data center outfitted with expensive Sun servers. Amazon’s motto was “get big fast,” and site stability was critical. Every second of downtime meant lost sales, and Sun was the gold standard for internet servers back then. I can recall them having a significant software business led by a VP named Eric Schmidt.

Sun’s proprietary stack was “expensive & sticky,” as Rose put it, and it was designed that way. Back then the Unix market was Sun, HP, IBM, and SGI, and they all had variants of Unix operating systems that were designed to be less than portable.

But it is safe to say the early internet was built on Sun—and not Solaris, either. Sun had a second OS, a BSD derivative called SunOS that had no GUI. (In the pre-DNS days while in college, I cut my teeth on SunOS 3.x and had to learn Unix command line. My first ISP out of college dropped you into a SunOS shell, and if you didn’t know what to do staring at the % prompt you were lost.)

As startups died in 2000 and liquidated their data-center gear, brand-new Sun servers started appearing on eBay for 10 cents on the dollar. As a result, Sun sales took a big hit, which marked the beginning of the end for Sun.

Amazon could have used that predicament to negotiate a better deal with Sun, Rose posted, but Jeff Bezos chose a more radical approach. Amazon’s CTO Rick Dalzell pivoted the entire engineering organization to replace Sun with HP/Linux. Today no one would blink at such a move, but in 2000 it was relatively daring. After all, Linux 1.0.0, the first production-ready version, had only been released in 1994.

“Six years later we were betting the company on it, a novel and risky approach at the time,” said Rose. Product development ground to a halt during the transition. The company froze all new features for over a year as it faced a huge backlog, but nothing new could ship until they completed the shift to Linux.

During this time period revenue slowed. The dot-bomb implosion was continuing, the economy was sliding into recession in 2001, and then 9/11 happened. Amazon came within a few quarters of going bankrupt, Rose said.

“But once we started the transition to Linux, there was no going back. All hands on deck refactoring our code base, replacing servers, preparing for the cutover. If it worked, infra costs would go down by 80%+. If it failed, the website would fall over and the company would die,” he wrote.

When it completed the transition, the site chugged on with no disruption. “Capex was massively reduced overnight. And we suddenly had an infinitely scalable infrastructure,” said Rose. “Then something even more interesting happened. As a retailer we had always faced huge seasonality, with traffic and revenue surging every Nov/Dec. Jeff started to think—we have all this excess server capacity for 46 weeks/year, why not rent it out to other companies?”

Around the same time, Bezos was also interested in decoupling internal dependencies so teams could build without being gated by other teams. The architectural changes required to enable this loosely coupled model became the API primitives for AWS. Bezos took an interesting view of things.

“He framed the idea in the context of the electric grid. In 1900, a business had to build its own generator to open a shop. Why should a business in 2000 have to build its own data center?” Rose wrote.

He acknowledged that cloud infrastructure would have emerged eventually even without AWS, but how much later and at what opportunity cost? With the availability of AWS infrastructure as a service, the cost of rolling out a startup was reduced dramatically, innovation exploded and the modern venture capital ecosystem was born, he said.