Zen and the art of platform engineering

Achieving and maintaining any sense of Zen requires a commitment to keeping everything in balance. Too much of any one thing inevitably creates levels of friction that, over time, become unsustainable. Within a DevOps context, the current manifestation of Zen is organizations embracing platform engineering methodologies that enable them to standardize around a common set of tools and practices, all while empowering – without burdening – their developers.

Platform engineering, at its core, is a methodology for enabling DevOps at scale without sacrificing innovation. In this digital era, as the number of applications continues to increase, there is a critical need to rethink the building and deployment of applications so that it becomes more efficient for all involved parties, developers, cybersecurity, compliance, and IT operations teams alike.

DevOps’ goal has always been to increase the drumbeat at which organizations get to continuously deliver business value through software, by getting all IT functions to smoothly collaborate. But as the adoption of DevOps grew in organizations, so did the complexity of the world they live in: from highly powerful but complex cloud infrastructure to increasingly sophisticated security attacks, and constantly evolving and cumulative compliance requirements. In many organizations, this has started to feel like an untenable situation.

Now is the time to step back and define a unified approach that strikes a delicate balance between efficiency and the freedom to innovate that so many developers prize, all while making sure organizations are able to standardize their tools and processes, as well as trust their overall security and compliance posture. As the protagonist in the novel “Zen and the Art of Motorcycle Maintenance” discovers, not every issue can be rationally addressed when people who invariably value and cherish their own insights and ideas are involved.

Platform engineering: A brief history

Leaders intuitively understand that, for better or worse, every action creates an equal and opposite reaction. As DevOps adoption grew in organizations, it became very clear that one of its most central constructs, continuous integration and continuous delivery, that is, pipelines that fully automate the lifecycle of software development and delivery, led to huge gains in development velocity: developers were able to focus on creating business value through their code, and, for any new piece of code contributed, a pipeline would promptly execute and give them quasi-instant feedback if anything wasn’t OK. Witnessing this success, an increasing number of new “missions” of other functions were forced into those pipelines, from security to compliance and a lot more, shifting left towards developers the burden to instantly react when things go wrong – without them necessarily equipped to understand the reason for the failures.

In parallel, the world developers lived in increasingly got more complex, with powerful but complex cloud native infrastructure, open-source projects, and practices constantly emerging and being adopted. This led to an overflow of cognitive load, and, ultimately, an inability for developers to truly deliver on their initial mission: coding business value! A new balance had to be redefined, one that allowed organizations to standardize their tools and practices, keep a strong security and compliance posture, leverage modern and constantly evolving cloud technologies, all while giving back velocity (and happiness!) to developers.