Journey to a software-defined access fabric network, part 2 – Cisco Blogs

In the first installment of this two-part blog series, I described Cisco IT’s motivations for shifting to a software-defined access fabric network – part of our ongoing journey to an advanced network architecture. In this blog, I will explain our guiding principles for implementing Cisco Software-Defined Access (SD-Access), along with lessons learned and outcomes achieved thus far.

In Cisco IT’s Customer Zero organization, we partner with various internal business entities to deploy the company’s latest solutions in order to prove quality, value, and scale. As part of Cisco IT’s  overall network strategy aimed at supporting greater business agility, resilience, and security, we prioritized deploying Cisco SD-Access on the campus. And like most initiatives, our SD-Access deployment required an equal focus on two key areas: process and technology.

Let’s take a closer look at how we blended these vital elements to implement Cisco SD-Access across six buildings on Cisco’s San Jose campus.

As our Cisco SD-Access implementation process unfolded, we became increasingly aware of the importance of adopting an agile, “NetDevOps” mindset. DevOps is a software-development process that aims to provide fast, reliable, relevant software lifecycles. When this concept is applied to network engineering, and we call the process NetDevOps.

NetDevOps is all about starting small, and then building, iterating, and improving processes and results along the way. With NetDevOps, the journey truly is the reward. As such, we wanted to create an environment that would allow us to learn as we went.

Our first technology step was to build out a new SD-Access fabric core environment. Because the core was a parallel infrastructure, we didn’t have to impact existing services to build it. Instead, it provided an additional environment on which we could work much more iteratively as we migrated services from the old platform to the new (see illustration below).

With our fabric core in place, we moved to converting our first building. As expected, we encountered many process or implementation inefficiencies in our first conversion, but it provided the experience needed to improve.

We quickly learned two critical lessons:

Lesson No. 1: The  value of segmenting and orchestrating work across a team. Because fabric conversions have many moving pieces, it’s essential to develop a clear model that allows team members to work in parallel, with tasks synchronized and completed in the proper order. The work should be equally distributed across a team of  engineers (rather than by a solo engineer), with careful thought given to orchestrating that work.

Our collective experiences across the first building migrations allowed us to develop an Orchestration Guide that became the cornerstone of the NetDevOps process for SD-Access. The Guide detailed the approximately 200 steps needed to onboard each Cisco building to its respective fabric. It broke each process step into clear, “bite-sized,” actionable tasks scheduled to occur before, during, and after the onboarding change window. We appointed one engineer as the migration leader (the “Orchestrator”) to make sure tasks were accomplished in the correct order.

Lesson No. 2: The importance of creating strong communication channels with stakeholders. Networks are all about meeting clients’ service expectations. Because clients are incredibly diverse (for example, security, facilities, print services), their expectations vary widely. As a result, we needed a robust communication channel that would allow us to interact with each stakeholder individually, in near real time. By adopting the Webex application, we were able to communicate with stakeholders asynchronously, saving dozens of hours of engineers’ time (vs. conference calls).

When it came time to implement SD-Access in our third building, we had refined our processes to the point that we could begin to drive value through automation. This step included leveraging automation capabilities already built into Cisco DNA Center (LAN Automation), as well as developing small, customized Python scripts for tasks not handled by Cisco DNA Center. As time passed, we began to derive more value from automation. Instead of solving issues the old way, with spreadsheets and computer keyboards, we decided to let automation carry more of the load.

Our process accelerated through repetition, with innovations (such as scripting) added in small increments. Engineers rotated through the tasks, increasing team knowledge and removing the hesitancy that sometimes comes with adopting a new solution.

Initially, some engineers were uncomfortable with our NetDevOps approach – they wondered why they simply couldn’t follow a traditional, preplanned set of steps to complete their work. Eventually, however, they began to see the value of following an agile process designed to improve results “as you go.”

An integral piece of our NetDevOps process was the retrospective meeting held following each building conversion. After giving the engineers a day to collect their thoughts, we simply asked them, “how do you think we can get better next time?”

As we honed our processes, performance steadily improved (see chart below). Our initial production deployments took 13 hours. After bugs and process issues identified by Customer Zero were fixed, subsequent deployments required only about eight hours. In addition, we also decreased the time between migrations from about 30 days to seven days.

We began operating as a true NetDevOps team by the middle of this timeline – an indication that we had become agile and fast, without sacrificing quality.

Our six-building deployment showed that, compared with a traditional, manual network approach, Cisco SD-Access full fabric offers other significant time savings and quality improvements.

Note: A related blog explains how we established a framework for measuring value for SD-Access, along with specific results achieved to date.

Learn more about our journey to an advanced network
architecture by clicking through our interactive journey map

Follow Cisco IT on social!

Twitter
Facebook
YouTube

For more information: 

Journey to a software-defined access fabric network, part 1

Measuring the impact of a software-defined access fabric network

Cisco DNA Assurance and SD-Access Solutions Provide Optimization for Enterprise Networks and Applications

Share: