Optimizing a Centralized Approach for the Modern Distributed Data Estate

With the focus shifting to distributed data strategies, the traditional centralized approach can and should be reimagined and transformed to become a central pillar of the modern IT data estate.

Decentralized data strategies are gaining traction, due to the rise of the edge, where data is being collected in droves to fuel real-time insights and in-the-moment decision-making. IDC estimates that there will be 55.7 billion connected Internet of Things (IoT) devices by 2025, generating almost 80 billion zettabytes of data at the edge. At the same time, IDC projects, worldwide spending on edge computing will reach $176 billion this year, an increase of 14.8% over last year.

Although centralized data models and architectures, including data lakes and data-center-based warehouses and repositories, may no longer be the leading data strategy, elements of a centralized approach remain a critical part of the mix.

Specifically, the concept of having a centralized view of data and standardized data processes can help organizations bring order to the increasingly distributed IT data estate through proper governance. It can also ease data accessibility. In fact, Gartner estimates that by 2024, three-quarters of organizations will have established a centralized data and analytics (D&A) center of excellence (CoE) to support federated efforts and prevent failures.

This next manifestation of centralized data strategy emanates from past experiences with trying to coalesce the enterprise around a large-scale monolithic data lake. In many cases, this created a mostly unusable swamp. Alternatively, companies created data lakes that were centralized for a specific function such as product development but weren’t centralized to support the entire company.

More often than not, these centralized data initiatives were built on a foundation with limited governance, which impeded data’s usability; alternatively, they were designed with too many rules, preventing a broad constituency of users from deriving real business value.

“Organizations struggled with centralized data strategies due to misuse of technology that was meant to scale but was deployed in a way that few in the organization could benefit from because it became unwieldy,” says Matt Maccaux, field chief technology officer (CTO) of HPE GreenLake Cloud Services. “Today centralized and decentralized data strategies are two sides of the same coin — modern enterprises have to adopt a dual strategy.”

Reinterpreting the centralized strategy

The rise of 5G connectivity and more processing power at the edge has created new opportunities for machine learning (ML) and artificial intelligence (AI)–based workloads that thrive on a decentralized data model. Yet that momentum doesn’t negate the need for a complementary centralized data strategy, particularly as it relates to having a unified view and a set of governance policies even though data might be distributed throughout the organization.

Now when companies consider a centralized data strategy, they need to think about it from a logical perspective in which there are multiple places to retrieve information, plus a layer of intelligence and automation that allows for data discovery and different use cases, from building AI and ML models to business intelligence and reporting.

“It’s more about centralized policies, access patterns, and taking advantage of centralization through federated queries,” Maccaux explains. “It’s not about physically bringing all that data together into a centralized repository.”

One way to drive that transition is through an executive-level chief data officer (CDO) role, focused on aligning the data and analytics experience so that the organization can extract value from distributed data through effective governance policies. As part of the agenda, CDOs should lead an effort to create a data catalog that shows where data can be found and put tool sets in place that allow access to that data, preferably buoyed by automation. In addition, the CDO should be tasked with establishing policies to discover and qualify how data is used throughout the organization, with a focus on breaking down organizational and technical silos.

“It’s the role of the CDO that is going to bring this decentralized view of data into a unified view, done through policies, organizational processes, and some unifying technology,” Maccaux says.

Where HPE can help

As an edge-to-cloud strategic partner, HPE can help organizations build bridges between the decentralized and centralized data worlds. Among the ways HPE can add value are the following:

• Meet customers where they are. Whether an organization is ready to fully adopt a distributed edge and have everything delivered as a service or wants to take more programmatic steps in that direction, HPE can deliver at their pace. “We are the best at meeting our customers where they are in their journey and moving at their pace to modernize and transform wherever they are,” Maccaux says.

• Take an agnostic approach. Most organizations will have some form of legacy capabilities in their data estate, not to mention multiple clouds and disparate vendor infrastructure. HPE can connect third-party software and infrastructure into its virtual ecosystem and fully leverage that technology until it’s fully capitalized or run through its end-of-life cycle.

• Help address the talent gap. Every organization across all industries is struggling to attract and retain talent, regardless of where it is in the technology stack, whether it’s sophisticated data scientists or devops specialists. HPE experts can deliver those requisite skills, enabling internal engineering and technical talent to focus on business problems, not infrastructure. More importantly, HPE will manage the infrastructure to meet business-specified metrics.

“That is a huge differentiator: our ability to deliver an entire solution as an SLA [service-level agreement]-driven outcome,” Maccaux says. “HPE, because of our expertise and ownership of the technology, will be responsible for delivering the business outcomes the solution provides.”

For more information on what HPE has to offer, click here.