(Change is inevitable – Lets plan for it)
dan.wolfson@pdr-associates.com
Introduction
As enterprises evolve, they look to support new ways of doing business and improve existing ones. They are faced with an overwhelming number of decisions on technical strategies, architectures and processes. New approaches vie for attention while tried and proven ones are sometimes denigrated as old-fashioned.
The IT landscape churns with shooting stars, navigating data oceans and the ripping sounds of fabrics stretched too thin. However, there is one constant that we believe should be the guiding light leading the way through these choices:
Anticipating and accommodating change
Change is truly the only thing we can count on – and it often comes in unexpected ways – economic trends, new regulations, changing organizations, and of course, the latest technology fads.
Change comes in many flavors each with their own dimensions of scope, risks, opportunities, timeframes, and costs.
By recognizing that change is fundamental, organizations can consider the impact of change on different technical strategies and architectures – asking simple questions about what kinds of downstream changes become easier or harder with a particular approach.
For example:
- Is it as cost- effective to get my data out of a system as it is to put it in?
- Is there a “right place” to put my data?
- Are open interfaces available that enable integration and automation with other runtimes and tools?
- In what ways can we evolve the system to meet changing needs?
- Can we incorporate the system into broader (governance) initiatives?
- Can the size and scope evolve as needs evolve?
- How specialized are the skills needed to create, maintain and evolve the technology?
- How do costs and performance change with scale-out?
These can be hard, sometimes uncomfortable questions to answer. They force us to address what appear to be trade-offs between expediency and sustainability.
There are, however, strategies that we can employ that can contribute to both speed of delivery and evolvability. It begins with some basic assumptions:
- Assume change will happen – some changes can be anticipated and even planned for, others will be a complete surprise.
- Recognize that costs matter – both in terms of people as well as other resources.
- Assume that you can collaborate with other teams – sharing of knowledge and skills is the key to rapid change.
- Teams operate best when they can influence their own destiny.
- And of course … you can’t change the speed of light. Do not believe in miracles, silver bullets and magic wands. The use of the term “just do xxx” should be a red flag.
These assumptions were foundational in the conception and creation of Egeria – an open source project from the Linux Foundation AI & Data organization that provides open metadata and governance for enterprises – automatically capturing, managing and exchanging metadata between tools and platforms, no matter the vendor.
How Egeria helps you accommodate change?
The Egeria Project was created to help organizations accommodate and embrace changes in their digital eco-systems. Egeria has many technical capabilities that are used by teams and enterprises in different kinds of digital transformations.
Egeria:
- Helps virtually consolidate business data from across multiple subsidiaries.
- Makes it easier to find and use appropriate data.
- Manages and controls visibility of digital assets within an enterprise.
- Simplifies the development of new applications through (re)use of shared assets and vocabularies.
- Supports the roll-out of governance programs to enable cross-cutting concerns such as data privacy, AI fairness, and Sustainability.
Just as importantly, Egeria was designed and built to knit together tools and data from different vendors, sources, and technologies. It enables transparent exchange of metadata between different tools and supports governance processes that can be coordinated across organizations and teams. Egeria itself is entirely open source which means that organizations can tailor it for their specific requirements.
An Egeria topology can fluidly evolve as business, organizational or technical changes take place. Its design is based on decades of customer experience – Egeria has been architected for ongoing evolution with well-defined interfaces, modularity, and extensibility.
Everything from the open metadata type system to the storage layer, to the communications subsystem can be altered to support the needs of an organization.
Perhaps most importantly, Egeria can be deployed and used incrementally – an initial deployment for one project can be configured to support more projects as they are interested. This promotes the accumulation from subject matter experts and the sharing of knowledge across communities of interest.
Communities can continue to use the tools they are familiar with while benefiting from integration and collaboration with others – sharing (meta)data can be selective and managed when it needs to be – and more open and exploratory when policy allows. This allows both teams and the Data Officers to navigate the tensions between privacy, protection, and data availability for (re)use and exploration.
A governed data environment can provide data owners and stewards the confidence to share their data with other teams in a managed way – and give consuming teams the confidence that they can trust the data they receive. Building this mutual trust between data producers and consumers can help teams recognize and adapt in a dynamic environment.
Moving Forward
Egeria provides a powerful technology base to support evolving data landscapes. However, we recognize that technology is just one key ingredient. Planning, designing, and delivering new capabilities and systems to support change requires a combination of:
- People – education, collaboration, organization
- Processes – business and technical process evolution
- Technology – augment what works and integrate
Our team at Pragmatic Data Research Ltd leads and supports the Egeria community, and is working to broaden Open Data ecosystems around data and analytics. We bring decades of pragmatic experience and a deep interest in helping organizations succeed in their digital evolution.
Choosing where to start
Choosing where to start can itself be challenging – often there are so many deserving projects that can show-case an Egeria Evolution that selecting one can be hard. Here are a few thoughts on what makes a good showcase project:
- It is good to focus on a project with immediate value that may be difficult with current approaches.
- A relatively narrowly scope helps to demonstrate rapid success
- If the results can be foundational to downstream projects, then future investment can build on the initial success.
Our goal is to consistently deliver business value as we build knowledge, integrate teams, tools, and communities. It is an evolutionary approach.
We are interested in your thoughts and observations so please fill out the form or send us an email at info@pdr-asssociates.com