Data Management Strategy at Microsoft E-Book

Aleksejs Plotnikovs

• Herausgeber: Packt Publishing
• Kategorie: Lebensstil
• Sprache: Englisch

Seitenzahl: 510

Veröffentlichungsjahr: 2024

Beschreibung

Microsoft pioneered data innovation and investment ahead of many in the industry, setting a remarkable standard for data maturity. Written by a data leader with over 15 years of experience following Microsoft’s data journey, this book delves into every crucial aspect of this journey, including change management, aligning with business needs, enhancing data value, and cultivating a data-driven culture.
This book emphasizes that success in a data-driven enterprise goes beyond relying solely on modern technology and highlights the importance of prioritizing genuine business needs to propel necessary modernizations through change management practices. You’ll see how data-driven innovation does not solely reside within central IT engineering teams but also among the data's business owners who rely on data daily for their operational needs. This guide empower these professionals with clean, easily discoverable, and business-ready data, marking a significant breakthrough in how data is perceived and utilized throughout an enterprise. You’ll also discover advanced techniques to nurture the value of data as unique intellectual property, and differentiate your organization with the power of data.
Its storytelling approach and summary of essential insights at the end of each chapter make this book invaluable for business and data leaders to advocate for crucial data investments.

Leseprobe

Data Management Strategy at Microsoft

Best practices from a tech giant’s decade-long data transformation journey

Aleksejs Plotnikovs

Data Management Strategy at Microsoft

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

The author acknowledges the use of cutting-edge AI, such as ChatGPT, with the sole aim of enhancing the language and clarity within the book, thereby ensuring a smooth reading experience for readers. It's important to note that the content itself has been crafted by the author and edited by a professional publishing team.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Associate Group Product Manager: Niranjan Naikwadi

Associate Publishing Product Manager: Sanjana Gupta

Book Project Manager: Hemangi Lotlikar

Senior Editor: Tiksha Lad

Technical Editor: Kavyashree K S

Copy Editor: Safis Editing

Proofreader: Tiksha Lad

Indexer: Tejal Daruwale Soni

Production Designer: Shankar Kalbhor

Senior DevRel Marketing Coordinator: Vinishka Kalra

First published: June 2024

Production reference: 1110624

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK.

ISBN 978-1-83546-918-7

www.packtpub.com

To my beloved son, Sasha, and my wife, Tatiana, for inspiring me to write my first book, and for the amazing support and care they provided while writing it, fueling it with fun, laughter, and success, and surrounding me with love.

– Aleksejs Plotnikovs

Contributors

About the author

Aleksejs Plotnikovs has been leading global data delivery at Microsoft for more than 15 years, navigating from the early stages of the establishment of data management practices up to the modern data platform strategy. He led diverse data office teams and helped to build market-leading products fueled by data and AI capabilities, emphasizing data value and data monetization, while always emphasizing the enormous potential of data to address emerging business opportunities. A passionate data professional and an active ambassador of data culture, he actively contributes to the broader data community by coaching, consulting, running data master classes, and inspiring fellow professionals on LinkedIn.

I want to sincerely thank all our team members who have been close to me and supported me on this book-writing journey! Special and massive thanks to Karthik Ravindran for his exceptional contribution and leadership, and to Wilhelm Klepsch, Chang Wee Loh, Antonina Steshenko, and Justin Tomboulian who tremendously helped with initial content, critical reviews, and motivation!

About the reviewer

Chang Wee Loh’s career in IT spans 27 years across IBM, SAP, and Microsoft, where he has worked on various aspects of data as a discipline, from analysis and services to governance. He is based in Singapore.

Preface

Microsoft pioneered data innovation and investments into the state of data ahead of many in the industry, leading to a distinctive and noteworthy advancement in data maturity. The book comprehensively explores every crucial facet of this journey and emphasizes that the wellspring of data-driven innovation is most often found with the data’s business owners—the individuals who depend on data daily for their operational needs. Empowering these professionals with clean, easily discoverable, and business-ready data marks a significant breakthrough in how data is perceived and utilized throughout an enterprise.

This book unveils the comprehensive journey of successful end-to-end data enablement—beyond mere technology, it encompasses change management, alignment with business needs, enhancing data value, and fostering a data-driven culture. The book guides you along the path of maturity and offers insights into where to channel efforts most effectively. It also establishes a connection with your organization’s own data enablement narrative. While many believe they face unique challenges, the reality is that data challenges tend to be strikingly similar across businesses. This enables the direct application of existing best practices. Such insights prove invaluable for business and data leaders, aiding them in advocating for essential data investments and competitive data and AI strategies.

This book adopts a storytelling approach to ensure a seamless reading journey, with essential insights summarized at the close of each chapter.

Moreover, the book is written by a data leader and practitioner with over 15 years of data experience at Microsoft, who has seen the company through numerous transformations and the reinvention of business and data processes to support ever-evolving organizational needs. This book covers how to foster the team spirit and one company approach that’s needed to drive data and business outcomes, emphasizing the power of having a data-driven culture.

Who this book is for

This book is aimed at all data leaders, data practitioners, data engineers, data scientists, and data enthusiasts, along with modern business leaders who are interested in the power of data. From data stewards and data management experts to data governance professionals, it will be equally relevant to top leaders looking to capitalize on the rise of data and AI and enable digital transformation. This book reflects Microsoft’s journey in transitioning into a data-driven enterprise and ultimately becoming a leader in the data and AI space. Unveiling an untold real-life, decade-long journey, this book takes a storytelling approach for a seamless reading experience and, unlike other tech-heavy books, is equally positioned for both experienced and inexperienced readers.

What this book covers

Chapter 1, Where’s My Data and Who’s in Charge?, addresses a common challenge for any enterprise: gaining clarity about its own data, how it is managed and stored, who is responsible for it, and how to make data discoverable.

Chapter 2, We Make Data Business Ready, is where we look at creating a data office, which involves appointing a dedicated data team to take care of enterprise data, make it highly qualitative and business-ready, and enable the core functions of data management.

Chapter 3, Thousands to One – From Locally Siloed to Globally Centralized, looks at how to simplify and streamline data usage and discoverability across an enterprise by executing an end-to-end inventory of data domains and connecting previously disparate data with respective business processes.

Chapter 4, Reactive! Proactive? Predictive, explores the maturity curve for various data services and data management operations – from the reactive and immediate approach for business issues to having a proactive catalog of services and predictive in-flight data corrections.

Chapter 5, Mastering Your Data Domains and Business Ownership, helps you to understand the various data domains across an enterprise and how they could be effectively combined and interrelated, as well as properly owned and responsibly consumed by both data and business teams.

Chapter 6, Navigating the Strategic Data Dilemma, navigates the golden balance between the scalability and cost efficiency of outsourcing work with the natural flexibility of working with in-house data teams, covering the various pitfalls and benefits.

Chapter 7, Unique Data IP Is Your Magic, explores how to develop and advance your company’s data know-how and how data supports the business processes, which is often referred to as the data’s Intellectual Potential (IP). We look at effective ways to protect your data IP and use it for continuous business success.

Chapter 8, The Pareto Principle in Action, covers the famous 80/20 principle, also known as the Pareto principle. It can inspire highly effective frameworks and methodologies and was used to advance the state of data at Microsoft, helping to build innovative, multi-billion-dollar-revenue data products.

Chapter 9, Data Mastering and MDM, covers Master Data Management (MDM). MDM is undoubtedly the cornerstone of modern data management and data governance, but ultimate success with it depends on the underlying business cases and implementation strategy. We will learn about this in this chapter.

Chapter 10,Data Mesh and Data Governance, looks at the Data Mesh approach, which was a much-needed innovation in the recent data architecture space, enabling a federated, de-centralized approach to data governance, ownership, and management. A significant part of this impressive win is attributed to business enablement through change management.

Chapter 11, Data Assets or Data Products?, looks at having a data-product-focused strategy, which is another great example of non-stop innovation with data. To truly unleash the power of product thinking, the relationship between raw data assets and data products must be understood. This chapter helps with that.

Chapter 12, Data Value, Literacy, and Culture, answers questions such as, What makes data distinctively valuable? How can we monetize data and drive immense revenue increases and business growth using existing data? These questions are tackled through the lens of data literacy and data-driven culture.

Chapter 13, Getting Ready for GenAI, talks about how data is everything in AI, as it plays a critical strategic role in any successful AI deployment. Unpacking this notion leads us to identify specific prerequisites for succeeding in a business transformation with data and AI.

Download the example code files

A referral link to Github has been included for any updates or erratas in future https://github.com/PacktPublishing/Data-Management-Strategy-at-Microsoft. If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Conventions used

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Data Management Strategy at Microsoft, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781835469187

Submit your proof of purchaseThat’s it! We’ll send your free PDF and other benefits to your email directly

Part 1:Thinking Local, Acting Global

This part takes us through the foundational aspects of data delivery – from setting up a data team and data office to finding an immediate fit for emerging business needs with data while completing a comprehensive inventory of existing business and data processes. A set of highly useful, universal data services available in a well-defined catalog completes the picture of addressing the most urgent demand for data.

This part includes the following chapters:

Chapter 1, Where’s My Data and Who’s in Charge?Chapter 2, We Make Data Business ReadyChapter 3, Thousands to One – From Locally Siloed to Globally CentralizedChapter 4, Reactive! Proactive? Predictive

1

Where’s My Data and Who’s in Charge?

This first chapter delves into common business challenges stemming from the necessity of maintaining data quality and effective data management within a company. Often, there is either a complete absence of these practices or, at the very least, a partial implementation. This challenge typically impacts the end users of data the most, including sales, marketing, pre-sales, go-to-market, and finance teams. Meanwhile, the IT team tends to lag behind and might not even fully comprehend the issue until they are confronted with urgent user concerns.

This was notably the case for Microsoft in its early years of data enablement. The IT division had limited exposure to data-related problems as its focus was primarily on engineering and developing long-term solutions within a specific ecosystem. Consequently, there was no single global team responsible for overseeing data, ensuring data quality, and managing data across key Tier 1 Line of Business (LOB) applications.

On the flip side, the sales, marketing, and other teams grappling with data quality challenges had to implement rapid fixes and day-to-day strategies to navigate these issues, striving to achieve success in their core responsibilities.

In this chapter, we’re going to cover the following main topics, addressing the most popular and simple questions from the users of data. You will learn how to position data challenges alongside easy-to-implement investments that immediately will help to improve the “state of data”.

Where’s my data? – a common question from users of any enterpriseWho’s in charge of owning the data and fixing issues?How do we fix the data quickly and efficiently?

The journey begins

As I settled into my seat on the plane bound for Prague, Czech Republic, I couldn’t contain my excitement. This journey was unlike any other; it was the first time I would meet the individuals who represented the virtual data organization within our company face to face. Previously, our interactions had been limited to countless email exchanges and phone calls.

These individuals came from diverse backgrounds and hailed from various countries in Central and Eastern Europe. We were all part of something in its infancy, something that held the promise of significant transformation. The prospect of meeting them in person and learning from their experiences filled me with a lot of anticipation!

At the same time, I found myself wondering what we could offer them to enhance their daily work and address the unique challenges they faced. Our shared mission was clear: to improve data quality within the company. However, our path to achieving this goal was far from straightforward.

The team members brought with them a wealth of local data practices, each shaped by their specific regions. These practices often stood in contrast to the global standards we aimed to establish. Despite our business systems being classified as Tier 1 applications (the most critical) and operating on a global scale, there was a noticeable disconnect. It became evident that the technical data flow between systems was far from seamless, and the absence of consistent data quality standards posed a significant challenge.

The lead-to-order process, which appeared simple on the surface, harbored hidden complexities. There was a lack of clarity about how data flowed through this process, what triggered specific data creation events, and, most importantly, who owned the data. There were hundreds of individual users who relied heavily on this data for their daily tasks, yet they often found themselves asking a seemingly simple but critical question: “Where’s my data?”

In their minds, an abstract corporate-level service was responsible for ensuring data accuracy and reliability. It was a common misconception that data maintenance was someone else’s responsibility. This misalignment of expectations, combined with interruptions in data flow between key systems, led to a steep learning curve for our business partners.

I was determined to address this challenge head-on.

My role as the Central and Eastern Europe (CEE) area data lead, which I had assumed in the summer of 2008, was to bridge the gap between the business teams (predominantly in sales and marketing) and help establish a consistent approach to tackling data-related issues. Despite having no direct reports and limited influence over local business leaders in this vast and relatively undeveloped region, I drew on my extensive experience of navigating complex global data challenges at Microsoft.

In this new role, I was known to many of my colleagues, having previously been part of the consultancy team that worked on various data-related work aspects. The most common question I encountered from employees across the company was seemingly straightforward but carried profound implications: “Where’s my data?”

In reality, we were all in the same boat. Many had not yet grasped the fundamental concept that data was an asset they needed to take ownership of. They believed that an abstract corporate entity would perpetually ensure that data remained up to date and in perfect condition.

As I delved deeper into this challenge, I soon realized that my region faced a more significant hurdle than others. While my peers in Western Europe, the Middle East, and Asia had already developed practices that suited their regions, CEE grappled with the lowest data quality standards and a less structured approach to data management.

Nevertheless, this presented an opportunity.

Starting from the bottom allowed us to develop modern data practices tailored to our unique needs, with each improvement meticulously noted. Although we were the last to embark on this journey, we benefited immensely from established data foundations, the learning curve, and the ability to test and refine our approaches, ultimately achieving comprehensive data management practices relevant to any other part of the company. So, how did we do this, and where to start?

Forging collaboration

In my quest to tackle the data challenges we faced, I initiated a close collaboration with my colleagues in sales and marketing.

Our first step was to understand the exact nature of their challenges and identify potential solutions. We quickly realized that some issues could be addressed with straightforward guidance and better education. By fostering a deeper understanding of the overall data flow and demonstrating how improved data handling could benefit each team, we made significant progress. Here’s how we did this:

One striking example of collaboration was the collaboration between the marketing and sales teams. The marketing team aimed to generate leads quickly from various internal and external sources, with minimal restrictions. They were focused on the quantity of data. On the other hand, the sales team required well-qualified leads, as they relied on accurate segmentation to identify sales potential and create profit opportunities.

Bridging this gap required more than just a shared business language; it demanded a discussion in the language of data. We used real-world examples of good and bad data and illustrated how improvements in data handling by the marketing team could significantly benefit the sales team. We also made it clear that unqualified, low-quality data only goes to waste in the end, reducing the clarity of sales and marketing activities while it is present in the CRM system. Both teams immediately realized that they were sharing the pain of having bad data and urged for better cross-team collaboration.

This practice was highly successful, combining elements of change management and stakeholder management. It hinged on the idea that using data as the common language could drive efficient collaboration and emphasize the benefits of getting it right the first time.

Our second practice involved identifying the most critical business challenge that organizations faced. It didn’t take long to pinpoint a recurring issue: managing top customer accounts in a structured and consistent way. These accounts were vital for revenue generation, yet they often had inaccurate revenue figures and faced challenges with segmentation. The complexity arose when dealing with larger enterprises, which often had multiple branches and distinct company names under a single legal entity. In our case, this challenge, now known as Master Data Management (MDM), was limited to resolving the flow of revenue generated by these enterprises. It was vital to map this revenue back to existing accounts and maintain consistency to ensure effective forecasting and account management. Our proposed solution was the creation of a Managed Accounts List (MAL), which aimed to unify account definitions in Customer Relationship Management (CRM) and revenue collection flows, along with additional details from licensing, billing, and incentive compensation systems. It was a game-changer for us. It not only addressed a significant challenge but also demonstrated the value of data management and global data consolidation. The solution visualized one of the most critical data sets for business usage in one place, seamlessly connected and integrated into operational routines.

Before MAL, similar attempts were made at the country or subsidiary level but lacked a centralized, company-wide data standard. Our approach was to provide a global solution that could be adapted to local rules while offering global capabilities. This new, dedicated data management platform, separate from CRM systems and data warehouses, was operated by a virtual data team (more on this later). While creating the platform’s foundation wasn’t overly complex, the key takeaway was that when faced with a pressing business problem, we had to take action. The platform was designed to be global and reusable but not overengineered, focusing on delivering critical business capabilities immediately. In today’s cloud-native environments, creating such a platform is simple, but back then, it required a bold approach driven by business needs, courage, and a network of data professionals worldwide.

Speaking further of our virtual team, another common question arose after the well-known “Where is my data?” – the question of “Who is in charge of fixing the issues?”

Unveiling the ownership

The question of who was in charge became increasingly pressing as businesses encountered persistent data quality issues hindering their operations. Business leaders were eager to pinpoint responsibility for particular systems, processes, and data-related problems. Dealing with Tier 1 systems and applications as such posed no challenge; corporate engineering teams owned and managed them, and handled developments, change requests, and fixes. However, when delving into the intricacies of data collection and processing, things got complicated.

One issue was the lack of data education among business units. A disconnect existed between various teams, each focused on its immediate needs. However, data was the common thread that connected them all.

It became evident that educating stakeholders about data issues, their root causes, and potential solutions was crucial. We aimed to instill a sense of data ownership within these business stewards, clarifying what could be fixed and what couldn’t. Data examples were used to illustrate desired outcomes and encourage businesses to take greater responsibility for their data assets.

The second challenge was related to the corporate ownership of systems. In large enterprises such as Microsoft, the central IT department typically owned most Tier 1 business applications and their content, including data and business rules. However, in our case, corporate ownership wasn’t so effective and well-defined due to the complexity of local rules, diverse languages, and varying data capture practices worldwide. It was unrealistic to expect corporate engineering to address data quality challenges comprehensively because they lacked the necessary context and knowledge of local intricacies and policies.

To bridge this gap, we needed to foster efficient connections between the global and local sides of the organization. Corporate system owners were instrumental in resolving technological and application issues but couldn’t address data quality challenges effectively. On the other hand, local data professionals were hired by sales, marketing, and operations teams focused on improving data quality and reporting. However, they often operated in isolation, mirroring the segmented approach of business groups.

This scattered but dedicated community of data professionals presented a unique opportunity. My goal, as the area data lead, was to transform them from individual contributors into a virtual data team. This team wouldn’t report to me but would act as a collective entity, sharing knowledge and supporting one another across geographical boundaries while continuing to serve their local business units.

As I contemplated this during my flight to Prague, I realized that I needed to offer something transformative that could unify and energize this diverse group. It had to be more than a technical solution; it needed to inspire and empower them. I envisioned a knowledge-sharing community where everyone could contribute and learn, a space where local practices could evolve into global best practices.

Additionally, I saw the potential for economies of scale. Despite their busy schedules serving local geographies, collectively, we could provide a shared service model. By establishing standardized data management and a common catalog of offerings, we could efficiently deliver data services across the Central and Eastern Europe region. It would be like a swarm model, where any member of the virtual team could execute data modifications from anywhere in the area.

Overcoming language barriers and the specificity of local data requests was challenging, but we successfully created a virtual team who were knowledgeable about each area’s aspects.

The members of this team could step in for each other, providing efficient and high-quality data services. It was a simple yet effective model, and it outperformed siloed work within specific business domains. Data is a common asset across an enterprise, deserving of professional care and dedication. Having this virtual shared services team, along with tailored data solutions, also addressed the questions of “Where is my data?” and “Who is in charge?”.

The MAL solution we provided became a one-stop shop for many sales and marketing professionals, simplifying data access and quality monitoring. I encourage you to read the story of MAL ahead and, hopefully, this knowledge will be useful in your business and data environments. That being said, the virtual shared services team effectively became responsible for data management and associated data manipulations and processing.

This marked the beginning of a new era in data management.

The birth of MAL

MAL was a crucial concept within Microsoft’s business framework, and was particularly relevant for businesses dealing with many business-to-business customers. It aimed to segment customers, focusing primarily on the most valuable ones.

This list was a hierarchy consisting of parent accounts and child accounts, with each geographical area initially developing its own list to emphasize important local customers. These local lists gradually evolved into a more global approach.

MAL served as a powerful tool for Microsoft’s sales and marketing teams. It helped them concentrate their effort on the most valuable accounts and track various activities related to them, including marketing campaigns, sales promotions, and significant deals.

From a data management perspective, a significant challenge was maintaining the connections between branches (child accounts) and well-established top (parent) accounts. Ensuring that child accounts were correctly related to top-tier accounts was essential for accurately assessing the overall revenue generated by these top accounts. This task also involved handling issues such as duplication and dealing with malicious accounts.

However, at that time, there was no dedicated application for MALs. These lists initially existed as offline or semi-online Excel spreadsheets stored in different countries and regions, lacking central management.

Attempting to manage them solely within the CRM system raised usability constraints and other challenges, such as the following:

Firstly, the CRM system didn’t provide a direct connection to the revenue collection system, which was a separate system. This lack of integration made it difficult to consistently link account revenue data and other critical attributes, such as changing company names or addresses. To address this, we had to establish a bridge and maintain continuous links between CRM account definitions and those in the revenue collection system.Secondly, maintaining additional attributes specific only to MALs within CRM posed challenges. These attributes, such as assignments between account teams and aspects of parent-child account hierarchy, were essential for MAL but not suitable for CRM. Customizing CRM to accommodate these attributes would have been more complex than managing them within the MAL application.

As a result, we decided to store and manage many of these attributes, especially the account hierarchy, in the dedicated MAL application. This data would be synchronized with the rest of the data eco-flow, essentially making MAL an MDM solution for selected top enterprise accounts. The rest of the accounts remained managed in CRM. We facilitated synchronization between the MAL application, the CRM system, and the revenue collection system, ensuring data quality and maintaining the necessary data governance criteria and standards.

Figure 1.1 – Data overlap between key business systems

One significant aspect of this work was managing the hierarchy between top parent accounts and child accounts.

Interestingly, we realized that there would be no single source of truth for this hierarchy. Legal views on how a company is structured and organized often differ from the views necessary for sales processes, especially for multinational corporations with branches worldwide. We needed to respect the sales view of the hierarchy to enable effective sales processes, even if it deviated from the formal legal view. This flexibility and alignment with the needs of sales and marketing were key to MAL’s success.

The MAL solution was designed to be immediately relevant and useful to the business, not just accurate from a data perspective. It addressed critical business needs and gained traction quickly, helping drive data quality improvements and corrections. While we made certain trade-offs, such as building a custom application and hierarchy, ensuring the solution met the specific requirements of the business and provided the required flexibility was most important.

Next, let’s dive into a few stages of MAL development and what each stage focuses on.

Development overview of MAL

This was an extensive process that unfolded in several key stages:

Stage one: Our initial focus was to consolidate data from various offline sources spanning different regions. Our primary goal was simply to clean this data, preparing it for its initial release on the data platform. Of course, it also made it a very simplistic UI to operate.Stage two: In this phase, our aim was to centralize data availability on the data platform. We also introduced a more advanced user interface for data manipulation. This allowed users to perform tasks such as editing, adding new records, adjusting hierarchies, using preset basic workflows, and maintaining data accuracy. To safeguard the sensitive data involved, we implemented robust controls and security measures. Access to the data was limited to eligible users (primarily account managers) and specialized experts from teams of business and sales operations.Stage three: As we progressed, we began contemplating more advanced functionalities. These features were valuable but not immediately essential in the earlier stages. We also sought to integrate the MAL application more seamlessly into the broader Microsoft ecosystem and make it scalable, mature, and designed for many years of usage.

Let’s discuss all three stages in the following sections.

Stage one

In stage one, our initial challenge was determining the components that should comprise the MAL. We undertook a comprehensive analysis of existing definitions and all the Excel worksheets across various regions.

Our aim was to distinguish elements with global relevance from those that were highly localized or specific to certain areas. This process helped us identify what would be suitable for global release.

In parallel with examining attributes, we started looking deep into the data itself. We embarked on a journey of deduplication and the establishment of a single source of truth at the subsidiary level, progressively globalizing the MAL. However, it’s worth noting that not all regions initially embraced the project. Some (such as Japan and the United States) had highly specialized processes related to managed accounts.

We opted to concentrate on creating a product that could be swiftly launched and iterated upon in most regions, deferring more complex work and adoption challenges until subsequent stages.

In terms of data, a substantial portion was extracted directly from the CRM system.

We also integrated data from the revenue collection system and other business applications, including seller’s compensation tracking and some licensing data. These efforts culminated in the creation of what we called a golden record. These records were clean, free from duplication, and met the criteria for data quality and content. They served as the foundational dataset for the application. When users began accessing the application, they interacted with this consolidated, globalized dataset.

Soon enough, the challenge arose to keep this dataset current and accurate.

Synchronizing updates made within the MAL application with the CRM and revenue collection system became pivotal. To facilitate this bidirectional synchronization, we developed a specialized API. Although this setup required some user education regarding the potential overriding of changes made in CRM or the revenue system during the next MAL data refresh, it enabled us to swiftly address business needs and implement what businesses were primarily looking for.

This marked the culmination of stage one.

The bare minimum

At this point, we had a small but functional data platform, region-specific MALs, certain synchronization mechanisms, and a global deployment strategy. This deployment allowed us to centralize data within a single storage space, accessible through a unified user interface.

Now, let’s transition to stage two.

Stage two

This phase centered on elevating the user experience within the application. We introduced automated workflows and dispute resolution processes, encompassing tasks such as managing segmentation and updates to parent-child hierarchies.

We also initiated additional synchronizations with systems that could leverage the master data in the MAL for their specific requirements. Achieving the optimal user experience needed time and effort, given that this was a custom-built application.

At this stage, MAL became a global application used worldwide by the entire sales organization. Although some areas were investing more effort and control than others, overall satisfaction with MAL was already high.

It was a truly game-changing experience for the folks who, just a few months back, were operating on the Excel level! It was certainly a game changer for us as the virtual data team as it formed one of the core monetary investments and work focus areas for us back then.

Stage three

Moving forward to stage three, this phase brought forth substantial enhancements.

One noteworthy development was the introduction of a Future Year (FY) view. This feature allowed users to maintain the current account structure within the MAL while simultaneously simulating proposed changes for the upcoming financial year. This simulation held immense value for sales leaders, as it enabled them to experiment with the restructuring of account data for future financial targets.

At the finish line of the fiscal year, this FY view would supersede the current year view, effectively implementing the simulated changes. This process entailed intensive synchronization with our CRM and revenue collection system, which posed challenges due to the volume of updates.

Building trust

For our team of data professionals, this approach not only bolstered credibility but also encouraged the annual cleansing of data, ensuring a high-quality view of managed accounts for each fiscal year. It also established a repeatable routine for both data and sales teams, fostering collaboration and adherence to timelines.

In summary, the technical journey of constructing MAL unfolded in multiple stages, encompassing data consolidation, user interface development, and advanced functionalities. It required meticulous synchronization with existing systems and user education to navigate data overrides during updates. This approach not only addressed immediate business needs but also set the stage for enduring data quality and collaboration practices.

From a data perspective, the MAL project taught us the importance of focusing on what data is considered valuable and ensuring that the data perimeter doesn’t become a limitation. We introduced recommendations and logic elements to help sales and marketing teams identify data that could be candidates for inclusion in the MAL but hadn’t yet made it onto the list. This approach encouraged a broader perspective on data within the company.

Summary and key takeaways

By telling you the story of our initial data journey, I’ve unfolded three invaluable insights. These insights not only guided our path but transformed the way we perceived data within our organization. Even more importantly, they helped to address immediate issues with the data, win the trust of the business stakeholders, and set the foundation for future growth.

Takeaway 1 – becoming the change agent

Imagine a scenario where business units in your organization feel like islands, disconnected and isolated. This was our reality. My first insight is about becoming a catalyst for change – a change agent who bridges these disconnected units using data as a common language.

You don’t need a formal mandate for this; all you need is the determination to influence and advise.

Over time, this influence will earn your trust, making you a trusted advisor. By raising awareness about the importance of data, you will ignite a spark for change. Invest in individuals who exhibit curiosity about understanding data challenges. Educate and guide them in the world of data.

Eventually, these individuals will become your stakeholders and key sponsors, driving the change you envisioned.

Takeaway 2 – discovering the killer feature

Now, picture this: you’re on a quest to find the “killer feature” that will kickstart the transformation of your business through data. It might sound straightforward, but it’s not. It requires deep empathy and a meticulous exploration of existing data problems. You need a willingness to tackle these issues with unexpected and innovative solutions.

Our story of MAL exemplifies this. We didn’t just address our data challenges; we went a step further. We discovered a solution that not only navigated our data hurdles but also illuminated the path for others. It answered the question, Where is my data?, and provided the first steps in caring for data quality and fixing the most essential data issues.

Takeaway 3 – building the power of a virtual team

In every organization, there are individuals who deeply care about data, even if their roles don’t explicitly involve it.

My third insight revolves around building a virtual team.

When building a dedicated team isn’t feasible, assembling a ring of knowledge and data expertise is the next best thing. This team, lacking formal leadership or a hard mandate, becomes an incredible force for change. They unite to tackle real-world problems, much like the challenges we face with managed accounts.

They address the critical question of who is in charge, even if it’s not official yet.

Transformation starts with you

In our data journey, these three insights reshaped our approach, transforming isolated business units into a harmonious orchestra, with data as the conductor. We learned that change starts with influence and that the “killer feature” can revolutionize your data landscape, and even a virtual team can be a driving force for change.

These lessons aren’t just about data; they’re about the power of transformation and the people who drive it.

In the next chapter, we will cover building a central data team and inventing a simple yet powerful marketing motto that made our lives much easier.

2

We Make Data Business Ready

This chapter is about initiating and building a data team, sometimes called the Data Office, as this is a joint venture between data professionals and business stakeholders.

We will be covering about gaining support from key stakeholders and business teams with a successful, unique motto that laid the groundwork for future data management, fostering alignment around a shared and straightforward objective across diverse organizations and departments. After all, who could oppose the necessity for “business-ready data” to achieve success in marketing campaigns, CRM utilization, or revenue recognition? The ability to convey this uncomplicated message and goal marked a transformative moment, uniting like-minded individuals from various sectors, and encouraging them to actively embrace Data Quality (DQ) thinking.

During this stage, a centralized data team did not yet exist, and the company had not committed to such an investment. Consequently, the concept of a virtual data office emerged, operating under the banner of the Data Management Organization (DMO).

In this chapter, we will cover the following topics:

The power of one sentenceLocally inspired, globally connectedThe rise of data management organization

The power of one sentence

In the preceding chapter, we delved into the early stages of Microsoft’s journey toward becoming a data-driven organization. To be candid, those initial steps were far from smooth sailing. We encountered a multitude of challenges, not only in assembling a virtual community of data enthusiasts and partners but also in delivering on our promise to make a tangible impact on the business.

The story of Managed Accounts List stands out as an inspiring and comprehensive narrative. However, its inception was far from straightforward. While I won’t delve into all the intricacies and hurdles we faced during its development in this discussion, it’s worth noting that the journey from conceptualization to the first Minimum Viable Product (MVP) was time-consuming. Transforming it further into a pivotal application within the company was an iterative process.

Reflecting on our path and accomplishments as a data organization, one key factor looms large in my mind: the enthusiastic buy-in from stakeholders, particularly those in sales and marketing. Surely, many of them were on the forefront of DQ challenges, seeing and working with low quality data every day. Yet why would they trust us, why would they spend precious time supporting our initiatives? This support was vital for achieving our initial successes. It was a logical starting point since the individuals I interacted with in Prague were, for the most part, local data experts from various countries and subsidiaries. They played a crucial role as go-to data specialists, addressing basic data hygiene, resolving DQ issues, and managing data maintenance and updates. Their knowledge was deeply rooted in the unique challenges of local business operations and data.

However, our community lacked substantial connectivity, especially across geographical boundaries. Bridging this gap was essential, linking data specialists in Western Europe with their counterparts in Central and Eastern Europe, and so on, across the world. We were yearning for the wealth of best practices and diverse approaches that existed globally but were not readily accessible at a regional level.

Breaking down these silos became a priority.

So, beyond bolstering the virtual team of area data specialists and supporting newly established data communities in Central and Eastern Europe, I aimed to foster cross-regional connections. Our goal was to ensure we didn’t miss out on valuable best practices or solutions already developed elsewhere that we could swiftly adopt for mutual benefit.

Concurrently, as mentioned earlier, it was crucial to keep our team engaged in continual learning, remaining open-minded and connected to regional and global capabilities. I wanted them to remain highly relevant to the local sales and marketing teams while evolving to meet our evolving business requirements. As our knowledge grew with each passing day, we began contemplating how to deliver greater value to the business.

It was one thing to connect across regions and learn about fantastic practices in various locales, but translating these insights into practical solutions for our business leaders was another challenge altogether. We had to be highly pragmatic and discerning in our approach, extracting what could be adopted while also thinking expansively and proactively.

We aimed to collate diverse practices and start structuring them. This process involved documenting locally proven data practices, data terminologies and dictionaries, business rules, keywords, metadata, FAQs, exceptions, and their resolution procedures. While we weren’t certain whether all these practices would fit into our future strategy or merge into cross-regional capabilities, we were committed to capturing current knowledge and preparing for comparison, grouping, optimization, and more defined categorization.

Strategic assets

All these efforts eventually culminated in becoming the Intellectual Property (IP) of our DMO, a topic we will delve into more deeply in a dedicated way in Chapter 7.

Simultaneously, we realized the need to present our work and its impact more professionally to our stakeholders and business leaders.

We needed a statement that encapsulated our data mission, our ability to enhance the sales and marketing organizations, and our commitment to providing better data. This statement had to be bold and ambitious, resonating deeply with our business peers without being overly complex or theoretical.

It must bridge the gap between business needs and our capabilities. But how?

At this point, there were several teams working on DQ and data management across various geographic areas. However, these teams lacked comprehensive cohesion. In most cases, only the area data leaders were aware of each other’s existence.

Here, my unique position came into play. I was the last area data lead to join the company, and I already had extensive connections across various areas due to my prior work in implementing global data cleaning processes. I had interacted with teams in the UK, France, Germany, Western Europe, the Middle East, the US, and more.

I observed how data issues or data wins were presented to stakeholders and how internal customers reacted to various business challenges. Sometimes they were highly supportive, recognizing our relevance, while at other times, they doubted the value of data and believed that alternative corporate-level initiatives were needed. Deep engagement and empathy were essential to navigate this landscape.

Inspired by a successful approach I had seen with the UK team, which focused on aiding marketing activities and processes, I decided to take a similar route.

The UK team specialized in the intelligent preparation of marketing data, and they had developed a motto that resonated with me: “We make data business-ready.”

This simple, powerful statement encapsulated our mission perfectly. It conveyed that our community was responsible for “making data” and effectively answered the perennial question of “Who is in charge of data?” While the term “making” might sound strange in the context of data, at that time, it signified the creation of something real and well understood, as we totally lacked robust data capabilities, platforms, or standardized data services.

This statement was instrumental during our experimental phase, helping us gradually move toward well defined and standardized data capabilities.

Even as we experimented with different approaches and solutions, it served as a beacon for our stakeholders, providing a clear and easily digestible representation of our mission. We introduced the statement We make data business-ready widely across our work and it became our guiding motto for several years. It united us with the business and their needs and proved effective in garnering support. We even put it in our email signatures and PowerPoint decks, incorporating it into our branding as much as we could.

Today, while data teams operate differently, with concepts such as data mesh and federated governance gaining prominence, I still believe that this simple yet powerful statement, along with the commitment it represents, can be a game-changer when there is a need to emphasize the value of data and make it more understandable and relevant to stakeholders.

We needed to start modestly, adapt to the organization’s needs, and gradually evolve. This flexible approach allowed us to discover the most pressing business needs, learn about emerging issues, and progress step by step.

Did that lack of defined data strategy randomize us? I guess not, as “by default,” we were reactive to what matters most and tried to capitalize on learnings and enhance our own data IP with every newly resolved business challenge. We had to build our approach and our effective data strategy from bottom up, adding every single win as continuous best practice and focusing our attention on where business priorities were.

As you know, back then, we didn’t have a centralized data team, and our approach and success were based on being adaptable and responsive. Looking back, this proved to be the right strategy.

Here we see the early evolution of our data work:

Figure 2.1 – Data work evolution

In less than 12 months from the starting point in our globalization and centralization journey, we established our first full data services catalog and a comprehensive data portfolio for the organization. But that’s a story for another time – a bit later.

Business needs lead the way

The key takeaway from this experience is simple: aligning with business needs is paramount, and the journey often begins with addressing those needs as they arise. This dynamic approach allows you to fine-tune your offerings, adapt, and build credibility with stakeholders. Our commitment to making data business-ready, step by step, was instrumental in our long-term success.

Having one unifying and inspiring motto helped to connect various regional teams into a strong virtual community, cementing the foundations for the next stage.

Locally inspired, globally connected

A single sentence, “We make data business-ready,” quietly sparked a steady data revolution within our global company. This phrase wasn’t just an off-hand statement ; it became a powerful motto, a force that transformed our approach to data.

Something critical to note was that at the time, this simple sentence did more than unify and position the efforts of data practitioners within the company. It served as a universal replacement for what we now call a data strategy or data management strategy. Believe it or not, in those early stages of our journey toward a global, well-defined data approach, there was no centralized data strategy or data management strategy in place.

It may sound surprising, but it’s true.

We had a corporate data team, and they did work toward enabling global capabilities and defining a much-needed, multi-year so called “Horizon” model for data applications, yet the actual usage of data was in the so-called “Field”, or, in other words, the geographically based sales, marketing, and finance teams outside of Redmond Corporate Headquarters. They were the users and consumers of data, and we had no strategy to address their needs.

Moreover, every corner of the company, every subsidiary, and even individual business groups had their own unique perspective on how data should be handled. They each faced their own distinct data challenges. Sounds familiar for many, I guess.

We, as data practitioners, had a unique ability during this time. We could bridge the gaps between different geographical areas and act as the leaders of data practices within those regions. We focused on the practical aspects of data, delving deep into the operational side of things. Simultaneously, we were making substantial contributions to improving DQ and ensuring proper data maintenance to benefit the business.

That being said, in each geographical area, we encountered a mix of stakeholders. Some were fully aligned with and supportive of our mission, while others were skeptical or misaligned with the value of data.

In this challenging situation, what truly helped us was the universal language of data. Intentionally keeping ourselves outside of corporate politics, we made a first attempt to lead with data-driven decisions, visualizing DQ issues in front of our stakeholders while offering simple, practical follow-up solutions how to improve their experience with the data. When I say “universal language,” I mean the language of data itself – factual data points, examples of bad and good data, and the impact of successful data initiatives. When we shared common findings and examples across different regions, it became clear that DQ challenges in the CRM, for example, were remarkably similar in France and Germany. Yes, local languages and practices differed, but the core data challenges were surprisingly alike. By using concrete data examples, we began consolidating our data management knowledge across regions, along with winning more and more trust with our stakeholders.

We transformed into hands-on data practitioners, deeply immersed in the systems. We could recap primary keys and unique IDs by heart, which helped us truly understand the data landscape within the company. We empathized with each challenge and data issue reported by the business, and we still had high empathy for any new data challenge that emerged.

Data strategy – keep it simple

In a way, our data strategy in those days was simple: be empathetic and ready to react to any new data challenge that came our way.

However, we surely knew that being only reactive wasn’t the most efficient approach, and we aspired to shift toward being proactive. We aimed to be not only empathetic but also relevant, responsible, and highly accountable. We constantly searched for ways to innovate and elevate our data game.

One of our early solutions was the creation of a game-changing application known as Managed Accounts List, or simply MAL. We aimed to empower users to handle data issues and challenges themselves,