Salesforce Data Architecture and Management E-Book

Ahsan Zafar

• Herausgeber: Packt Publishing
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 587

Veröffentlichungsjahr: 2021

Beschreibung

As Salesforce orgs mature over time, data management and integrations are becoming more challenging than ever. Salesforce Data Architecture and Management follows a hands-on approach to managing data and tracking the performance of your Salesforce org.
You’ll start by understanding the role and skills required to become a successful data architect. The book focuses on data modeling concepts, how to apply them in Salesforce, and how they relate to objects and fields in Salesforce. You’ll learn the intricacies of managing data in Salesforce, starting from understanding why Salesforce has chosen to optimize for read rather than write operations. After developing a solid foundation, you’ll explore examples and best practices for managing your data. You’ll understand how to manage your master data and discover what the Golden Record is and why it is important for organizations. Next, you'll learn how to align your MDM and CRM strategy with a discussion on Salesforce’s Customer 360 and its key components. You’ll also cover data governance, its multiple facets, and how GDPR compliance can be achieved with Salesforce. Finally, you'll discover Large Data Volumes (LDVs) and best practices for migrating data using APIs.
By the end of this book, you’ll be well-versed with data management, data backup, storage, and archiving in Salesforce.

Leseprobe

Salesforce Data Architecture and Management

A pragmatic guide for aspiring Salesforce architects and developers to manage, govern, and secure their data effectively

Ahsan Zafar

BIRMINGHAM—MUMBAI

Salesforce Data Architecture and Management

Copyright © 2021 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Alok Dhuri

Publishing Product Manager: Kunal Chaudhari

Senior Editor: Storm Mann

Content Development Editor: Nithya Sadanandan

Technical Editor: Pradeep Sahu

Language Support Editor: Safis Editing

Copy Editor: Safis Editing

Project Coordinator: Deeksha Thakkar

Proofreader: Safis Editing

Indexer: Pratik Shirodkar

Production Designer: Joshua Misquitta

First published: June 2021

Production reference: 2100822

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80107-324-0

www.packt.com

To my parents for instilling the belief that I can do anything that I set my mind on, and to my wife for her immense support and keeping our young kids busy so that I could think and write uninterrupted.

- Ahsan Zafar

Contributors

About the author

Ahsan Zafar is Salesforce Technical Architect. He is 15x Salesforce certified and is a Salesforce Certified Technical Architect (CTA) candidate. He has spent around two decades working with different aspects of data, including data architecture, master data management, and data migrations. Ahsan has also had stints as a Business Analyst, Project Manager, and Salesforce consultant on various projects that gave him a unique understanding of how data is perceived by different roles. He has also worked as a developer and as a technical lead, spending almost a decade implementing Oracle ERP systems that typically involved migrating significant volumes of data from legacy systems.

About the reviewers

Ali Najefi is an enterprise architect with 18 years of experience, of which 13 years has been in Salesforce. He is a Salesforce CTA with a track record for delivering complex digital transformation projects on the Salesforce platform.

Experienced in leading Salesforce architecture and design authority teams, he was responsible for ensuring all Salesforce delivery teams adhered to Salesforce's best architecture and design principles. He has a significant interest in cybersecurity, identity and access management, digital data management, MDM, analytics, and AI, where he has worked on several enterprise clients both directing the architectural domain and delivering solutions that have proven to be secure and scalable for millions of customers and thousands of enterprise users.

Ram Iyer is a certified Salesforce application architect and system architect with 20 years of overall experience in enterprise architecture, solution design, data architecture, business analysis, and project management. He has over 12 years of architecture, design, and development expertise in Salesforce.com, Lightning, and Einstein Analytics. Ram has been a guest speaker at Dreamforce (2016, 2017) and is an evangelist for enabling continuous integration and delivery capabilities in large organizations. Passionate about technology, Ram has worked extensively on API design, application integration, business intelligence, DevSecOps, Agile, and Waterfall models of software development, with proven leadership abilities in managing large, complex projects.

Table of Contents

Preface

Section 1: Data Architecture and Data Management Essentials

Chapter 1:Data Architect Roles and Responsibilities

Defining architecture

Exploring architecture roles

Business architect5

Data architect5

Solution architect6

Domain architects6

Why is data architecture important?

The benefits of data architecture9

Data architect responsibilities

Data architect skills

Technical skills13

Soft skills14

Becoming a data architect20

A day in the life of a data architect

Summary

Questions

Further reading

Chapter 2: Understanding Salesforce Objects and Data Modeling

Exploring data modeling

What is a data model?26

Normalization30

Denormalization32

Design principles for data modeling35

Reviewing object relationships38

Differences between SQL and SOQL42

Understanding Salesforce architecture

Multi-tenancy44

Metadata-driven architecture45

New releases55

Introducing Salesforce objects

Standard objects56

Custom objects56

Big objects56

External objects57

Fields in Salesforce57

Summary

Questions

Further reading

Chapter 3: Understanding Data Management

What is data?

Is data valuable?67

Introducing data management

Benefits of data management68

Challenges of data management70

Introducing the data life cycle

Data creation73

Storage73

Usage73

Archival74

Purge74

Data life cycle – A Salesforce example74

Data management operating models

Centralized operating model76

Decentralized operating model77

Hybrid operating model77

Learning data management best practices

Understanding data and metadata80

Introducing data backup and recovery

Reasons to back up data95

Devising a strategy for data backup

Costs100

Security100

Service levels101

Ease of use101

Solution comprehensiveness102

Performance103

Types of backup

Restoring data

Missing data105

Sandbox seeding105

Citizen development106

Nuances of the restore process106

Backup and recovery – tools of the trade

Data Export Service109

Data Loader109

Odaseva109

OwnBackup109

Summary

Questions

Section 2: Salesforce Data Governance and Master Data Management

Chapter 4: Making Sense of Master Data Management

Understanding master data

What is master data?116

The need for master data management117

Categories of data119

Solving problems via master data management121

Deciding what data is master data124

Multi-org scenario126

Defining Master Data Management (MDM)130

Reviewing the basics of data quality131

Implementing MDM136

Considerations for selecting an MDM solution141

The Golden Record

Why is the Golden Record important?144

Detractors of the Golden Record145

MDM and CRM strategy

Customer 360

Common Information Model (CIM)150

Summary

Questions

Chapter 5: Implementing Data Governance

Enterprise data governance

What is data governance?157

Understanding the need for data governance158

Benefits of data governance159

Understanding the difference between data governance and data management160

Metadata management162

Guiding principles for data governance programs164

The keys to the kingdom – making your data governance program successful167

Successfully governing Salesforce171

Technical change control176

Business backlog177

Assessing the current state of data governance

Assessing the current landscape179

Need for data governance maturity models184

Data privacy and privacy laws

Understanding the need for privacy laws187

Understanding the business risks189

Global Data Protection Regulation (GDPR)191

California Consumer Protection Act (CCPA)197

Salesforce tools for implementing privacy laws201

Putting it all together

Problem205

The solution approach205

Summary

Questions

Further reading

Chapter 6: Managing Performance

Salesforce Platform performance

Importance of performance monitoring212

Reasons for performance-related issues215

Performance tools

URL suffixes221

Speedtest221

Salesforce Optimizer223

Salesforce Shield’s event monitoring223

Salesforce Page Optimizer226

Salesforce Lightning Inspector226

Salesforce reports226

Query Plan tool227

Improving performance

Conducting performance testing in Salesforce

Approaching performance testing in Salesforce256

Conducting a successful performance test260

Monitoring performance

What to monitor?262

Summary

Questions

Further reading

Section 3: Large Data Volumes (LDVs) and Data Migrations

Chapter 7: Working with Large Volumes of Data

Revisiting databases

Relational databases271

Non-relational databases272

Large Data Volumes (LDVs)

Knowing the implications of LDVs277

Multitenancy and search architecture278

Considerations for integrating or migrating LDVs279

Preventing LDV scenarios284

Optimizing LDV operations287

Salesforce Connect and external objects

Ways to connect297

Introducing big objects

Benefits of big objects300

Considerations for big objects301

Use cases for big objects303

Summary

Questions

Further reading

Chapter 8: Best Practices for General Data Migration

Assessing data

Components of a data migration assessment311

The Preparation phase

Best practices314

The Execution phase

Best practices316

Tools for loading data into Salesforce

Data Import Wizard (DIW)320

Salesforce Data loader320

Understanding APIs

The SOAP API322

The Bulk API323

The REST API328

The Streaming API328

Summary

Questions

Further reading

Assessments

Chapter 1 – Data Architect Roles and Responsibilities

Answers333

Chapter 2 – Understanding Salesforce Objects and Data Modeling

Answers333

Chapter 3 – Understanding Data Management

Answers333

Chapter 4 – Making Sense of Master Data Management

Answers334

Chapter 5 – Implementing Data Governance

Answers334

Chapter 6 – Managing Performance

Answers334

Chapter 7 – Working with Large Volumes of Data

Answers334

Chapter 8 – Best Practices for General Data Migration

Answers335

Other Books You May Enjoy

Preface

Data is becoming more and more valuable to organizations as they realize its true potential and how it can help them to achieve their business objectives. The reason for this is that data can be turned into actionable intelligence that organizations can leverage to keep their internal and external customers happy while maintaining their competitive edge.

The key here is actionable intelligence derived from data otherwise it takes up storage space and is an expense to the organization. The Salesforce Platform facilitates turning unactionable data into actionable intelligence, but for this to happen on an on going basis, the data architecture must be properly designed and allow scalability. Data modeling blunders include creating roll-up summary fields on an account record that have the year hard coded in them to show account revenue and require the roll-up summary filter to be updated every year. Blunders also include worse situations where a lookup relationship may have been used for an application, but a parent-child relationship was created, leading to a tightly coupled relationship. The sole reason for choosing this type of relationship would be due to it providing the roll-up summary field capability.

In this book, we will start with the very basics of understanding what data architects are expected to do and how you can be a successful Salesforce data architect. You will then learn about data modeling and data management. Once we have the basics covered, we will delve into master data management, data governance, and how we can ensure performance for our applications.

Data keeps on growing at a fast pace and knowing how to design solutions effectively involving large volumes of data is a crucial skill set. Therefore, we will extensively cover Large Data Volumes (LDVs) and what we can do as architects to effectively manage them while keeping scalability and Salesforce governor limits under consideration.

I have included questions at the end of each chapter to ensure you have clearly understood the concepts discussed in the chapter. Moreover, there are extensive examples and diagrams throughout the chapters to ensure that a firm understanding of the concepts is developed.

When all is said and done and you have reached the end of the book, you will have gained a solid understanding of data architectural skills and best practices that you can apply immediately within the context of Salesforce. The content covered in this book is also very relevant to the Data Architecture and Management Designer exam and the data architecture domain that is tested in the Salesforce Certified Technical Architect (CTA) exam (https://trailhead.salesforce.com/credentials/dataarchitectureandmanagementdesigner).

Who this book is for

This book is for both aspiring and experienced architects, Salesforce admins, and developers who want to learn more about the core architecture of the Salesforce platform in the context of data management and architecture. Whether you are just starting off in the Salesforce ecosystem as an admin or a developer, or whether you are an experienced architect with several years of experience under your belt, this book has something for everyone to benefit from.

To get the most out of the book, you will have had experience with data migration and integrations in large, complex technology landscapes consisting of disparate systems. You will also have had to deal with data with privacy regulations and topics such as master data management to further manage data in your technology landscape. Salesforce professionals that have large orgs with millions of records who are concerned about performance will also benefit from the book as scalability and performance have been at the forefront when writing.

Although the book covers a multitude of topics that overlap different Salesforce certification exams, Salesforce professionals specifically preparing for the Data Architecture and Management Designer exam or the Salesforce CTA review board will find the book helpful as supporting material for these exam preparation.

What is this book covers

Chapter 1, Data Architect-Roles and Responsibilities, describes the role of a data architect and the core skills and experience that are required for it. It will also go in to detail on what soft skills are required to be successful in the role. You will also get to have a look at a day in the life of a data architect.

Chapter 2, Understanding Salesforce Objects and Data Modeling, will take you through the unique architecture of the Salesforce platform and how it is optimized for read access rather than write operations traditionally seen in relational databases. Data modeling concepts, how they get applied in the context of Salesforce, what de-normalization is, and why it is important to spend the time designing your data model properly will be discussed. At the end of the chapter, Salesforce objects and how they are created, different types of fields on them, and their use cases will be covered.

Chapter 3, Understanding Data Management, will explain data management, what it is, and why it's important. The different aspects of managing data, including the data lifecycle, will also be discussed. With Salesforce's discontinuation of data recovery services, data backup and archiving have come to the forefront, so we will discuss that in detail as well. At the end, some tools that are available to manage data effectively will be reviewed.

Chapter 4, Making Sense of Master Data Management, will discuss the key attributes of master data, what the Golden Record is and why it is so important for organizations. We will look at how to align your MDM and CRM strategy with a discussion on Salesforce's Customer 360 and its key components. MDM is a platform-agnostic concept that can be used within the context of non-Salesforce landscapes as well. The chapter is concluded with a brief discussion of the Common Information Model (CIM).

Chapter 5, Implementing Data Governance, covers the importance of enterprise data governance, the relationship between data governance and data management, and how to assess the current state of data governance. Two major privacy protection laws, the GDPR and CCPA, will also be covered in detail. To conclude and firm up understanding of the content, a sample case study will describe a hypothetical scenario and the solution approach to solve it.

Chapter 6, Managing Performance, will explore foundational aspects of performance on the Force.com platform, how to use the Query Plan tool to determine performance-impacting queries, and query costs when using indexes versus full table scans. The chapter also covers the various tools that can be used to monitor the platform for performance and auditing changes. Multiple code blocks will be used to drive the point home of how performance can be determined and optimized. Performance testing is critical especially when dealing with large volumes of data, so an extensive discussion around aspects of performance testing will be covered, followed by a discussion on monitoring the performance of the Salesforce org.

Chapter 7, Working with Large Volumes of Data, will introduce you to the concept of relational and non-relational databases. LDVs, which are becoming more and more relevant in the Salesforce ecosystem as orgs generate or consume lots of data, will be discussed extensively, from identifying LDV scenarios to managing LDV orgs and integrating data into these org types.

We will look at some options in cases where large volumes of data don't necessarily have to be brought into Salesforce, but the data can still be made available to users. We will cap our discussion with Big Objects which is yet another way to deal with very large data volumes.

Chapter 8, Best Practices for General Data Migration, will introduce you to data migration - how to assess, plan, and execute data migrations. Considerations and best practices related to data migration will also be discussed. Close to the end, we will discuss some commonly used tools that can be used for data migration. We will cap our discussion by discussing the different APIs that are available in Salesforce within the context of data migration.

To get the most out of this book

Theis book has multiple SOQL query code blocks throughout the chapters, and you are strongly encouraged to follow along and try them in a Developer Edition of Salesforce. For chapters that may appear to be theoretical in nature, for example, ones on data governance, I would encourage you to think of your own current or past organizations and try to apply the principles learned in the chapters, including the practical aspects of these topics, for example, metrics or KPIs for data quality.

The book assumes an understanding of basic Salesforce functionality, so at times, advanced topics may be challenging to grasp in the absence of the required knowledge.

Download the color images

We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://static.packt-cdn.com/downloads/9781801073240_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "Changing the criterion to LastModifiedDate will fix the issue."

A block of code is set as follows:

SELECT Id, Name, Description, LastRunDate, LastModifiedBy.Name

FROM Report

WHERE LastRunDate < 2019-06-01T00:00:00Z

ORDER BY LastRunDate DESC

LIMIT 50

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

SELECT Id, Name, Description, LastRunDate, LastModifiedBy.Name

FROM Report

WHERE LastRunDate < 2019-06-01T00:00:00Z

ORDER BY LastRunDate DESC

LIMIT 50

Any command-line input or output is written as follows:

ping -f -n 25 -l 1200 na111.salesforce.com

ping -f -n 25 -l 1300 na111.salesforce.com

ping -n 25 -l 1400 na111.salesforce.com

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: "In Developer Console, click on the Logs tab."

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you've read Salesforce Data Architecture and Management, we'd love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we're delivering excellent quality content.

Learn more on Discord

To join the Discord community for this book – where you can share feedback, ask questions to the author, and learn about new releases – follow the QR code below:

http://packt.link/sfdserver

Section 1: Data Architecture and Data Management Essentials

By the end of this section, you will have understood the role of a data architect and the skills that are required to be successful in the role. You will understand what data modeling is, how to apply data modeling concepts in Salesforce, and how they relate to objects and fields in Salesforce. Concepts related to data management and the different aspects of data management will be discussed. Backup and archiving strategies and some tools to use to manage data will also be discussed with hands-on exercises.

In this section, there are the following chapters:

Chapter 1, Data Architect – Roles and ResponsibilitiesChapter 2, Understanding Salesforce Objects and Data ModelingChapter 3, Understanding Data Management

Chapter 1:Data Architect Roles and Responsibilities

In this chapter, we will first define the term architecture in order to establish a baseline understanding of what the term means. This will help us to establish what it is, bring other stakeholders on the same page, and prevent scope creep in roles and responsibilities. We will then look at the types of architects, their roles, and why it's important for them to work together in order to both produce a cohesive architectural strategy and to deliver value to an enterprise. Often, people may mention needing a solution architect, but in reality, the work they need to do may be more suitable for a data architect because the skill set required is different. So, understanding the different types of architects is important.

In this chapter, we will also look at the responsibilities of a data architect and the soft and hard skills needed to be successful in the role. Covering these topics will help you to communicate with architects more effectively in order to engage the right resources for your project. Understanding this from the beginning will help you to see the overlap that sometimes exists between the skills that data architects and solution architects have and understand when a task may require the skills of one role more than the other.

We will conclude the chapter by discussing a typical day in the life of a data architect, which will allow us to understand the work they do and the potential challenges they face.

In particular, we will cover these topics:

Defining architectureExploring architecture rolesWhy is data architecture important?Data architect responsibilitiesData architect skillsA day in the life of a data architect

Defining architecture

Architecture is a broadly used term and can mean different things to different people. Therefore, it is important to first define it so that we can establish a common understanding of this term before proceeding further. The term architecture is defined as the practice of designing systems, buildings, or things. In the context of technology, it refers to the practice of designing structures or systems with individual components with the intention of optimizing the function, cost, and performance of the overall structure.

Thinking in physical terms, a construction architect designing a home will architect how large each room is going to be, as well as decide on the number of windows and their placement in the room. This may also include planning where each room should be located to maximize the use of space and what direction the windows should be facing to maximize sunlight. In a nutshell, a building architect aims to maximize the use of space while keeping optimal functionality and costs in mind.

Similarly, a technology architect designs solutions by making use of the available components, considering the pros and cons of each component, overall costs, and the overall performance of the system. Of course, the desired functionality needed from the overall system and its ability to meet the business objectives will also be a critical factor in determining what individual components to use for the system. The key difference between physical and technology architecture is that most technology architecture used to be based on the waterfall methodology and, as such, required extensive upfront planning, requirements gathering, and documentation. The waterfall methodology is a linear project methodology where a project is broken down into phases, and in most cases, the next phase cannot start until the previous phase has been completed with deliverables signed off. With the advent of Agile and other methodologies, the focus has now turned toward designing solutions that are scalable and can be quickly adapted to changing market conditions. This is important because in order for a business to stay competitive in the global economy, they cannot afford to spend weeks and months updating their systems.

For enterprises, architectural needs vary depending on the needs and stage of the issue the enterprise is trying to solve. Over several years, technology professionals have realized this and have delineated architecture into multiple domains. Since our discussion topic is the role of a data architect, it would be prudent to discuss the various domains of architecture because they are interlinked and exert constraints over each other.

In the subsequent sections, we will look at why data architecture is important as well as explore the different types of architects, their roles, and some of the deliverables they are responsible for producing and maintaining. This will give us a greater understanding of the differences in responsibilities and an appreciation of how the type of architect fits into the architectural puzzle of an enterprise.

Exploring architecture roles

Architects have been around for as long as business systems. They may have had different titles over the years, such as systems analyst, technology analyst, and the like, but fundamentally, they have always designed systems that are scalable and efficient. Today, the titles have become more refined and there are multiple types of architects specializing in their own domains. Understanding each of these is important because it will help us have better communication with them and enable us to bring other stakeholders on the same page as well.

Business architect

Business architects are concerned with identifying how value can be created for internal and external stakeholders of the enterprise. This usually involves producing strategy documents, process maps, capability maps, and commonly used business terms within the enterprise or the industry in which the enterprise operates.

Data architect

Data architects are the key topic of this book. Of course, they are mainly concerned with data: how data is organized, how it is moved internally or externally, and how it can be made accessible to users when it is needed. Data architects create data models and make decisions on how data will be stored, consumed, and archived. Some data architects will also analyze and communicate how insights and intelligence can be derived from data that can be useful to stakeholders. Typical deliverables include data models, data definitions, the flow of data, and integrations.

Solution architect

A solution architect is concerned with designing systems that deliver business value to stakeholders. Working closely with Business Architects (BAs), they ensure that the solution will provide the business value that the business requires. One of the key responsibilities of solution architects is to identify the most optimal technology that will meet business requirements. This can include packaged software such as Salesforce or designing custom solutions from the ground up. Deliverables for this type of architecture include functional and technical design documents, system landscape diagrams, and the actual software that's delivered in line with the design that the architect puts forward.

Domain architects

These are architects that have expertise in a particular domain of technology. For example, Information Security (IS) architects, technology architects, and cloud architects have very specific expertise in their respective areas and are considered domain architects. A Salesforce Certified Technical Architect (CTA), although required to have a broad knowledge of certain areas, are experts in implementing Salesforce solutions.

Information security architect

The IS architect has been becoming more important as data integrations between systems become common and cybersecurity becomes critical for enterprises to protect their data and, by extension, their reputation. IS architects are responsible for designing, implementing, and maintaining security solutions that can protect the organization's network and hardware. They regularly conduct different types of testing to identify vulnerabilities and proactively fix them before a security incident materializes. In cases where the organization's security is breached, they work on a root cause analysis and identify fixes to remediate those vulnerabilities.

Technology architect

A technology (or infrastructure) architect is concerned with the physical infrastructure needed to enable organizations to deliver their software applications to their stakeholders. Servers, networks, and cloud-based Infrastructure as a Service (IaaS) fall into this category. Their responsibilities include the following:

Designing and implementing the infrastructure solutions needed by organizationsSupporting the hardware needs of projects in an enterpriseMonitoring production environments to ensure that they are running optimally by monitoring certain attributes such as throughput, latency, and redundancy, among others

Deliverables include network diagrams, server-to-server diagrams, and others. With the advent of the cloud, another type of architect that's gaining traction is the cloud architect.

Cloud architect

These architects are mainly concerned with an enterprise's cloud strategy and its implementation. Like technology architects, they are concerned with implementing cloud solutions that can support software that runs on the cloud. The key difference between technology and cloud architects is that traditionally, the former has been more focused on on-premises systems whereas the latter has been focused primarily on cloud solutions. Salesforce, for example, runs entirely on the cloud and although customers don't have to be concerned with how that cloud is implemented, maintenance schedules, and security patches, Salesforce has internal resources that focus on maintaining the cloud infrastructure, ensuring that it can meet the needs of its customers. Cloud architects are also change agents as they must be able to effectively communicate the benefits of using the cloud and be knowledgeable enough to address the questions and concerns that people not so familiar with cloud technologies have.

Let's look at why data architecture is important and what its benefits are.

Why is data architecture important?

A common mistake that rookie software developers make is to jump in and start writing code, thinking that the design can be modified later to fit requirements. Sometimes, they may refer to this as Agile thinking and approach. However, this is a fallacy as the data model, once defined, will be used as the foundation upon which other required business functionality is built; it is usually not straightforward to make changes to it quickly.

Let's take an example of an integration that pushes data from an external webinar system into a Salesforce Campaign Member object. Here, the webinar system takes the amount of time a person watched the live webinar and records it as a percentage. The developer has created a custom field of the number type called Webinar % Attended on the Campaign Member object. This field will only accept a number and therefore rejects any letters or special characters.

When the integration is run for the first time, it errors out because the webinar system is sending the percentage attended with the special character, %, and Salesforce therefore rejects it. This is a simple example to demonstrate how the data model needs to be well thought out and designed, not only considering the business requirements, but also understanding system functionality and limitations. In this case, the business doesn't need the values in the Webinar Completed field to make any calculations, so changing the field type to Text will work. In most cases, though, a change like this would require a thorough analysis of impacts on existing functionality and integrations and this can be a time-consuming effort.

Moreover, owing to the large number of systems that are now used in an organization, the sharing of data between these systems (that is, integration) has become crucial. Proper data architecture can avoid costly mistakes such as integrating two systems only to find out at a later stage in the project that there are technical limitations on how often the data can be moved between the two systems.

Important note

In architecture, decisions made earlier affect decisions made later. For example, should a Team Member-related list, intended to record team members working on the lead, be a lookup or a master-detail relationship? These are important design decisions to consider upfront as a change during build, or worse, when the application is in production, can be costly. For example, a decision to change from a multi-select picklist to a single picklist when the app is in production can lead to data loss. Consequently, our earlier decision affects decisions we will have to make surrounding the subsequent data loss.

Another consideration for data architecture is data security. Nefarious individuals or state-sponsored actors understand the value of data, and they look to exploit vulnerabilities for their own advantage. Data architecture ensures that the proper security processes and techniques are applied to the data when it's at rest or in transit. Modern data architecture also needs to conform to data protection regulations such as General Data Protection Regulation (GDPR) andCalifornia Consumer Protection Act (CCPA).

Organizations recognize the value of data and derive insights from it. Moving data from one system to another is not done in vain, rather, there are clear business objectives that are to be achieved. The most common of these objectives is decision making based on hard data. Proper data architecture will take into account how data from a multitude of systems can be brought together to meet the operational and analytical requirements of the business. An intense topic of interest these days, which we will discuss in the upcoming chapters, is how to achieve that single, unified view of the customer commonly referred to as Customer 360, and data architecture plays a key role in that.

The benefits of data architecture

Having discussed the importance of data architecture, let's look at some of the benefits it offers:

It sets the stage for discussion and easy communication with different stakeholders: Creating a data architect diagram can facilitate discussions with solution architects and Business Analysts (BAs). It also facilitates getting approval from security teams that are concerned with how data is utilized and the boundaries it's going to cross; for example, a multi-national organization that has multiple legacy Customer Relationship Management (CRM) systems will want to address how data is going to move and be stored across continents in consideration of compliance with the General Data Protection Regulation (GDPR).It plays a vital role in driving non-functional requirements (NFRs): NFRs are related to the quality of the system being designed, such as performance, usability, and maintainability. Considering maintainability, for example, if the architecture is well defined, we will be able to enhance our application with relative ease compared to when the architecture has major flaws and in certain situations may require a complete redesign of the application, which is a costly and time-consuming endeavor.It enables change management: Change is inevitable. Now more than ever, internal and external forces are driving change at a rapid pace. Internally, enhancement requests, new business processes, and technological changes drive this change, whereas externally, raw market forces such as competition, geopolitical changes, and regulatory policies can drive this change. Having a well-defined data architecture that is properly documented and uses commonly understood language and standards enables stakeholders to respond to change quickly.It enables fast ramp-ups: Part of the change that we discussed in the last point also applies to changes in resourcing, meaning people in the company come and go. Having a good architecture established can be very helpful in training new personnel. Take the example of a new hire who joins the support team of an AppExchange partner that sells a popular recruitment app. Having the data model and reviewing it with the new hire can cut short the time needed to ramp up the resource to provide quality support to their customers.It is reusable: If an architecture was used once for an application and it was successful, it can be easily adapted by other areas in the enterprise or used for similar-natured projects in the future. Architecture work is intense and time-consuming and for large projects, it requires a lot of collaboration with multiple stakeholders. Reusing architecture eliminates or reduces the need to put in the same level of effort as the first time and this alone can translate into hard dollar savings for an enterprise.

There is also the quality aspect: when you are reusing your architecture the second or third time, you will be thinking of further improving it rather than rethinking the architecture from scratch. Over time, this cycle of continuous improvement can lead to an architecture that will start to be accepted in the company as a reference architecture. More and more, projects will start to adopt it, or the Enterprise Architect (EA) may mandate it as the starting point for designing other projects.

Having learned about the importance of data architecture and its benefits, we will next discuss in detail a data architect's responsibilities in the next section.

Data architect responsibilities

Enterprise architecture is the highest level and the starting point from where guidelines, best practices, and overall enterprise parameters are set. For example, all integrations, whenever possible, will use the enterprise middleware and tightly coupled point-to-point interfaces will be avoided. Alternatively, when integrating two systems, the receiving system must pull data from the sending system rather than the sending system pushing data into the receiving system on a set schedule. This impacts the data architecture directly because, when designing interfaces, the data architect will need to be cognizant of these constraints set by the EA. Similarly, the solution architect would need to consider what other integration options are available if a system cannot be directly integrated using a point-to-point design (the point-to-point integration pattern is generally discouraged and should not be used when other viable options are available).

In this section, we will focus on the responsibilities of a data architect. However, it is important to understand the goals of data architecture before discussing the roles and responsibilities of a data architect. In any enterprise these days, usually there are volumes of data generated or flowing into or out of the enterprise. The data architect's goal is to design blueprints to facilitate the short-term and long-term data needs of the enterprise securely. This requires understanding the long-term vision of the enterprise (business strategy) so that the data architect can propose and implement processes that will align data management with the business strategy and maximize the ROI in the enterprise's data initiatives. The responsibilities of a data architect include the following:

Understanding, assessing, and documenting the current state of the organization: This is the first and probably the most important step for a data architect, ensuring that they understand the current state thoroughly. This helps in identifying current issues in data flows and integration pain points as well as in formulating a plan for the future. Furthermore, documenting the current state aids in communicating with other stakeholders, such as EAs, solution architects, and business architects. This helps in securing project funding from the executive leadership.Developing a dictionary for data across the organization: Data architects define and maintain data dictionaries. A data dictionary is an inventory of data items that are used to convey information about data, such as metadata.Aligning data architecture activities with enterprise architecture: The data architect works closely with the EAs to ensure the initiatives that the data architect takes align with policies and guidelines established by the EA.Developing a data requirements plan for the long-term storage, archiving, processing, and transmitting of data: On an ongoing basis, data architects work on ensuring that the organization's data remains viable over a long period of time. They also need to anticipate industry and technology changes to ensure initiatives taken today will not obstruct the future use of newer technologies and that the organization can continue to evolve to face new marketplace realities.Guiding domain architects and external partners on optimal ways to access data: Data architects assist and provide guidance to domain architects and other stakeholders that may be trying to access the organization's data. They are expected to provide guidance around integration design as well to ensure that they align with EAs' policies and guidelines.Evaluating applications: Often, data architects are asked to participate in evaluating Commercial off-the-Shelf (COTS) applications from the context of data. They will look at data flows and how the new application would interact with the data. What changes, if any, need to be made before the application can work with existing data or data that is getting produced by the organization?Data governance: The data architect is also an integral part of data governance activities in the organization. They are responsible for documenting and maintaining processes related to technical data governance and data models for master data subject areas and reference master data.Coaching and mentoring: Lead data architects may also have team members that specialize in certain domains reporting to them. These individuals, as well as individuals or architects from other teams, may need coaching and mentoring with respect to matters pertaining to data architecture.Being the primary contact for vendors: Data architects also act as the primary technical contact for vendors pertaining to data, particularly Master Data Management (MDM) solutions or other data-focused applications within their portfolio.Delivering innovative solutions: With the rapid increase in the volumes and sources of data, regulatory requirements, and policies, organizations require innovative solutions to solve their data-related business problems. A data architect is required to maintain an extensive knowledge base of industry trends, best practices, and the available tools in the marketplace to propose optimal solutions.Compliance: Compliance is another area where data architects will spend a considerable amount of their time. To be clear, many organizations will have data privacy officers that understand the privacy laws and develop requirements to adhere to those laws. The data architect, on the other hand, is responsible for the technical implementation and ensures that data flow, storage, and retrieval are in line with these laws and organizational policies.

Now that we have looked at the responsibilities of data architects and their role in an organization, let's review the soft and hard skills that data architects need to be effective in their roles.

Data architect skills

A combination of technical and soft skills is required to be successful in a data architect role. Both sets of skills are equally important; it will be difficult to become a successful data architect if either set of skills is lacking. But don't worry, these skills can be learned, and once practiced, they will, over time, make you a very productive and successful data architect. In this section, we cover the skills required to become a successful data architect. Let's start with technical skills.

Technical skills

We will now discuss some of the technical skills that are needed to be a successful data architect. Keep in mind that these are high-level and this is intentional because otherwise, we would need to list individual skills that are needed:

Data modeling: The ability to define data models is a must-have skill to be successful as a data architect. This forms a very handy tool to communicate with different stakeholders in projects or in the general day-to-day operations of the organization. Knowledge of well-known and commonly used data vendor models, application models, and industry reference models is also an asset.Data integrations: The ability to design effective, secure, and scalable integrations using a multitude of tools, as well as knowledge of various integration concepts, such as pub/sub, request/replay, batch processing, and fire and forget.Data processing: Understanding and designing flows that take in data and output meaningful information is a skill that is becoming more and more important for data architects. With the advent of huge volumes of data that organizations have been storing over the years, knowing how best to make use of that data to provide insightful wisdom to decision makers is key for data architects.

These are some of the high-level technical skills needed by data architects but typically, they are required to have programming skills such as working with Python or another language, experience in data mining and modeling tools, and an understanding of operating systems.

Soft skills

In this section, we will talk about the character attributes of an exceptional data architect. There are many others that may not be listed here but these are some of the key personality attributes that a good data architect should have:

Problem-solving: A multitude of data issues are faced by data architects. Therefore, they must be able to apply a beginner's mindset in order to solve these problems. Communication skills including active listening and clear articulation of the problem and what the approach would be to resolve them are also characteristics that help in shaping a successful data architect. Knowing how to solve problems using systems thinking is also a very valuable skill to have. In a nutshell, systems thinking is the understanding that individual parts in a system behave differently compared to their behavior when they are by themselves. Utilizing systems thinking, along with the ability to model systems in a way that they have enough detail and convey the intent of the modeler, is a key advantage to have when solving problems.Storytelling: This is an essential skill that makes it easy to communicate with different technical and non-technical stakeholders. The ability to communicate to audiences using stories and metaphors that make the subject matter easy to understand for a non-technical audience can help to build rapport with stakeholders quickly.Objectivity: The ability to think and act objectively while keeping emotions in check and out of the decision-making process is an important skill for a data architect. Often, you will be interacting with other technical resources that may prefer one solution over another. The ability to objectively analyze each option and make a recommendation after considering each option will earn your stakeholders' respect and trust.Strategic thinking: Data is not something that is around for a month or two, but rather one of the most critical assets of a business and it needs to be treated as such. This requires understanding current and future business needs and the direction an organization is taking. It also involves anticipating the future trends of the industry and its best practices.Adaptable: A data architect should be flexible; they should be able to adapt to changes and operate in a fast-paced environment. Many organizations do not have the discipline and the necessary governance processes in place, resulting in a very chaotic environment from a data perspective. The data architect must be able to sift through that, understand what's required, and deliver on what will add value to the business.Leadership: As an architect, you will be expected to be responsible for designing and developing an optimal solution, which is all fine and dandy if you were doing all the work yourself, but that's not how the real world works. In many projects, you will be working with a team of architects, developers, BAs, and other stakeholders. Your decisions may be challenged and the rationale of certain decisions questioned. How do you deal with that? Using your technical or positional authority to push through your decisions? That may work in the short term, but in the longer term, you will face more challenges from the team and may also lose respect in the process. Having leadership skills that influence people to follow your decisions by inspiring them to do the right thing is a better longer-term approach. This requires establishing trust by walking the talk and being transparent with your peers. Keep in mind that you don't have to have an official title to be a leader; rather, you can still influence people regardless of the position or title you hold in the company.Analytical: Data architects need to be observant and have the ability to investigate a problem and find an optimal solution in a timely and efficient manner. This skill is very important because the ability to collect data from various sources, sift through non-relevant information, and analyze the rest to get to the root cause of a problem, can mean the difference between suggesting a solution that will solve the problem versus wasted time and effort.

One of the things that I want to drill down further is communication: the barriers to effective communication and the famous seven Cs that can be used to efficiently communicate. The importance of these cannot be stressed enough because unless you are an efficient communicator, your mileage with being a successful data architect will vary and you will face challenges. Now, let's look at these in the next section.

Barriers to effective communication

From the time we wake up to the time we go to bed, most of us are communicating with the people in our lives. Communication is not only verbal but can also be via body language or even social media these days.

The PRovoke Media, a reputable PR firm, estimated that a staggering $37 billion dollars were lost among 400 companies (100,000+ employees) surveyed in the US and UK and, according to the report, this may just be the tip of the iceberg – the actual losses may be much more. Let's look at some of the major barriers to communication. Recognizing these will help you to be cognizant and more careful next time you are giving a presentation, conducting a global team call, having a discussion with your children, or coaching a direct report at work.

Physical barriers

These are barriers to communication that are caused by physical conditions, for example, if your car breaks down on the highway and you are speaking with the tow-truck driver. Unless you are speaking loudly, you might be telling the driver to tow your car to your house, but they may hear it wrong and tow it to the garage. Another example is when you have an important presentation to give and your boss emails you 5 minutes before the presentation asking you to remove a slide but doesn't mention which one. You try to send them an instant message, but the service is down. Thinking they must have meant the slide with too many diagrams, you remove that slide but find out after the meeting that it was in fact a different slide.

This type of barrier is especially felt when people cannot have face-to-face interactions; for example, during the COVID-19 pandemic, people were reduced to communicating through phone, text, email, or on-camera meetings. Although on-camera meetings can be very effective, important information related to non-verbal cues, such as body language, eye contact, and gestures, can still get lost, all of which form an integral part of communication.

Psychological barriers

This is the type of barrier where communication cannot be effective due to psychological issues. A somewhat common manifestation is when you notice your co-worker is not performing their job well and you find out that they are going through debt issues or, worst yet, a divorce. Or, when you are in a meeting and one of the attendees appears to be absent-minded and feels slightly embarrassed when they are asked a question. Speaking with them afterward, you find out that they witnessed a horrible accident that day when driving to the office.

Language barriers

This type of barrier is related to the language of the speaker and the listener. Let's say the speaker is a native English speaker whereas the listener is not well versed in English; the speaker might say one thing and the listener may understand it to be something else. This could also be affected by different dialects, so the language may be the same but the dialects may be so different that speakers of one dialect may have a hard time understanding speakers of another dialect.

Semantical barriers

This barrier is related to the meaning and interpretation of words. It's important that the speaker and the listener have the same understanding of the words or terms used. If a school counselor is counseling a student and they say, you need to turn a new page in your life, unless the student knows what that idiom means, the intent of the speaker by using this idiom will be lost. Another common example is the use of buzzwords or phrases, which are especially prevalent in technology. Using the phrase eat your own dog food means using your own software internally that you sell to customers. Unless properly explained, this may sound disgusting and abhorrent to a listener who is not aware of this phrase. That's why you should avoid the use of buzzwords and idioms as much as possible to ensure your listeners are not confused and communication is not lost.

Information overload barriers

This type of barrier happens when the information processing demands placed on a person exceed their capacity to properly process them. Many of us can relate to this when we are getting emails, Slack messages, text messages, and meeting reminders and we feel we are drowning in work. This also happens in projects that are not very well organized and the BA, for example, is sent similar requirements through email, a meeting that just happened, instant messaging, or verbally by someone stopping at their cube for 2 minutes. This can lead to important pieces of information being either dropped completely or mixed up with another requirement.

Cultural barriers

These barriers relate to different cultures, and with increasing diversity in workplaces, you will likely be dealing with people from different cultures. That's why it's important to recognize these barriers to ensure your communication is understood and effective. Some examples of cultural barriers include following strict org hierarchies and reservedly answering questions from your manager's boss because the cultural norm at your previous workplace in a different country was to have all communication upward or downward come through your immediate manager. Another example is how in some cultures, anyone higher than you in rank should be addressed with either Sir or Ma'am. A third example, in some cultures, is that it is rude to make eye contact when talking with the opposite gender, whereas in some cultures, it would be considered rude not to make eye contact and could be construed as lying or trying to hide something.

Now that we have looked at the barriers, let's look at what constitutes effective communication that can be used to mitigate these barriers and have our message easily understood and acted on.

The seven Cs of effective communication

The seven Cs of communication give us a way to effectively communicate and have something tangible to evaluate ourselves on whether we are communicating effectively or not. The seven Cs are also an effective tool to manage the barriers to communication discussed in the last section. Think of these as a checklist to ensure your next presentation, speech, email, or even a simple text message to your kid is understood. Remember, the most important goal of communication is getting your message across to the audience.

Clarity

Have a clear goal and purpose for your message. It can be helpful to think of a quote by Steve Coveys: begin with the end in mind. Think about what you want your listener to do at the end of your message. Given that the human mind can comprehend a limited amount of information at a time (think of the information overload barrier discussed earlier), try to break down your message into parts that the listener can easily understand.

Conciseness

Keep your message concise and short. If you have written an email with 3 paragraphs and 19 lines, could you convey the same message in 1 paragraph and 5 lines? Here are some examples of wordy sentences:

The pandemic caused a great deal of anxiety and frequent mental health problems in society.

The poverty usually suffered by children in inner-city neighborhoods caused Anna to deeply think about how she could change things for the better in her neighborhood.

The following are examples of concise sentences:

The pandemic caused intense anxiety and health problems in society.

Seeing the suffering of kids in inner-city neighborhoods caused Anna to think of ways in which she could improve her neighborhood.

Concreteness

This relates to how precise the message is that is being delivered. Avoiding ambiguity helps your audience understand your message and act on it. This is where you can also provide facts and figures to ensure the message is understood. Take this example: according to the research firm IDC, Salesforce and its ecosystem will create 4.2 million new jobs between 2019 and 2024. Providing this research reference is better than saying there are going to be lots of jobs generated in the Salesforce ecosystem in the next few years.

Coherence

This relates to how well the beginning, body, and ending of a message are aligned with each other. This is probably one of the reasons why communication trickling down from top management in companies is intended to be something else, but by the time it gets to the bottom, it can be easily misconstrued because the original message is not coherent and there are contradictions in it or the ideas are not connected with each other.

Courtesy

Showing respect and being polite in your communications will help you get your message across smoothly and make your audience more perceptive to receive and accept your message. They will be more willing to act on or approve your proposal. Try to be inclusive in your communication and use words that reflect inclusivity and are deemed culturally acceptable.

Completeness

This relates to how far the message is complete. Does it have the information the audience requires to act? Are you jumping from topic to topic, leaving the listener confused on what the expectation is of them?

Consideration

Consider your audience and present your material accordingly. Try to understand your audience beforehand: what are their viewpoints, opinions, or constraints? Try to put yourself in their shoes and try to be empathetic. This can help you craft your message so that it is more easily understood and accepted by your audience. Many people struggle with the right amount of information to present. The ladder of abstraction is a mental framework that can be used to determine the right amount of information to present. As seen in the following diagram, the higher up the ladder you are, the more abstract the idea is, and the lower you are on the ladder, the more concrete the idea is:

Figure 1.1 – The ladder of abstraction

The diagram depicts how you can move on the ladder of abstraction and tailor your message for your audience. Typically, C-level executives want to understand the why and leave the implementation details to their direct reports.

Let's look at an example:

Idea: We want to improve our customer service.Elaboration: Provide an omnichannel experience to customers. Customer Service-Level Agreements (SLAs) must be met and the system should facilitate meeting the SLAs. Product development teams need to know the nature of cases coming in to understand where break/fix efforts should be directed.Features: A feedback survey is sent to the customer regardless of which channel they used to contact support. Cases can be created directly from social media applications such as Facebook. The system should be able to store customer entitlements and send reminders and escalation notifications at appropriate times. Product development teams can run reports and dashboards to gauge the nature of cases being opened and the average time spent on resolving issues.Process: The customer can send an email or fill out a form on the website to create a case. That will create a case in Salesforce and assign it to the relevant team using case assignment rules. The entitlement process assists in ensuring that your team is following a methodical process to solve customer cases and are meeting their SLAs.

We have looked at the seven Cs of communication and the ladder of abstraction to tailor our communication so it is more effective. In the next section, we will look at the path to becoming a data architect.

Becoming a data architect

There can be different paths to becoming a data architect and it is hard to define a fixed set of steps to become one, but the following is the path that most people take on their journey toward becoming a data architect:

Education: A large majority of data architect jobs require a bachelor's degree in computer science or information technology with a focus on courses related to data analysis, data modeling, and analytics. Internships while studying can be extremely beneficial and help in launching your career as a data architect. A master's degree in business intelligence and analytics can further deepen your understanding of the subject matter.Experience