Business Intelligence Career Master Plan E-Book

Eduardo Chavez

• Herausgeber: Packt Publishing
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 447

Veröffentlichungsjahr: 2023

Beschreibung

Navigating the challenging path of a business intelligence career requires you to consider your expertise, interest, and skills. Business Intelligence Career Master Plan explores key skills like data modeling, visualization, warehousing, organizational structures, technology stacks, coursework, certifications, and interview advice, enabling you to make informed decisions about your BI journey.

You’ll start by assessing the different roles in BI and matching your skills and career with the tech stack. You’ll then learn to build taxonomy and a data story using visualization types. Additionally, you’ll explore the fundamentals of programming, frontend development, backend development, software development lifecycle, and project management, giving you a broad view of the end-to-end BI process. With the help of the author’s expert advice, you’ll be able to identify what subjects and areas of study are crucial and would add significant value to your skillset.

By the end of this book, you’ll be well-equipped to make an informed decision on which of the myriad paths to choose in your business intelligence journey based on your skillset and interests.

Leseprobe

Business Intelligence Career Master Plan

Launch and advance your BI career with proven techniques and actionable insights

Eduardo Chavez

Danny Moncada

BIRMINGHAM—MUMBAI

Business Intelligence Career Master Plan

Copyright © 2023 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Kaustubh Manglurkar

Publishing Product Manager: Arindam Majumder

Book Project Manager: Kirti Pisat

Content Development Editor: Manikandan Kurup

Technical Editor: Kavyashree K S

Copy Editor: Safis Editing

Proofreader: Safis Editing

Indexer: Pratik Shirodkar

Production Designer: Joshua Misquitta

DevRel Marketing Coordinator: Nivedita Singh

Cover design: Angel Cristerna

First published: July 2023

Production reference: 1100823

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK.

ISBN 978-1-80107-795-8

www.packtpub.com

To my mother, Irma, who was the main pillar of my growth; Carolina, Ania, and Alicia, my sister and nieces - they are my purpose in this life; Donald Yerbic for all his support; to all the great professors I’ve absorbed knowledge from - Gilberto Alapizco, Roberto Aramburo, Jose Antonio Garcia at Tecnologico de Monterrey, and Yicheng Song, De Liu, and Ravi Bapna at University of Minnesota. Matt Nuttall, Brian Krupski, and Paul Volkmuth for always believing in me. And to my friends, who kept me alive and inspired me during my most challenging times, especially Angel Cristerna, Alejandro Caro, Abdel Vargas, Grace Burgos, and Yuleth, and of course my co-author Danny Moncada.

– Eduardo R. Chavez Herrera

To my mother, Dora, and father, Ariel, thank you for sacrificing so much for Marilyn and me, so that we could have a better life in the US. You have both shown us the true meaning of fearlessness, generosity, compassion, love, respect, and perseverance, and have given so much of yourselves to the world around you. To my sister, Marilyn, and nephew, Luciano, your big bro and tío will always be here for you. To my abuelito Jose, you are in my heart always and forever. To the Moncada and Marin families, you all live in my heart always and forever, and I hope I have made you proud. To my coauthor, Eduardo Chavez, I will treasure our friendship for the rest of my days, and you have my eternal gratitude for this opportunity of a lifetime to co-author this book with you. To all the people in my life who told me my potential was limitless and that I could do whatever I put my mind to, I finally listened. To anyone that offered encouragement or support for this book, this is for you. Y para todos los Colombianos y gente Latina: nunca dejen de brillar que el mundo necesita de su luz.

– Danny Moncada

Foreword

It gives me great pleasure to introduce Business Intelligence Career Master Plan to the world. Written by two seasoned analytics and business intelligence professionals, Eduardo Chavez and Danny Moncada, this book is a well-laid-out roadmap for anyone looking to enter and flourish in the field of business intelligence, or BI as we call it. As we know there is a lot of hype around BI, and many companies struggle to extract meaningful intelligence from their data. A substantial portion of the variance in extracting business value from data is explained by heterogeneity in the people and processes around it. More often than not, I find different employees, different business units, and different teams using different language around business intelligence within a firm. This causes unnecessary friction and leads to suboptimal use of resources. It is also linked to burnout and employee churn.

In simple but insightful terms, Eduardo and Danny lay down the foundation of the field of business intelligence so that there is a common language and understanding that gets professionals to work together to create business value. The book serves as a guide for anyone looking to enter the BI profession and advance through it to other layers, such as analytics and AI. Using real-world examples and stories from their experiences, they showcase essential concepts in a jargon-free manner. Readers of this book can expect to gain an intuitive understanding of key BI skills such as data modeling, data visualization, data analysis, and data warehousing.

I want to congratulate them for putting together this fine career roadmap and I look forward to seeing the book have a positive impact on the BI community.

Ravi Bapna

Curtis L. Carlson Chair Professor in Business Analytics and Information Systems

Academic Director, Carlson Analytics Lab, and Analytics for Good Institute

Carlson School of Management

University of Minnesota

Minneapolis, MN 55455

Contributors

About the authors

Eduardo Chavez is a Business Intelligence professional with over 18 years of industry experience and hails from Mazatlan, Mexico. He is a certified GCP professional data engineer who has extensively engaged with leading cloud platforms such GCP, Oracle, Azure, and AWS. With a bachelor's degree in information systems and three master's degrees, including IT, business administration, and business analytics, Eduardo specializes in SQL, semantic layers, and data modeling. He has worked for prominent private companies like Accenture, Oracle, and Google, as well as the University of Minnesota in the public sector. Eduardo is known for advocating the rapid development cycle, emphasizing late binding data warehousing, rapid prototyping, and a top-down approach.

Danny Moncada is a seasoned Business Intelligence professional with 15 years of experience, excelling in project implementations related to database administration, data analysis, and data engineering. He has contributed his expertise to renowned companies like Cigna Healthcare, Indeed, IOTAS, as well as the University of Minnesota in the non-profit sector. In 2020, Danny completed his M.S. in Business Analytics from the Carlson School of Management, where he was recognized as the most helpful student by his peers. He has achieved numerous certifications, including Python programming from NYU, data engineering, big data, and machine learning on GCP, Google Business Intelligence, and most recently, Google Advanced Data Analytics from Coursera. Danny's specialties lie in data visualization, analysis, and data warehousing.

About the reviewers

Brandon Acosta has been working in the realm of data for seven years now. While data has always been an interesting topic to him, it wasn’t until completing his B.S. in computer science that he learned the real-world value of data. From there, his first several years of experience in the industry were as a business analyst in small and large company settings. This led him to an interest in the engineering aspect of data, where he was then able to pursue a role in data engineering. Although his current title is data engineer, he, like most working on a small data team, covers a wide range of the BI spectrum. As he continues to hone his skills as a data engineer, he hopes to work his way into predictive analytics and begin a role in data science.

Atul Kadlag, a seasoned professional in the business intelligence, data, and analytics industry, possesses diverse experience across different technologies and a proven track record of success. A self-motivated learner, he has excelled at working at various multinational companies for more than 15 years, leading transformative initiatives in business intelligence, data warehouses, and data analytics. Atul has immense experience in handling end-to-end projects in business intelligence and data warehouse technologies. Atul is dedicated to driving positive change and inspiring others through continuous learning, making a lasting impact in the data industry. His expertise involves SQL, Python, and business intelligence and data warehousing technologies.

Chandan Kalra's journey in the business intelligence and analytics industry spans over 20 impactful years, marked by an impressive portfolio of work across ERP, business intelligence, and consulting for Fortune 100 behemoths. His expertise doesn't stop at just knowing the theoretical, but delves deep into the practical. Whether it's dissecting convoluted data or strategizing for multinational corporations, he has done it all. His ability to translate complex data concepts into digestible content comes from years of understanding the subtleties of the business world. His seasoned perspective, enriched by the practical wisdom of handling raw data and converting it into powerful strategic insights, brings a depth of understanding that sets him apart. No doubt, he is a unique guiding voice in the literary landscape of business analytics and technology.

Table of Contents

Preface

1

Breaking into the BI World

Where to start?

BI roles

Problem solving

Specific industry knowledge and subject matter expertise

Communication skills

Statistical analysis

Technical knowledge

Business acumen

Keep up with innovation

Potential entrances to the BI world

BI roadmap

ETL developers

Data architects

Data modelers

BI developers

Data scientists

Technology solutions stack

Non-technical data analysis

Case 1

Case 2

Case 3

Case 4

Case 5

Summary

2

How to Become Proficient in Analyzing Data

Building a taxonomy of your data sources

How to use a BI tool to explore data

Understanding your data needs

Summary

3

How to Talk Data

Presenting data

Know your audience

Choose the right visualization

Keep it simple

Use color and formatting effectively

Provide context

Tell a story

High-level dashboards

Operational reports

Talking to stakeholders

Data visualization taxonomy

Bar charts

Line charts

Pie charts

Scatter plots

Area charts

Heat maps

Bubble charts

Gauge charts

Tree maps

Box plots

Advanced data visualizations

Sankey diagrams

Bullet charts

Taxonomy diagrams

Pareto diagrams

Decision tree for picking a visualization

Storytelling

Summary

4

How To Crack the BI Interview Process

Finding the right interview

Building a business problem and data solutions matrix

Potential business cases and solutions

Teamwork

Customer service

Adaptability

Time management

Communication

Motivation and values

A hypothetical BI interview process and what to expect

General business intelligence interview questions

Scenario-based BI questions

Business case study questions

SQL business intelligence interview questions

Database design business intelligence questions

Python business intelligence interview questions

Summary

5

Business Intelligence Landscape

The current landscape and the most effective technologies to study

Collecting data

Storing data

Cleaning and preparing data

Analyzing data with BI tools

Focusing on user experience

The use of AI

BI developer versus BI architect versus data engineer versus data modeler versus business analyst

Business acumen

Increased productivity

Automation

Templating

Standardization

Monitoring systems

Efficient team management

Summary

6

Improving Data Proficiency or Subject Matter Expertise

Data proficiency

Concluding thoughts

Subject matter expertise

Concluding thoughts

Data behavior in different business units

Data behavior in marketing

Data behavior in sales

Data behavior in finance and accounting

Data behavior in operations

Data behavior in human resources

Data behavior in academia

Analytical thinking and problem-solving techniques

Problem-solving

Leveraging online tools for problem-solving

Web (Google) search

Stack Overflow

AI chatbots (ChatGPT/Bard)

Summary

7

Business Intelligence Education

Academic programs, training courses, certifications, and books

Academic programs

Training courses and certificates

Data architecture books

Data modeling books

Data analysis and visualization books

Summary

8

Beyond Business Intelligence

Business analytics

Cross-pillar reporting

Measuring your BI success

Executive briefing books

Customer-focused processes

Automation

BA final thoughts

Data science

Data exploration

Summary

9

Hands-On Data Wrangling and Data Visualization

Technical requirements

Data analysis using Python

Data wrangling and visualization with Python

Step 1 – loading the dataset

Step 2 – an initial exploration of the dataset

Step 3 – remove duplicate rows

Step 4 – cleaning and transforming data

Step 5 – saving the cleaned dataset

Step 6 – load the cleaned dataset

Step 7 – creating visualizations to summarize the data

Tableau for data visualization

Tableau dashboard hands-on

Summary

Conclusion

Index

Other Books You May Enjoy

Preface

The path to entering the field of business intelligence (BI) is both vast and challenging to navigate. Throughout our experience in various BI roles, we have encountered individuals with diverse skill sets and backgrounds. This spectrum ranges from highly skilled machine learning (ML) and AI engineers holding PhDs in statistics and mathematics to graduate students transitioning from academia to pursue careers as business analysts. Through this book, we aim to convey to you that there is no single predefined path to follow. Numerous different routes can lead to the same destination, depending on your area of expertise, level of interest in various subject areas, and proficiency in both technical and soft skills.

We have two main objectives in this book. The first is to help business professionals understand the foundations of BI in a way that is tool and technology-agnostic. Our second objective is to show the process of starting and advancing in BI. Furthermore, we aim to provide insight into the broader realms beyond BI, encompassing data exploration, business analytics, and data science.

Who this book is for

This book is for BI professionals who are passionate about data and its complexities, have some foundational knowledge of BI, and would like to become more proficient in BI tools and technologies to advance their career in any BI organization.

We extend a warm welcome to readers, analysts, and developers of all skill levels and backgrounds. Whether you are a seasoned ML engineer well-versed in implementing ML pipelines seeking to delve into data visualization, or a business analyst looking to transition into data engineering, we acknowledge and value the individuality and diversity of each reader’s expertise, level of interest in various subject areas, and proficiency in technical and soft skills.

What this book covers

Chapter 1, Breaking into the BI World, offers a general overview of BI, an introduction to BI roles, and an overview of BI technologies and entering the BI space.

Chapter 2, How to Become Proficient in Analyzing Data, explores how to build a taxonomy of data sources, using the right tools to explore data, and understanding an organization’s data needs.

Chapter 3, How to Talk Data, delves into presenting your findings, talking with business stakeholders, data visualization basics, and telling a story with data

Chapter 4, How to Crack the BI Interview Process, examines the tips and tricks to find the right BI interview and techniques to approach common BI interview questions

Chapter 5, Business Intelligence Landscape, looks at the current state of the BI landscape, including common tools, the most effective technologies to study, and the differences between BI roles.

Chapter 6, Improving Data Proficiency or Subject Matter Expertise, explores data behavior in different business units and leveraging online tools to increase and improve BI skills.

Chapter 7, Business Intelligence Education, delves into certifications, academic programs, training courses, and books to continue your BI education.

Chapter 8, Beyond Business Intelligence, examines business analytics, data science, and data exploration.

Chapter 9, Hands-On Data Wrangling and Data Visualization, looks at simple data wrangling, visualization, and dashboard exercises with Python and Tableau.

To get the most out of this book

This book caters to developers and business analysts who possess a foundational understanding of data analysis tools and are enthusiastic about delving into the intricacies of data. It offers an opportunity to enhance proficiency in these tools, thereby enabling career growth within any BI organization. A fundamental grasp of tools such as Microsoft Excel, working knowledge of SQL, Python, and Tableau, and experience with major cloud providers such as AWS or GCP are basic requirements. However, the most crucial prerequisite is the capacity to extract valuable business insights from data and a genuine eagerness to embrace and master challenging new concepts.

Software/hardware covered in the book

Operating system requirements

Python

Windows or macOS

Tableau

Windows or macOS

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Within this book, you will find clear, detailed explanations of vital concepts, along with real-world examples that demonstrate their practical application. By delving into the taxonomy of BI and data within an organization, you will develop a comprehensive understanding of the subject matter, providing a solid foundation for your journey.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Business-Intelligence-Career-Master-Plan. If there is an update to the code, it will be updated in GitHub.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Conventions used

There are several text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “Inspect the first five rows of the dataset using the head() function.”

A block of code is set as follows:

def plot_salary_distribution(df):    plt.figure(figsize=(10, 6))     sns.histplot(data=df, x='Average Salary', kde=True)     plt.title('Distribution of Salaries')     plt.xlabel('Average Salary')     plt.ylabel('Count')     plt.show()

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

Biotech & Pharmaceuticals                  66IT Services                                61 Unknown Industry                           60 Computer Hardware & Software               56 Aerospace & Defense                        46 Enterprise Software & Network Solutions    43

Bold: Indicates a new term, an important word, or words that you see on screen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “ Drag the Longitude and Latitude marks into Columns and Rows, respectively.”

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share your thoughts

Once you’ve read Business Intelligence Career Master Plan, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/978-1-80107-795-8

1

Breaking into the BI World

This book follows the philosophy of the path of least resistance, which states that in nature and physics, organisms and elements always try to find the shortest, most efficient way to their destination.

Imagine that you are standing in front of a large skyscraper in downtown Manhattan called “Business Intelligence.” As you look up at it, you can see floor after floor as it slowly reaches into the clouds and out of your immediate eyeline. While it can be very awe-inspiring and daunting, the building, just like every other one, has many entrances – one entrance at the front, one at the back, and some on the sides of the building. And like many buildings with proper security protocols, each of those entrances require different keys to enter and exit the building. Once you step inside, you will notice right away that there are a myriad of different rooms and workspaces.

Now that we have set the scene, we can begin discussing how this relates to entering the Business Intelligence (BI) world. In this chapter, we will explore entering the BI world (the skyscraper), what background you may come from (the various entrances to the building), what skills you require to be successful (the keys to open the doors), and what career paths or roles you may find once you are (the myriad of rooms and workspaces).

Now, you might be thinking to yourself – how do they even know to let me in the building? How do I prove that I belong and have the capabilities to even get in? The short answer is, you don’t… yet. But our goal is to show you that even if you may not have the “right” title like a PhD in statistics and mathematics (don’t worry, we don’t either), you might have the right skills and experience to get you a key to the kingdom, and you didn’t even know it. How do we start? Well, let’s dive right in, shall we? It’s probably a little cold out there; it is Manhattan, after all!

Where to start?

If you are reading this book, good chances are that you have been exposed to BI or have some notion about its capabilities and benefits, but let’s begin by establishing a common concept – BI is like a powerful telescope that helps a business look deep into its operations and the surrounding market. It’s a set of techniques and tools that turn raw data – such as sales numbers, customer feedback, or supply chain info – into easy-to-understand, actionable insights.

Think of it like this – if a business is a ship sailing in the sea, BI is the captain’s map and compass. It helps the business understand where it’s currently docked, which direction it’s heading in, and what obstacles might be in the way. It does so by analyzing the company’s own data (like looking at the ship’s logbook) and also data from the market (like watching the weather and sea currents).

BI can help answer questions such as, “What products are selling best? Who are our most valuable customers? How can we reduce costs? Which market should we enter next?” This allows businesses to make smarter decisions and better plan for the future. However, embarking on a journey into BI can indeed be a lengthy process. It requires patience and dedication, as the multitude of potential paths to success can sometimes make the journey seem perplexing.

A good starting point is to examine the background of other people that enter and exit the building every day, which is “everyday” BI professionals, such as the two co-authors of this book. What sets us apart from the rest of the population going about their day in downtown Manhattan? While we all come from a diverse set of backgrounds and experiences, there are certain common themes and skills that seem to come up again and again – among them, problem solving, analytical thinking, the ability to think through challenging concepts logically and conceptually, proficiency with computers, communication skills, and business acumen.

Another thing to point out is that many professionals work in different areas of a business before starting in BI. It’s quite common to find accountants, business administrators, marketing experts, and other skilled professionals looking to break into BI because they all have the same goal in common – making their business unit more efficient and data-driven. What makes these professionals similar is they came from a common path, which is a background molded by and refined by time spent working in the business. Accountants spend their days crunching numbers and building reports to deliver to senior leadership, business administrators rely heavily on metrics and key performance indicators (KPIs) to help improve their decision-making, and marketing experts use tracking tools to analyze trends for their company’s conversion rate and cost per sales. What can we infer from the preceding examples? In these business professionals’ search to improve their companies and business units’ goals, they take a winding path up 5th Avenue and through Central Park and then end up at the front door of our imaginary skyscraper, but they are now ready to give BI the chance to not only be leveraged as a tool for their organizations but also, even more critically, become a possible career progression for themselves.

At the other end of the BI spectrum, we have IT professionals, composed of programmers, server administrators, QA testers, database developers, and administrators, among many others, whose relationship to data and technology is not casual but much more intimate and nuanced. IT professionals have in-depth knowledge of the ins and outs of how data flows through a building; like plumbers working at a water treatment facility, they very carefully calculate and monitor the flow of data as if it were water going through pipes. Programmers understand and adhere to strict computing principles and understand the importance of CPU usage when developing programs. Server administrators design systems to minimize system bottlenecks and maintain a secure platform for their customer base. Database developers have expertise in handling tables with millions of records, updated at different intervals, and experience using a wide-ranging collection of database technologies. These potential candidates take a different path through lower Manhattan and enter the skyscraper through the back door, preferring to dedicate their careers to BI from a technical point of view.

Now that we’ve covered the front and back entrances to BI, you might be asking, what are the side doors to enter the building? For us, BI as a career represents a commitment to concepts and technologies that take time (sometimes years) to master and may not be in line with the career you originally chose or your original career path. Here’s an example – let’s say you are the Chief Medical Officer (CMO) at a hospital in a major metropolitan area, such as Chicago, and your HR generalist has made you aware that the hospital faces a major challenge in scheduling enough qualified nurses to cover all the shifts required to treat patients throughout the day. The most experienced nurses with the most amount of medical training are allocated the night shift, which typically does not have as much activity as during the day. Your suspicion is proven right by reviewing historical trends of the number of lives saved when the hospital had the proper coverage for day shifts. If you, as the CMO, had the right data, the right BI platform, and experienced IT professionals, you might be able to obtain some answers – but what if you could combine that with your domain expertise and years of experience leading the hospital? With your knowledge, a more accurate optimization algorithm could be built to allocate nurses accordingly. However, does this mean you want to become a BI analyst for life? These are the side doors of BI, and we’ll discuss more about BI as a tool in the upcoming Non-technical data analysis section.

As we mentioned and demonstrated previously, there’s no one way to enter the BI world. In fact, there are so many different possible paths that it can almost seem like a maze, and you end up feeling like a business professional getting lost in downtown Manhattan on the way to an important client visit. While it can feel like organized chaos at times, there’s also a great deal of opportunity for growth and career progression, and the barrier to entry is much lower than in other professions, such as medicine or law. What we hope to demonstrate is that having a willingness to learn and master some basic skills and concepts can lead you not only to the skyscraper but also through the long and winding journey up to the top. So, let’s get started by diving into what those skills entail and what tools you’ll need in your BI journey.

BI roles

As you might have surmised, there’s a variety of different BI roles; what those roles have in common is the need to have some polished skills if you want to perform and be successful at them. It is important to assess which of the following skills you may have, and be honest in your assessment of your proficiency in each skill. We would also like to emphasize that the skills we cover here are not meant to be viewed as an exhaustive list, but we’ve specifically picked the set of skills we would like to emphasize because they have the biggest impact on any BI organization. We encourage you to keep track of and continually practice and hone these skills, as they require dedication and focus to keep sharp.

While some of these skills require years of training or years of practice to fully master, there are a few where the length of time isn’t as important as the use of your brain agility. One such skill is data analysis, which you can develop through different mental exercises – that is, using your imagination to try to picture and create mental visualizations of data. In BI organizations, professionals are often encouraged to use different data visualization tools and perform exploratory data analysis at their own discretion and direction. However, being able to identify how your organization’s data is structured and then determining what the best approach is to shape the data to the appropriate dimensions, level of granularity, and what its final “shape” looks like will help you when defining the appropriate business requirements, or when writing the correct algorithm/query that will produce the final output you need.

Here are some tips on how to develop proficiency in data analysis. These are not meant to be taken in any special order, and nor should this be viewed as a comprehensive list:

Start by asking some simple questions to determine the purpose of your data. Here are some sample questions to think through:What does the data describe?What entity does it represent?What relationships are present between data points (if any)? What types of relationships are being captured?What field or fields are the individual pieces that make every record unique?Can you visualize those fields and ascertain their relationship with the rest of the columns? Do they matter?Is there any “white noise” or random variance present in the data?Is there any metadata available for analysis?Can you determine the purpose of the data?How and where was the data created? Gather data about the systems, servers, databases, and so on.In what format is the data being stored?Follow this up by wrapping some general statistics around your data. You might try building commonly used, simple data visualizations as a starting point. For example, you can use techniques such as the following:Building a bar chart or histogram for discrete dataCreating histograms to determine the distribution of continuous dataUncovering patterns or trends using a scatterplot or heatmapCreating box plots containing mean and percentiles, and adding KPIs with metrics such as standard deviation, mode, max, and minsOur final suggestion is to do a “deep dive” and systematically pick apart a few records from your dataset. For example, you could start by finding out how many distinct values each variable in your dataset has to determine how large of a distribution you are working with. You could follow this up by grouping the most important columns in the dataset and determining the meaning behind the values they store. To provide a little more context, here’s an example. Let’s say your dataset has a column labeled STATUS that contains four unique values, [P0, P1, P2, P3], and another column labeled IS_CURRENT, with two distinct values of [1, 0]. You might select one row of data containing STATUS of P1 and an IS_CURRENT value of 0, and try to figure out what information this conveys. Then, get confirmation from a subject matter expert in your organization.Finally, you can take one additional next step and analyze a few rows of data (or maybe just one row, depending on the complexity of the dataset). Take the time to trace the route that a single record takes along your dataset and find out where it ends. Once you have that route mapped out, use the lessons learned from your initial exploration of the dataset combined with your intuition to narrate the story of that single record. The goal of going through this extensive exercise is to extrapolate your analysis to a much larger dataset and, with the help of technology, apply algorithms that perform the same steps you took on a localized dataset.

Problem solving

As a BI professional, having the ability to solve problems quickly and creatively is an incredibly important skill to successfully carry out complex BI projects. In our estimation, this is one skill that is the thread that ties all of the others together. Problem solving has many components to it, and the first involves the ability to identify and understand both potential problems and areas of improvement, and then couple that with developing and applying practical, cost-effective solutions based on data analysis.

Additionally, problem solving requires critical thinking, an ability to evaluate alternatives, and the courage to implement creative solutions quickly and effectively (especially if the problem is time-sensitive!). Problem solving also requires the ability to adapt to changing situations and make quick, effective decisions in an uncertain environment. Problem solving also requires the ability to collaborate with other departments and make data-driven decisions.

In the context of BI, it’s important to combine that problem solving mindset with advanced data analysis and machine learning techniques to help you identify patterns and trends in large datasets, using the insights gained from that analysis to make informed decisions and improve operational efficiency and the effectiveness of business processes.

To summarize, problem solving is essential for success in challenging BI projects and requires a combination of technical skills, leadership, and critical thinking.

To help you develop your problem solving skills, we suggest trying the following in your day-to-day roles:

Practice makes perfect: Take some time in your week to solve problems regularly to improve your skills (e.g., a crossword puzzle, Wordle, or a programming challenge)Break down complex problems: Divide large-scale, wide-reaching problems into smaller, more manageable parts firstSeek different perspectives: Work on the problem from different angles and assess a variety of solutionsCollaborate with others: Work with others to gain new insights and come up with innovative solutionsThink critically and creatively: Try to develop unconventional solutions and “think outside the box”Learn from failure: Reflect on mistakes and failures and use them as opportunities for growth and improvementInform yourself: Stay updated on new developments and the best practices in your fieldBe patient and persistent: Remember that developing effective problem solving skills takes time and continuous effort

Specific industry knowledge and subject matter expertise

There is no denying that domain expertise – that is, knowledge of a business – is the light that guides you in the dark. Subject matter experts are the ones that know what to look for and what to measure. For all of you who have no technical background but a great deal of business knowledge, you are also a great candidate to become proficient in BI, since you are capable of defining metrics that would make a business area, organization, or even an industry perform well. You also would be able to ascertain how data looks to make correct decisions. Learning about the specifics of a particular industry is out of the scope of this book, but we encourage you to evaluate yourself if you consider you have the required experience in your particular field. The more extensive your domain/industry knowledge, the greater the benefit for you as a BI expert.

Communication skills

Communication skills are extremely important in the context of BI. BI involves gathering and analyzing data to inform business decisions, and effective communication is crucial to convey the insights and recommendations that result from this analysis. Some of the skills you will need to develop in order to succeed in communicating about BI projects are as follows:

Clear presentation of findings: BI professionals must be able to clearly present complex data insights and recommendations to both technical and non-technical stakeholders in a way that is easily understandable and digestibleCollaboration: BI often requires collaboration with cross-functional teams, and effective communication skills are necessary to ensure that all stakeholders are on the same page and work toward a common goalInfluencing decision-makers: Effective communication is crucial in convincing decision-makers to adopt recommendations based on data insightsBuilding relationships: Strong communication skills can help BI professionals build strong relationships with stakeholders, which can be valuable in obtaining buy-in for their recommendations and insights

Communication skills play a critical role in the success of BI initiatives and can greatly impact the effectiveness and impact of data-driven decision making.

To help you enhance your communication skills, we suggest attempting the following techniques:

Practice active listening: Give your full attention to the speaker, ask questions for clarification, and avoid interruptingSpeak clearly and concisely: Use simple language, avoid technical jargon, and get to the point quicklyImprove your writing skills: Write clearly and concisely, and proofread for clarity and accuracy (leverage technology wherever possible, making sure to use your word processing software’s built-in spelling and grammar check)Learn to adjust your communication style: Adapt your communication style to different audiences and situationsSeek feedback: Ask for feedback from others on your communication skills, and be open to constructive criticismUse effective body language: Be mindful of your body language because non-verbal communication can greatly impact the effectiveness of getting your point across, so use appropriate visual cues to help reinforce your messageTake courses or workshops: Consider taking courses (online or in-person) or workshops to improve your communication skills, such as public speaking or persuasive writingObserve and learn from others: Pay attention to how great communicators deliver their message effectively, and try to incorporate what you learn into your own communication style

Developing strong, effective communication skills takes time and effort, so be persistent and keep practicing.

Statistical analysis

We may not need to be experts in statistics, and we do not need to write statistical formulas, because there are many available tools that will take care of that. However, as BI experts, we are required to know how and when to apply it. Statistical knowledge can be an advantage for BI because it provides a way to analyze data and make informed decisions based on that analysis. Statistics can help identify trends, patterns, and relationships in data that may not be immediately apparent. By using statistical methods, businesses can make predictions, test hypotheses, and assess the impact of decisions. This allows them to make data-driven decisions that are more likely to lead to success, rather than relying on intuition or guesswork. In short, statistical knowledge is a crucial tool to turn data into actionable insights in BI.

Becoming proficient in statistical analysis requires a combination of learning, practice, and experience. Here are some steps you can take to improve your skills:

Learn the fundamentals: Study basic statistics concepts such as probability, descriptive statistics, inferential statistics, hypothesis testing, and regression analysis. There are many resources available, including online courses, textbooks, and educational websites.Practice with real data: Gain hands-on experience by working with real datasets and applying statistical methods to answer questions and make predictions.Use software: Familiarize yourself with statistical software such as R or Python, which can automate many calculations and help you visualize your results.Collaborate with others: Join a study group, or seek out a mentor who can help you deepen your understanding and provide feedback on your work.Stay up-to-date: Keep learning and reading about recent developments in the field of statistics and its applications. Attend workshops and conferences, or take online courses to stay up to date with the latest techniques and tools.

It’s also important to note that proficiency in statistical analysis takes time and consistent effort, but with dedication and practice, you can become a skilled statistical analyst.

Technical knowledge

As the fishermen and explorers of the New World found out through various trials and tribulations, you cannot traverse the ocean without a proper boat and knowing how to navigate choppy waters. Similarly, when dealing with treacherous, murky oceans of data, the use of dependable tools is encouraged and critical to quickly turn around business problems. In this context, dependable tools are the ones that facilitate and speed up the process of analyzing data, since computers are faster at processing data than humans. This book will mention some (both authors agree that it is in the best interest of our readers to be as tool-agnostic as possible) of the tools and the basics to use them; however, we want to reiterate the importance of taking time to research and find which tool is right for your organization’s needs.

At the time of writing this book, these are examples of tools used in BI:

To move and transform data based on data modeling rules and designs, these are recommended:DataStageInformaticaFivetranData FactoryDatabase Migration ServiceApache AirflowStitchTalenddbtThere are many tools available for modeling data and governance. Here are a few:ER StudioErwinEnterprise ArchitectParadigmIDAPowerDesignerTerraformDBeaverLucidchartDbSchema ProSome of the main vendors of data visualizations and enterprise solutions for BI are as follows:TableauPower BILooker StudioSupersetQuickSightQlikThoughtSpotMicroStrategySisenseOracle AnalyticsSAP AnalyticsYellowfinBusiness ObjectsCognos

As you can see, there are numerous tools, software, and options to start your journey in BI; for any one of these examples, a programming language such as SQL, Python, or R will complement your technical expertise by a considerable margin. Another thing to keep in mind is that many of these tools (especially any software that is open source) offer a free version you can play with. It is important to learn them and be able to use them properly. Take time to train on your preferred tool, as only practice can help you become proficient and pave the way to start working on real-life projects.

Business acumen

Business acumen refers to a person’s understanding and knowledge of how businesses operate and the factors that contribute to their success. This includes an understanding of finance, strategy, market trends, and the competitive landscape.

Business acumen is important for BI because it allows individuals to put data into context and make informed decisions that are aligned with the overall goals of an organization. With a strong understanding of business operations and trends, individuals can better understand what data is relevant, what questions to ask, and how to interpret the results of their analysis.

Moreover, business acumen helps individuals to use data in a way that supports a business strategy, rather than just generating reports and visualizations. They can also understand how their insights can be leveraged to improve business performance and drive growth.

Business acumen is a crucial component of successful BI, as it enables individuals to turn data into actionable insights that support the success of an organization.

Keep up with innovation

Always innovate, research, and keep up to date with the latest technology, and get trained in it. Find out what the main issues in your BI world are and find tech that solves them. Automate and learn how to be lazy while productive. Don’t implement new technologies just for the sake of innovation but because they really solve an existing problem, and always monitor their return on investment.

Potential entrances to the BI world

Now that you have some idea of what skills and technologies are important, you might still be asking yourself, how do I enter the BI world? We’re ready to examine some plausible scenarios and find possible entrances to that skyscraper we described at the beginning of the chapter. Please keep in mind these are some ideas on how you could potentially approach starting a BI career given different situations.

Let’s say you’re an undergraduate student studying computer science, getting exposure to operating systems, database theory, and artificial intelligence for the first time. As someone just learning how to navigate the environment, you may be torn between the different disciplines and systems (i.e., networks, servers, programming languages, and relational database management systems (RDBMSs)). In this scenario, you might think that a good first step would be to get an internship within a BI department at a large multinational corporation to start getting a feel for how computer science is used in day-to-day operations. However, competing for a coveted internship at a highly sought-out company is sometimes a difficult and risky proposition without a guarantee of success. With that in mind, we surmise that a good starting point would be to learn how to extract, transform, and play around with data using a service such as Kaggle, or start with a simple approach, gather data around your life, make sense of the data, and keep looking for patterns. That being said, if you do manage to get an internship that might be non-BI related, you can still follow a similar process of collecting and analyzing statistics about the process taking place in that area. You could try doing it at a frequency of time that makes sense – that is, taking monthly snapshots of KPIs to measure a business unit’s performance. Chart your results and keep track of trends.

Now, let’s imagine that, instead, you are a business school graduate with several years of experience and an intermediate level of proficiency as a financial analyst, but your technical knowledge and skill set still needs some refinement. If you already have the ability and know-how to produce reports, then start gathering information on the critical KPIs for your business unit while keeping a monthly snapshot of these results (and, if possible, compare year over year). By creating these metrics on your own, you are able to showcase your data analysis capabilities and then potentially start working on your own BI projects. If there’s not much chance to get hold of data, think about indicators of the actual work your team does – for example, how many activities each one of your team performs, or what the usual duration it takes for your team to finalize a project is. Find out more about your customers, internal or external, and engage with them to obtain some of this information.

If you are an IT professional, with a computer science background, you may find yourself programming an interface; you may need to understand how an application will store the data, or see whether there are any database rules or constraints your interface needs to enforce on the user. In cases like these, you learn the schematics of the data model, how the data flows, and how to better present it to the user. This is a great window for you to enter into the BI world. If real data is restricted to you, think about usage monitoring; you may want to track how much CPU, memory, or storage your application consumes and then enhance it accordingly. Along with that, tracking your users’ peak hours and click activity could be a great source material for amazing analysis. This example applies to database administrators, database developers, server administrators, backend programmers, and so on. As long as you have the desire to measure performance, you will be able to find a window to enter into BI.

You may be a nurse, a teacher, a logistic coordinator, or a salesperson; you can always start your BI journey by measuring events and keeping track of those measures by any given frequency. What are the most critical times in the day, week, and month of patient traffic in a year? What subjects are students more proficient in? Which cost centers are slower to process input? Who are your most responsive customers? Understanding what the different processes involved in a BI project are can help you find out not only how to implement them in your current organization but also which ones align more with your skills, and which ones you will have to train yourself on.

BI roadmap

During a BI project, you will find a list of activities ordered almost in a sequential way, representing the flow of data throughout a pipeline. At the same time, these activities may belong to specific roles of their own; they are highly technical, and the use of modern tools is almost mandatory. Use this section of the book as a guide to position yourself and your skills along this roadmap. However, as with any other roadmap, you may want to go through all of the steps, but depending on your preferences, some of these may not be appealing to you. Skipping them is always an option, as the objective of this book is for you to find your path in the BI world.

The reason it is possible to skip some of these steps is due to the nature of the purpose. Some are a means to an end, meaning they are a necessary evil; if we could, we would avoid them altogether but, there are many impediments – computing power, memory, storage, even the laws of physics. Let’s examine them one by one.

In general, the data we work with in BI comes from different sources, and transactional systems are the most common ones in an organization. A company may have an enterprise resource planning (ERP) system on which all transactions are created, involving internal cost centers and several business processes. All these transactions translate into computing costs in a database. The database can only handle so much when trying to input every single transaction happening across a company. Depending on the size of the company, this can scale up to unmeasurable amounts. Imagine for a second that while you try to read all that information and create reports and dashboards, thousands of employees enter information in the same database. The servers holding the database would collapse, and you may cause a lock on the database, halting all activity until the database administrators take care of it. It would be like trying to read a book at the same time the author is writing it. Because computers struggle to handle both writing and reading data simultaneously, ETL was born.

ETL developers

ETL, which stands for extract, transform, and load, is a data integration process that combines data from multiple sources into one data store, which typically gets loaded into a data warehouse. With the wide-scale adoption of cloud technologies, companies are taking advantage of the increased processing power, which is shifting towards ELT. However, either process has the same basic flow – a scheduled task/job will execute and extract a desired portion of data, then the data is transformed based on some logic (business requirements) that has been defined beforehand, and finally, it is loaded into a database, where the sole purpose is to allow analysts and developers to produce reports and dashboards. Generally speaking, this database is called a data warehouse, and it is typically large enough to hold all of a company’s analytics sources. In the BI world, the responsibility for building these data pipelines may fall upon an ETL developer or data engineer, depending on the company.

Usually, an ETL developer would have to master SQL or, sometimes, a procedural SQL-based language such PL/SQL, T-SQL, or a programming language such as Python, Scala, or Java. The ETL developer may use tools such Talend, Informatica, or Fivetran. The ETL developer gathers requirements from business customers or data modelers and, after assessing the source of the data, analyzes the desired output and builds an algorithm or a process that will extract the data, performs the proper transformations, and then loads it to where the data will be consumed. Sometimes, this data may reach its final destination further down subsequent pipelines, depending on the needs of the projects and the architecture of the platforms.

Data architects

One part of working with tools is having an infrastructure that can hold and scale a workload derived from processing data. The main role of a data architect is to design and oversee the overall data strategy and architecture of an organization. This includes defining data structures and systems, ensuring data security and integrity, and ensuring the efficient and effective use of data to support business objectives. Data architects also work to align an organization’s data strategy with its overall business strategy, and they collaborate with stakeholders from various departments to understand their data needs and requirements.

Data modelers

Gathering data needs and requirements is the art of translating business needs into computational parameters. Sometimes, this role is played by data modelers or even data analysts; they produce the necessary steps that developers need to take in order to generate the desired output the business requires, and they are responsible for the schematics and blueprints of data structures and the relationships between them. A data modeler is responsible for creating, maintaining, and managing conceptual, logical, and physical data models for an organization. This involves analyzing the data requirements of various business processes, defining data elements and relationships, and creating visual representations of the data structure.