Managing and Visualizing Your BIM Data E-Book

Ernesto Pellegrino

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 461

Veröffentlichungsjahr: 2021

Beschreibung

Business intelligence software has rapidly spread its roots in the AEC industry during the last few years. This has happened due to the presence of rich digital data in BIM models whose datasets can be gathered, organized, and visualized through software such as Autodesk Dynamo BIM and Power BI.
Managing and Visualizing Your BIM Data helps you understand and implement computer science fundamentals to better absorb the process of creating Dynamo scripts and visualizing the collected data on powerful dashboards. This book provides a hands-on approach and associated methodologies that will have you productive and up and running in no time. After understanding the theoretical aspects of computer science and related topics, you will focus on Autodesk Dynamo to develop scripts to manage data. Later, the book demonstrates four case studies from AEC experts across the world. In this section, you’ll learn how to get started with Autodesk Dynamo to gather data from a Revit model and create a simple C# plugin for Revit to stream data on Power BI directly. As you progress, you’ll explore how to create dynamic Power BI dashboards using Revit floor plans and make a Power BI dashboard to track model issues.
By the end of this book, you’ll have learned how to develop a script to gather a model’s data and visualize datasets in Power BI easily.

Leseprobe

Managing and Visualizing Your BIM Data

Understand the fundamentals of computer science for data visualization using Autodesk Dynamo, Revit, and Microsoft Power BI

Ernesto Pellegrino

Manuel André Bottiglieri

Gavin Crump

Luisa Cypriano Pieper

Dounia Touil

BIRMINGHAM—MUMBAI

Managing and Visualizing Your BIM Data

Copyright © 2021 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Pavan Ramchandani

Publishing Product Manager: Kaustubh Manglurkar

Senior Editor: Hayden Edwards

Content Development Editor: Aamir Ahmed

Technical Editor: Saurabh Kadave

Copy Editor: Safis Editing

Project Coordinator: Rashika BA

Proofreader: Safis Editing

Indexer: Subalakshmi Govindhan

Production Designer: Shyam Sundar Korumilli

First published: November 2021

Production reference: 1251021

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80107-398-1

www.packt.com


To my partner in life, Vanessa De Vivo, and my sister, Alessia Pellegrino, for their love and support.

– Ernesto Pellegrino

Contributors

About the authors

Ernesto Pellegrino has background experience in computer science, start-ups, and web application development. He graduated in architecture from Roma Tre University. During the last 5 years, companies such as Microsoft and Autodesk have recognized him for contributing to their R&D projects. Also, Ernesto has been a speaker at national events dedicated to design technologies several times. The connection between computer science and academic knowledge has led to him being head of R&D in one of the most well-known Italian architecture and engineering firms, La SIA SpA. With BIM expertise and computer science skills, Ernesto is working on several projects implementing top-notch digital processes and technologies.

I want to thank the people who have worked closely with me and supported me during the book's development. Starting with the Packt team, Aamir, Kaustubh, Hayden, and Divij, who guided and advised me the whole time, thank you guys for your precious help. Vanessa De Vivo and Alessia Pellegrino, my partner in life and my sister. They overviewed the entire project from the beginning, helping me every time I needed it. Manuel, Luisa, Gavin, and Dounia, thank you all for your participation and support. Without all of those people, the book couldn't have been completed. One more thing, if you liked the illustrations in the book, you can get in touch with Vanessa on Instagram @sparkle_rabbit_art!

Manuel Andrè Bottiglieri is a full stack BIM developer at Lombardini22. Having graduated in architecture, he works as BIM developer and system integrator in the research and development team at Lombardini22. He deals with information modeling, custom APIs, cloud services, and full stack development to process, analyze, and share data through digital models. He is an enthusiast of coding, IoT, big data, and business intelligence, and is a member of the BIM User Group Italy (BUG Italy).

Gavin Crump is the owner of, and a BIM consultant at, BIM Guru. Having worked in many architectural firms over the past 10 years as a BIM manager, Gavin opened his BIM consulting firm, BIM Guru, at the start of 2020. Launching off the back of his YouTube efforts on the Aussie BIM Guru channel, his focus has shifted toward educating all levels of AEC professionals about software-based workflows, computational design, and how they can navigate the industry from a BIM perspective.

Luisa Cypriano Pieper is BIM manager at CES – Contracting Engineering Services. She is an Italian-Brazilian architect with a passion for automation and process improvement. Luisa has worked in offices in various sectors in Brazil, Italy, Belgium, and Spain. She was previously a BIM manager and BIM consultant before recently becoming head of the BIM management team. In this position, her responsibilities are implementing innovations and managing workflows across the multinational company CES, always looking from a BIM perspective.

Dounia Touil is an architect and design technology specialist at NSW Arkitektur. She leverages BIM and design technology to provide professionals with methods and practices to improve project outcomes. With a background in architecture, she has worked in France and Norway in all phases of building design and construction. Over the years, she has been part of large and complex projects in the commercial, residential, office, and transportation sectors. Her current focus is on design automation, data visualization, and software interoperability.

About the reviewer

Jisell Howe is an experienced technologist with expertise in CAD/BIM management, data visualization, software support, process management, and project management. Howe has a hybrid manufacturing and design background where communicating the data story is imperative for success with various stakeholders and clients. She holds a Bachelor of Science degree in applied management and an Associate of Applied Science degree in architectural drafting and estimating from Dunwoody College of Technology in Minneapolis, MN.

Table of Contents

Preface

Section 1: Overview of Digitalization and BIM Data

Chapter 1: Introducing Units of Digital Information

Exploring the beginning of the digitization era

Learning how simple digital data is

Getting to know types of digital data

Understanding how much data we produce

Learning about hybrid jobs

Summary

Chapter 2: Understanding Data Centers

Understanding data centers

Learning the different types of cloud solutions

Introducing the first data center in history, ENIAC

Learning about The Citadel and Project Natick

Getting started with Power BI

Setting up Power BI and exploring the World Happiness Report

Creating your first dashboard

Customizing and filtering the chart

Summary

Chapter 3: Warming Up Your Data Visualization Engines

Getting started with data visualization

Scatter plot

Column chart

Bar chart

Stacked bar chart

Line chart

Pie chart

Radar chart

Waterfall chart

Why analyzing BIM data is important

Exploring Microsoft Power BI charts

Downloading and importing the dataset

Formatting the other values

Creating the column chart

Creating a waterfall chart

Creating a line chart

Summary

Section 2: Examples and Case Studies from Experts around the World

Chapter 4: Building a Data Bridge between BIM Models and Web-Based Dashboards

Technical requirements

Installing Visual Studio Community

Identifying the problem

Creating your first Revit plugin

Understanding the environment

Configuring dotnet framework and PresentationCore.dll

Installing the Revit dll

Updating your local repository

Updating the RibbonBar.cs file

Testing the plugin startup

Exploring the toolbar

Preparing Microsoft Power BI

Building the Power BI report

Visualizing the data in Power BI

Publishing and sharing your final report

Summary

Chapter 5: Getting Started with Autodesk Dynamo and Data Gathering

Technical requirements

Understanding visual programming

Introducing Autodesk Dynamo BIM for Revit

Understanding the Dynamo UI and its basic components

Menus

The upper-right corner

Library

Execution bar

Nodes

Node states

Types of data

Setting up the environment

Exporting data from Revit

Updating data in Excel

Importing data back into Revit

Summary

Further reading

Chapter 6: Importing Revit Plans in Power BI Using Shape Files

Technical requirements

Setting up Dynamo

What are shape files?

Writing our data sources for Power BI

Understanding the script UI window using the Data-Shapes package

Collecting and filtering all the Revit rooms

Exporting room data to Excel

Processing room boundary geometry

Generating the Shape file

Manipulating the shape file to include metadata

Setting up Power BI with the Synoptic Panel

Visualizing our data in Power BI

Summary

Chapter 7: Creating a Revit Model-Check Dashboard Using Dynamo and Power BI

Technical requirements

Identifying what data to track in Revit

Criteria

Potential data to track

Collecting and exporting data using Dynamo

Collecting data

Dynamo script

Exporting data

Building the Power BI dashboard

Importing Excel data

Creating visuals

Inserting a project illustration

Exporting and sharing the dashboard

Updating the dashboard

Summary

Section 3: Deep Dive into Autodesk Dynamo

Chapter 8: Deep Dive into Dynamo Data Types

Technical requirements

Introducing lists and indexes

How to select elements

How to use levels to select elements

Learning IF statements and variables

Concatenating strings

Working with strings (searching and replacing)

Getting to know regular expressions

Summary

Chapter 9: Using Dynamo to Place Family Instances

Technical requirements

Creating the first family placement script

Creating the second family placement script

Environment setup

Introducing the second script

Placing a family instance on each window

Summary

Chapter 10: Gathering a Revit Model's Data from Multiple Models at Once

Technical requirements

Creating the data collector script

Collecting information from multiple models

Collecting information about Rooms

Collecting data from Levels

Collecting information from Sheets

Adding the filename to each list

Exporting the datasets to Excel

Adding the user interface

Summary

Further reading

Chapter 11: Visualizing Data from Multiple Models in Power BI

Technical requirements

Importing and formatting the datasets

Importing the datasets

Formatting the datasets

Creating the charts

Creating the table chart

Creating the stacked column chart

Creating the donut chart

Creating the scatter plot chart

Creating the map

Summary

Chapter 12: Having Fun with Power BI

Technical requirements

Building the form

Understanding how to publish the dataset

Connecting the cables on the Power BI side

Completing the Google form

Importing the data and creating the charts

Importing the dataset

Creating the charts

Summary

Other Books You May Enjoy

Preface

The book aims to give practical skills to those working in the AEC field who want to learn data visualization skills. Each chapter will focus on a specific subject to help you to master the use of Autodesk Dynamo and Power BI to gather, manage, and visualize BIM data. Along the way, we will talk about various IT subjects that are fundamental to understand for you to grasp what is going on behind the scenes.

The book is divided into three sections. The first one is a bit more theoretical. The second one showcases examples from colleagues worldwide of how to manage and visualize a model's data. The third section is all about creating scripts and visualizing data. Throughout the book, you will also see a lot of memes that have been hand-drawn by myself and my partner in life, Vanessa De Vivo, who works as a freelance illustrator. Those memes will break up the book's seriousness from time to time and hopefully make you smile while learning.

Who this book is for

The ideal reader is a professional working in the AEC field with a background in BIM processes and modeling. I am talking about BIM managers, BIM coordinators, BIM specialists, design technology professionals, and all professionals who want to start analyzing the data of their projects. You will learn about how to approach the data management side of BIM, the most common workflows, and the best tools available, both online and offline.

What this book covers

Chapter 1, Introducing Units of Digital Information, starts by providing information on the history of digital data units of measurement. The chapter focuses on introducing you to the world of digital data without talking about BIM, talking instead about data in general. I will refer to other industries and describe why everyone is digitizing and why digital information is so valuable.

Chapter 2, Understanding Data Centers, talks about everything to do with data centers. Cloud technologies, data structures, and data centers: what are they, and why are there are more and more data centers being built worldwide every year? Then, I'll talk about two exciting projects. One is the Switch data center, which is the largest globally, and then I'll talk about Project Natick, which is interesting because it will be built beneath the ocean! After this overview on data centers, I'll talk about how to get started with Microsoft Power BI, giving you just a bit of theory and a well-organized exercise to familiarize yourself with the data analysis tool.

Chapter 3, Warming Up Your Data Visualization Engines, introduces you to some of the most common chart types. We will explain what they are, how they are used, and how to achieve the best results. After that, I'll talk about the benefits of analyzing BIM data, giving a general introduction to the subject. Toward the end, we'll get back to some practical exercise with Power BI, this time with a new, more complex dataset. The goal is to allow you to develop, using Power BI, the chart types discussed at the beginning of the chapter.

Chapter 4, Building a Data Bridge between BIM Models and Web-Based Dashboards, continues on the last subject of the previous chapter, real-time data. This chapter is perfect for continuation because it shows an excellent example of real-time streaming data from a BIM model made with Autodesk Revit to a Power BI dashboard. The goal is to build a simple Revit plugin using Visual studio. You will learn more advanced computer science topics. Even if you don't want to learn programming in the future, you still need to use such skills to develop other, more complex Autodesk Dynamo scripts.

Chapter 5, Getting Started with Autodesk Dynamo and Data Gathering, is dedicated only to understanding the fundamentals of Autodesk Dynamo. I wrote this one with Luisa Cypriano Pieper. We first cover the theory, introducing you to the world of visual programming and giving you a bit of history and context. Then, we deep dive into the core features of Autodesk Dynamo, exploring the UI, the library, the menus, and other options. Next, we will create two scripts in Autodesk Dynamo. The first one will export data from the Revit model to an Excel spreadsheet. The second one will import an updated version of that data back into the Revit model.

Chapter 6, Importing Revit Plans in Power BI Using Shape Files, focuses on Power BI and how to import a Revit floor plan. Having learned the basics of Autodesk Dynamo, you will now be ready to go another step up on the Dynamo skills ladder. You will follow a step-by-step guide to create a script in Dynamo that enables the use of Revit floor plans inside Power BI. This is an advanced workflow, and it aims to show you that mastering Autodesk Dynamo BIM is fundamental nowadays in our AEC industry. This workflow will help many of you stand out and offer something extraordinary and advanced to your colleagues and managers. By the end of this chapter, you will be a lot more confident using Autodesk Dynamo. 

Chapter 7, Creating a Revit Model - Check Dashboard Using Dynamo and Power BI, is the last chapter written by the experts. Here, Dounia will showcase a workflow on creating an auditing model dashboard using Revit, Dynamo, and Power BI. These kinds of dashboards are helpful to check a model's integrity and issues. You will follow a step-by-step guide to create a Dynamo script that gathers all the necessary data from the Revit model. Next, you will learn how to make a dashboard and take advantage of that data. As in the previous chapter, we use some additional packages inside Dynamo to create shape files to visualize and interact with the Revit model view in Power BI.

Chapter 8, Deep Dive into Dynamo Data Types, is all about the more complex Dynamo data types. It is a computer science subject, a fundamental one, and relatively simple to learn. I will pick up from the topics in Chapter 5, Getting Started with Autodesk Dynamo and Data Gathering, where we introduced primitive data types, such as strings and integers. Then, we will continue that subject by adding more complex data types. I am talking about lists, variables, objects, arrays, and others. Along the way, I'll give examples and references to make sure you are comfortable with the subject. This is important to help you build more complex scripts using Autodesk Dynamo.

Chapter 9, Using Dynamo to Place Family Instances, is a fun chapter. After a bit of theory and hands-on experience, now is the time to create a script using what you will have learned in all previous chapters, especially Chapter 8, Deep Dive into Dynamo Data Types, where you will have learned how to start working with strings, variables, and if statements. In this chapter, you will create a script that places family instances (digital objects such as doors, chairs, or lamps) inside a Revit model. The script we will develop will automatically calculate a point in the XYZ coordinate space, and it will use that point to place the chosen object. This is a fun chapter for all readers, from beginners to intermediate users. Other than working with lists and strings, you will see how to automatically place objects inside a Revit model, which is quite a good goal to reach.

Chapter 10, Gathering a Revit Models' Data from Multiple Models at Once, will show you how to extract data and automatically place objects inside a Revit model. You will be ready to learn how to collect data from all families from multiple Revit models. This chapter will be divided according to three main learning goals. The first one is the data gathering part, where you will learn how to manage lots of lists of data. The second part will focus on preparing that data for Microsoft Excel. The last part will be related to using data to visualize everything inside Power BI, including a map showing the location of each Revit model. Each piece will be written as a step-by-step guide, using screenshots when needed.

Chapter 11, Visualizing Data from Multiple Models in Power BI, continues from where the previous chapter left off. In Chapter 10, Gathering Revit Models' Data from Multiple Models at Once, you will have created a script to collect multiple models' data. Here, you will focus on Power BI, visualizing multiple models' data. You will learn how to manage an extensive dataset coming from multiple Revit projects. I will talk about organizing data, formatting data, and creating and filtering charts. Also, you will learn how to show the location of a model using a map inside Power BI. Toward the end of the chapter, you will also learn how to customize the map to show data in different colors and shapes.

Chapter 12, Having Fun with Power BI, is the last chapter of the book. The book's primary goal is to give you data visualization skills. The previous chapter was a deep dive into Power BI. Here, I will let you have fun connecting Power BI to a Google form that pushes data in real time. This chapter will guide you in creating a dashboard using live data generated by you. I will explain the workflow to connect Google services and Power BI to stream data and populate our charts continuously.

To get the most out of this book

This may be obvious, but I think you should underline, highlight, and annotate everything that helps you fix a concept in your head. Repeat until the idea is well impressed. Also, you could read the book with a colleague, so you can help each other with the exercises, or maybe you could compete and have a fun challenge from time to time.

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book's GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Managing-and-Visualizing-Your-BIM-Data. If there's an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://static.packt-cdn.com/downloads/9781801073981_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "Now, we should start with the table's name as a string without any symbols or special characters. So, please type table_levels."

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: "When you have everything selected, please click on Format as Table."

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Managing and Visualizig your BIM Data, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Section 1: Overview of Digitalization and BIM Data

In this part of the book, you will develop a good overview of what is happening in relation to digitization in general and why analyzing BIM data is so important. The first three chapters will explain background IT subjects that are fundamental in terms of reaching the book's goal: managing and visualizing BIM data. For example, we will learn how series of ones and zeros create digital content on the screens of our devices, along with how much digital data we produce every day, what types of digital data there are, and how our job falls inside the category of hybrid jobs. These are just some of the subjects we will cover during the first three chapters.

This section comprises the following chapters:

Chapter 1, Introducing Units of Digital InformationChapter 2, Understanding Data CentersChapter 3, Warming Up Your Data Visualization Engines

Chapter 1: Introducing Units of Digital Information

Welcome to the world of BIM data and digitalization!

Before we start digging into the core of digitalization in the Architecture, Engineering, Construction (AEC) industry, BIM data visualization, or management with Autodesk Dynamo and its business intelligence tools, I would like you to know why every business on earth is becoming a data-driven business. If you look around, every type of company, small or large, non-profit or governmental, is walking through a digital transformation, implementing and applying ones and zeros to every process they can. However, in the end, all that matters for company owners is that their company has to adapt to the new business model. It will take time for all of us to go fully digital, but we all will, whether it takes 10 months or 10 years; we are all already in that queue.

Even though this book's aim is to give you some knowledge on BIM data visualization and management, in the first chapter, we will start talking about data. Though we will set BIM aside for now, we will return to it later on, once we get more comfortable with digitalization and basic computer science concepts. We are going to learn about the events that started our digital era, and how modern devices use binary code to represent things such as pictures and videos. Then we will talk about types of digital data and about all of the data that surrounds us, all the time, everywhere (that's why they call it big data, right?!). Finally, we will explore so-called hybrid jobs.

But don't worry, I won't go too much into technical details, although we will cover some technical notions when it comes to Dynamo, data manipulation, and databases. So, get a cup of coffee and if you can, get an Italian one (even better if it's Neapolitan), and meet me in the first chapter!

Figure 1.1 – Say "big data" one more time meme!

In this chapter, we will cover the following topics:

Exploring the beginning of the digitization eraLearning how simple digital data isGetting to know types of digital dataUnderstanding how much data we produceLearning about hybrid jobs

Exploring the beginning of the digitization era

Lots of companies across the globe started the digital transformation process around the 1950s, when Dr Presper Eckert and Dr John Mauchly invented the UNIVAC, the Universal Automatic Computer, after receiving funds from the Census Bureau. This computer was the first commercial computer for business and government applications! The real leap, though, when things started to become super-duper serious, was in the 1990s, due to the diffusion of the world wide web. If you think about it, ever since then, shopping, banking, working, health, education, and so on, changed forever!

I remember when I was a kid at the end of 1990s; during weekends, my family and I used to go to the shops. There, like everyone else, me, my twin brother, my sister, and our parents bought new clothes, groceries, or new games for the PC (which was my and my dad's favorite!). At that time, no one thought that one day you would be able to buy groceries using your PC or your smartphone. For example, while I'm writing this book, my partner and I are also moving to a new house and have bought a new sofa and a few other pieces of furniture online.

In the following graph, you can have a look at the growth, from 1996, of two of the biggest e-commerce companies – Amazon and eBay:

Figure 1.2 – Growth of Amazon and eBay online shopping from 1996 to 2019

The important thing here that I would like you to understand is that since the 1990s (more or less), the impact of digitalization has been huge for every field, not only for e-commerce. Things such as paper and photographs have been transformed into binary code, the ones and the zeros of computer storage.

Let me ask you this question: Why is data so valuable?

The short answer is that, today, data is profitable. Google and Facebook, for example, built an empire by collecting and analyzing people's data, and today, advertisement companies can even predict what you'll buy tomorrow, literally. When I'm talking to friends and colleagues about this subject, I like to ask them a question, so I want to do the same with you: Do you remember what you were doing or searching on the internet in May 2009, for example? No? Well, Google does. The data we provide to those companies is so vast that they found a new name for it: big data!

But to better understand this sea of data, to use that information to upgrade a company process or find possible mistakes, or even predict something somehow, companies needed to turn it from raw data into well-organized information. And once they did that, they could give advertisers, for example, a set of tools to target their potential customers with astonishing precision. At this point, with all of the data turned into useful information, they needed to build something technologically advanced to analyze and categorize everything deeply, and use that information to make future predictions. That's where Artificial Intelligence (AI) comes to the stage. To give you an example, let's look at the social media platform Facebook. Instead of merely offering advertisers the ability to target their users using data such as demographics, gender, or consumer preferences, they instead provided the ability to target them based on what they will think, how they will behave, and what they will buy. Facebook, back in 2016, revealed an AI engine with self-improving capabilities that could predict all of those things!

And as insane and frightening and unique as it may sound, this is not the only prediction engine out there. In my opinion, I hope that governments will better regulate those systems because they raise more and more questions every day, especially ethically speaking. Think, for example, about Cambridge Analytica, the Facebook scandal of early 2018. Without going into too many details, Cambridge Analytica was implicated in a massive data breach. They used almost 90 million people's private data without their consent. That's massive. And the final goal was to create customized and targeted ads to address your vote for the upcoming political election. Again, like everything else, AI can help our society in many ways, but this is a clear example of how dangerous those systems may be in the wrong hands.

In this section, we just scratched the surface, introducing the events that started the digital era. Coming up, we will cover quite a curious subject: how digital content comes to life on the screens of our devices, starting from a series of ones and zeros.

Learning how simple digital data is

My life and your life are full of data. When you commute to work, for example, you could be scrolling through Twitter, looking at friend's pictures on their social networks, buying groceries from your favorite online store, or streaming music. Everything represented by computers is made of ones and zeros; it is that simple. But how is that possible? How did pictures, videos, and songs start from a simple series of ones and zeros?

Although the answer could be a lot more complex and technical, we will stick to the basics to understand the general concepts. Essentially, to do that, we need to learn things from the point of view of a computer, something that you use all of the time and probably take for granted. This concept is essential to understand when it comes to learning computer science theory. Each one or zero stated in a single switch is called a bit, the smallest piece of data a computer can store. Just one circuit board can handle billions of switches, and computers are now able to use billions of bits to represent more complex data, such as text, songs, and images.

So, speaking of switches, an electrical current flows through switches, and when it does or does not travel through a switch, the switch goes on or off. To give you more context, imagine only using words to describe every note of your favorite song, or every scene of your beloved TV show. That is exactly what a computer does! Our devices use binary code as a language to create digital content we all know.

Here is a simple example of the electrical current that flows through switches:

Figure 1.3 – Example of electrical flow that lets switches go on or off

Before going forward with the bits and the switches thing, you need to remember that in the past, we had analog electronics. All of those older electronics, including televisions, used analog signals with variable wave height to represent sounds and images. The problem was that those wave signals were really small and the waveforms could be interrupted by interference caused by any other signals (and nowadays we are surrounded by signals). This caused snow visuals and static sounds, for example. During the last 30 years, analog technologies have all been digitized. Using bits instead of waveforms reduces the interference dramatically.

Fortunately, neither you nor I have to learn binary code to use Word, Photoshop, or Revit! Let's take an image, for example. Every image on a computer screen is made up of dots that we call pixels, and I'm not talking about object-oriented graphics, of course (vector artworks); let's keep it simple. Anywhere between a few hundred to a few billion pixels can constitute an image, and each pixel is made up of one color, which is coded with decimal values or hexadecimal code. Those decimal values and code have been transformed by the computer, starting from a series of ones and zeros, which started from the flow of electrical current into billions of switches.

Now that we know how computers process binary code, we are ready to talk about digital data types, which is fundamental to understanding more complex subjects, such as data manipulation and databases.

Getting to know types of digital data

Let's go straight to the point. To group or classify the data out there, three major groups have been defined: unstructured, semi-structured, and structured.

To give you a simple example, it is possible to compare these types of data to vegetation. I've used three pictures, one for each data type. Take a look at them in the following paragraphs. Data can be represented as trees, leaves, and branches; unstructured data is like a wild and uncultured forest or jungle, with all the nature that creates beautiful chaos. We can imagine semi-structured data as forest paths, where the path is a bit difficult, but not as difficult as the wild and uncultivated forest. The last one is structured data, which is represented by a very well-organized field that it is possible to walk easily through.

So, let's take a look at each of these data types in more detail.

As the name suggests, the unstructured data type is data with an unknown form. It can be a combination of images, text, emails, and video files, and it can create value only when it is processed, analyzed, and well organized.

Figure 1.4 – Unstructured data is like an uncultured forest or jungle

Some of the main characteristics of the unstructured data type are as follows:

It does not have any particular format or sequence.It does not follow any rules or semantics.It does not have an easily identifiable structure.It cannot be stored in a spreadsheet-like form (that is, based on rows and columns).It isn't directly usable or understandable by a program.

So, basically speaking, anything that isn't in a database form belongs to the unstructured data type.

Important note

Gartner estimates that unstructured data constitutes 80% of the world's enterprise data.

Figure 1.5 – Semi-structured data is similar to a forest with paths

The semi-structured type, in contrast, is a type of data that can be processed using metadata tagging, which will help us to catch useful information. With this type of data, it is difficult to determine the meaning of the data, and it is even more challenging to store the data in rows and columns as in a standard database, so even with the availability of metadata, it is not always possible to automate data analysis. To give you an example, please take a look at the following email structure:

To: <Name>

From: <Name>

Subject: <Text>

CC: <Name><Name>

Body:<Graphics, Images, Links, Text, etc.>

Those email tags are considered a semi-structured data type, and similar entities in the data will be grouped in a hierarchy. For each group, there could be a few or a lot of properties, and those properties may or may not be the same. If you read the email structure again you can immediately see that tags give us some metadata. Still, it is almost impossible to organize the data of the body tag, for example, because it will almost certainly contain no format at all. So, let's take a look at the most common features of the semi-structured data type:

Attributes within the same group may not be the same.Similar entities will be grouped.It doesn't conform to a data model, but it contains tags and metadata.It cannot be stored in a spreadsheet-like form, that is, based on rows and columns.

There are lots of challenges here, too, to better manage, store, and analyze semi-structured data. The computer science community seems to be going toward a unique standard format for storing semi-structured data. All you need to know, for now, is that there is a format that is hardware and software independent, which is XML, an extensible markup language, which is also open source and written in plain text. This XML format is more or less the alter ego of the Industry Foundation Classes (IFC) for BIM models!

The third data type is the structured data type, which is a database of systematic data that can be used by companies for direct analysis and processing. It consists of information that has been both transformed and formatted into a well-defined data model. Without going too much into the technical details, remember that this type of data is mapped into predesigned fields that can later be read and extracted by a relational database. This way of storing information is the best one out of the three types, and even though the relational model minimizes data redundancy, you still need to be careful because structured data is more inter-dependent, and for this reason, it can sometimes be less flexible.

Figure 1.6 – Structured data looks like a well-organized field

So, some of the most important features of the structured data type are as follows:

It conforms to a data model.Similar entities are grouped.Attributes within the same group are the same.Data resides in fixed fields within a record.The definition and meaning of the data is explicitly known.

At this point, I would like you to understand that we will have to carry out different tasks to transform our raw data into structured information, whether we are dealing with unstructured, semi-structured, or structured data.

As you probably already understand, data is becoming a fundamental tool for knowing and understanding the world around us. Simply put, we can think of data as a way to "approach" problems and to "solve" them in the end. And at this point, I would like to introduce you to the Data, Information, Knowledge, Wisdom (DIKW) pyramid, also known as the DIKW model. The pyramid helps us by describing how raw data can be transformed into information, then knowledge, then wisdom. Take a look at the following image. As we move up to the top of the pyramid, we look for patterns, and by imposing structure, organization, classification, and categorization, we turn data without any particular meaning into knowledge, and finally wisdom.

Figure 1.7 – DIKW pyramid

To better fix the concept in your head, I would like to give you a simple yet practical example of the DIKW pyramid by talking about the weather! Imagine that it is raining; this is an objective thing and has no particular meaning, so we can associate raining with data. Then, if I tell you that it started to rain at 7p.m. and the temperature dropped by 2 degrees, that is information. Continuing on that path, if we can explain why this is happening, like saying that the temperature dropped because of low pressure in the atmosphere, you're talking about knowledge. And in the end, if we get a complete picture of things such as temperature gradients, air currents, evaporation, and so on, we can statistically make predictions about what will happen in the future – that's wisdom!

Although we didn't go into the technical details, I would like you to remember that businesses and organizations all around the world use processes such as the one described here, the DIKW pyramid, when it comes to organizing their data.

Here, we've learned some fundamental concepts, such as types of digital data and their differences. We've also learned about the DIKW pyramid. Next, we will talk about how much data we produce every 60 seconds!

Understanding how much data we produce

In this section, we will dive into a little bit of computer science, learning about digital data measuring units, but we will also talk about a curious subject. I'm talking about the data we produce every 60 seconds: it is unbelievable! So, first of all, let's talk about measuring units.

One byte consists of 8 bits, and since computers deal with ones and zeros, which means that they deal with math of base two instead of decimals (math of base ten), all increments in data storage units have to equate to the power of two, rather than the power of ten, like we all are used to. Consequently, one kilobyte (KB) consists of 1,024 bytes or 210, and not 103 as you probably expected.

Now, let's see some real-world examples:

1 hour of social network page scrolling will consume around 120 MB of data.1 hour of streaming music will consume about 150 MB of data.1 hour of watching full HD YouTube videos will consume approximately 1,500 MB (or 1.5 GB) of data.1 hour of high-definition streaming on Netflix will consume more or less 3,000 MB (or 3 GB) of data.1 hour of Skype calls with five or six colleagues will consume up to 4,000 MB (or 4 GB) of data.

Another example is that the entire original Super Mario Bros game consists of 32 KB, which is roughly 0.03 MB! There is an unbalanced proportion between the game's size, which is incredibly small, and the amount of happiness it brought people of any age from all over the world!

Now that we've got some understanding of the measuring units, let's talk about something more fun: data exchanges of the online services we all use, every day. We will also build, later on, a few Power BI charts using the data I am going to show you right now. We will create those charts to get some familiarity with this powerful business intelligence tool and discover some of its basic commands and features.

Have you ever wondered how much data we generate every 60 seconds?

The amount of data we generate is insane. Think, for example, of a quite simple thing that we do over and over every day: Google searches. We even invented a new verb for it, "googling"! It has also become synonymous with the internet. Anyway, we all are curious about everything all the time. People like me and you are always thirsty for information, and that's why we use Google all day long. The famous search engine is so dominant in the search engine market because it provides free access to information in a blink of an eye. But, of course, Google didn't become a giant overnight. Back in the days before Google or Facebook, the absolute king of the internet was named Yahoo, a tech company founded in 1994 that, today, still, handles 5% of web searches. A few years later, in 1999, the company was worth $125 billion, but unfortunately, the giant made some wrong moves and started to lose users. Probably the most significant wrong move was to refuse to acquire Google. Yes, in case you didn't know, Google's Larry Page and Sergey Brin approached the Yahoo CEO in 1998 and asked for the small amount of $1 million. You can deduce the end of the story; **Yahoo refused. But this wasn't the only mistake made by Yahoo. Yahoo declined to buy Google again in 2002 for an amount of $5 billion, and in 2008 refused to be acquired by Microsoft for $44.6 billion, but that is another story. Anyhow, Google today has passed $1 trillion in market value, which is astonishing.

Back to the data! At Google, nowadays, they process something like 5.4 billion searches per day! Breaking it down, you get 3.8 million searches per minute. Insane. The giant search engine now dominates the market with almost 93% of total internet searches.

But this is not the only crazy number of online services we are going to talk about. You might be wondering about LinkedIn, Amazon, Netflix, YouTube, Instagram, and so on. Earlier, I told you that we would be talking about what happens every 60 seconds. Well, if you go back a few lines, you'll see a sign with two asterisks right next to the phrase **Yahoo refused. I counted how long it takes to read the text from the "**Yahoo refused." to now, and guess what, it takes more or less 60 seconds to read those lines!

Here, I have listed a few things that happened in that timeframe:

Amazon shipped between 6,000 and 7,000 packages.Users scrolled more or less 350,000 times on Instagram.55,000 users, approximately, logged on to Microsoft Teams.Netflix users streamed about 100,000 hours of video.2,000 new Reddit comments were created.LinkedIn gained 120 new professionals.4,500,000 videos were watched on YouTube.

Figure 1.8 – Data everywhere!

These are just a few of the things that happen every 60 seconds on the internet, and even if it seems overwhelming, we, as humankind, just started the ladder of digitization. During the last 10 to 20 years, thanks to the spread of the internet, we saw a rapid evolution of the business landscape. In this short period, the digitization era has left us with a very eventful time.

In the next section, we will talk about a less technical yet quite interesting subject: hybrid jobs!

Learning about hybrid jobs

Today, because of the large amounts of data we produce and collect, companies from every industry are looking for new types of employees. Nowadays, an increasing number of professionals are developing IT skills that make them the perfect candidates for many companies that have invested in digital transformation. This interesting factor occurs in any field: financial, medical, engineering, hospitality, and so on.

I would like you to understand this concept because if you would like to apply for a new job, you probably would have a better chance if you have learned some coding or data science skills! This is why. People working in HR call them hybrid jobs. According to Burning Glass Technologies, a company that specializes in analyzing trends, skills demand, and job growth, one of every eight job postings asks for candidates with hybrid skills, no matter the field.

Employers indeed are requesting coding and data analysis skills, other than a degree in their business area. Here, we are talking about a mix of skills from different domains. For example, marketers rely on big data or data science abilities to coordinate their advertisement campaigns. At the same time, increasing numbers of architects and engineers need to work with data systems and robotics. Take the engineering field as another example; it is a common practice that students take computer science classes. So, we see a significant increase in the computer science skills demand from employers in various industries and sectors.

Figure 1.9 – A woman with hybrid skills!

Some of those hybrid jobs are as follows:

Data scientistsBusiness intelligence managersVirtual reality architectsVisual data scientistsWeb analytics managers

These roles are just a few among all of the new jobs that our digital economy is demanding. We also, as professionals in the AEC industry, have seen the spread of new roles in the last few years, such as design technology professionals, computational design specialists, developers with an architectural or engineering background, and so on.

Anyhow, we can say without any doubt that every company, no matter what the field is, is becoming little by little closer to a tech company. That's the reason why we will have to adapt to this change, and we all have to learn computer science by learning the logic of how software is created. You can't merely learn how to use a few pieces of software and use them your whole life! In our ever-changing world, we should learn the logic underneath the tools, and not the tools themselves.

Machines and algorithms will be our next best friends/colleagues! Nevertheless, we, as professionals in the AEC industry, have to do the same as other colleagues in other sectors are already doing. We're doing nothing special here. We need to increase our knowledge and work hard to overcome our next challenges, which will let us better understand, manage, and analyze our processes and model's data!

We just learned what hybrid jobs are and why IT and computer science skills are becoming more and more critical for all types of businesses. If you aren't yet, get ready to acquire new hybrid skills in the upcoming months or years. And, by the way, by the end of the book, you'll already have started that path. So, stick with me and follow the book without skipping any parts! You know the saying, "No pain, no gain!"

Summary

In this chapter, we have learned why every company is moving toward a data-driven business model. Then, we discussed how series of ones and zeros create digital content. Also, we started to explore the three types of digital data, unstructured, semi-structured, and structured, and how the DIKW pyramid helps us to organize our data into useful information. Next, we discovered how much data we produce every 60 seconds, and it is unbelievable that those numbers keep growing every month! Finally, we discussed how data science skills have become vital for so-called hybrid jobs.

In the next chapter, we are going to deep dive into the world of data centers: when the first data center came to be, what they are now, and why are they so important for businesses and organizations worldwide. Then we will talk about two of the most significant data center projects, and finally, we will take a first look at Power BI's core functionalities.

Chapter 2: Understanding Data Centers

Welcome to the second chapter! I want to introduce you to the world of data centers. This is an essential subject because before learning data science or programming skills, you should know everything about how data flows. You need to precisely understand where and how files and information are both stored and retrieved, and you need to understand the types of environments in which data resides. Here, we will learn the importance of data centers and the most common data center types that companies use to support their IT infrastructures. We will also talk about The Citadel and Project Natick, two of the most promising and advanced data centers worldwide. At the end of this chapter, we will be introducing Power BI by analyzing an exciting yet straightforward dataset, the World Happiness Report!

Today, we know that companies and organizations worldwide have IT systems that need to be managed, updated, and monitored. And that is how data centers come into play. They are used to guarantee the continuity of businesses' IT infrastructures. Their primary tasks are recovering and backing up data, as well as networking. Everything we do on our devices goes to or comes from a data center somehow. We can say without any doubt that times have changed, and the information demand is increasing month by month. Probably 90% of what we do every day travels from one server to another in the blink of an eye. And all of those networking activities are managed and hosted by data centers:

Figure 2.1 – "The cloud" is just someone else's computer!

In this chapter, we will cover the following topics:

Understanding data centersGetting started with Microsoft Power BI

Understanding data centers

A data center is a facility that contains networking and computing equipment. Its purpose is to collect, distribute, process, and store large amounts of data. It can consist of just one server or more complex systems with hundreds or thousands of servers on racks (frameworks for holding or storing hardware). So, every organization, no matter what its size is, needs one.

Data centers are mainly divided into two prominent families:

Traditional data centers: Always physically accessibleCloud data centers: Always accessible through the internet

Although those two types are so different from each other, it is difficult to find a facility that is only physically accessible or only accessible through the internet. Usually, traditional data centers are rooms or buildings directly owned by a private organization for internal use only. The majority of data centers are a combination of both worlds. Companies such as Amazon and Microsoft provide cloud data center services. I want you to keep in mind that cloud computing and cloud storage services are booming these days.

With traditional data centers, you need to purchase several things, from the server hardware to the networking hardware. You also need to hire a technical team dedicated to the maintenance of the system. Other than that, you have to design and buy the infrastructure to provide an uninterruptible power supply, a cooling system, backup generators, and so on. You would probably only choose to have a traditional data center if you have lots of money to invest, or if your company is named Amazon or Google!

However, if you are a company owner and are about to buy or switch to a cloud solution, please keep in mind that the human factor is as important as the others, or maybe more. People with IT-related skills are essential here. For example, if you've got the budget to sustain the initial investment to buy the hardware and the software needed, but you have no clue who to hire and what their skills must be, take a step back! Talk to IT managers when you get the chance, or at least some HR professionals who work with IT professionals. Even better, hire someone like a design technology manager or an IT manager who will not only help you out during the initial phases of planning, but will be the pillar of all your future IT decisions.

Software/hardware are just tools. People determine the success (or not) of the process.

So, let's get started and look at the different types of data centers.

Learning the different types of cloud solutions

I want to start by sharing my personal experience about the first time I came into contact with the cloud.