Learn Azure Sentinel E-Book

Richard Diver

• Herausgeber: Packt Publishing
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 396

Veröffentlichungsjahr: 2020

Beschreibung

Understand how to set up, configure, and use Azure Sentinel to provide security incident and event management services for your environment

Key Features

• Secure your network, infrastructure, data, and applications on Microsoft Azure effectively
  
  


• Integrate artificial intelligence, threat analysis, and automation for optimal security solutions
  
  


• Investigate possible security breaches and gather forensic evidence to prevent modern cyber threats

Book Description

Azure Sentinel is a Security Information and Event Management (SIEM) tool developed by Microsoft to integrate cloud security and artificial intelligence (AI). Azure Sentinel not only helps clients identify security issues in their environment, but also uses automation to help resolve these issues. With this book, you'll implement Azure Sentinel and understand how it can help find security incidents in your environment with integrated artificial intelligence, threat analysis, and built-in and community-driven logic.

This book starts with an introduction to Azure Sentinel and Log Analytics. You'll get to grips with data collection and management, before learning how to create effective Azure Sentinel queries to detect anomalous behaviors and patterns of activity. As you make progress, you'll understand how to develop solutions that automate the responses required to handle security incidents. Finally, you'll grasp the latest developments in security, discover techniques to enhance your cloud security architecture, and explore how you can contribute to the security community.

By the end of this book, you'll have learned how to implement Azure Sentinel to fit your needs and be able to protect your environment from cyber threats and other security issues.

What you will learn

• Understand how to design and build a security operations center
  
  


• Discover the key components of a cloud security architecture
  
  


• Manage and investigate Azure Sentinel incidents
  
  


• Use playbooks to automate incident responses
  
  


• Understand how to set up Azure Monitor Log Analytics and Azure Sentinel
  
  


• Ingest data into Azure Sentinel from the cloud and on-premises devices
  
  


• Perform threat hunting in Azure Sentinel

Who this book is for

This book is for solution architects and system administrators who are responsible for implementing new solutions in their infrastructure. Security analysts who need to monitor and provide immediate security solutions or threat hunters looking to learn how to use Azure Sentinel to investigate possible security breaches and gather forensic evidence will also benefit from this book. Prior experience with cloud security, particularly Azure, is necessary.

Leseprobe

Learn Azure Sentinel

Integrate Azure security with artificial intelligence to build secure cloud systems

Richard Diver

Gary Bushey

BIRMINGHAM—MUMBAI

Learn Azure Sentinel

Copyright © 2020 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Commissioning Editor: Vijin Boricha

Acquisition Editor: Meeta Rajani

Senior Editor: Arun Nadar

Content Development Editor: Romy Dias

Technical Editor: Mohd Riyan Khan

Copy Editor: Safis Editing

Project Coordinator: Neil D’mello

Proofreader: Safis Editing

Indexer: Rekha Nair

Production Designer: Alishon Mendonca

First published: March 2020

Production reference: 1030420

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-83898-092-4

www.packt.com

Packt.com

Subscribe to our online digital library for full access to over 7,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe?

Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionalsImprove your learning with Skill Plans built especially for youGet a free eBook or video every monthFully searchable for easy access to vital informationCopy and paste, print, and bookmark content

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.

At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

Foreword

It is my great pleasure to contribute the foreword to this piece of work by Gary and Richard. We are in exciting times! Not only is the technology of Azure Sentinel exciting, the opportunity that it presents is exciting as well.

Having been in the technology and security consulting business for around 25 years, I’ve seen many things that have been called “Next Generation” and “Game Changing” before. But I will say that what is happening right now only happens once in a career. Some would say a statement like this is hyperbole but hear me out. I doubt that we’ll have another opportunity in our careers to witness the coming of age of the public cloud, the coming of age of Microsoft’s security reference architecture, and the coming of age of cyber security in general...all converging at the same time. What I mean by this convergence is that these things have all hit critical mass in a way that each enables the other, so much so that it will be difficult to tell them apart in a few years.

With this convergence will come change, and disruption as well, which can create a certain amount of chaos and uncertainty. Should we be doing so many things so differently than we have been? Can this newly created technology really be as stable and capable as where we came from? Will we even be able to do things in the same way, and if we can’t, who will lead us out of the darkness? To be plain, Microsoft has made the right investments in security. They eat their own dog food in that everything they release is vetted on their own global network. They’ve quit developing security products as separate components and now focus on the full platform. They recognize that a multi-platform, hybrid infrastructure exists in most environments and they’ve attacked those problems head on.

Azure Sentinel is capable of bringing Microsoft’s own products together, but it additionally brings the capability of being a central component of an organization’s security operations center and that is a game changer.

Gary and Richard have embraced the latest tech from Microsoft’s security platform and worked with forward-looking clients that have the same vision to assess, architect, and implement this tech even with the (almost weekly) changing capabilities and consoles as Microsoft aggressively integrates and enhances their platform. Whenever there is something new, it takes some brave hearts to invest the time and effort to explore the landscape, make some assumptions, and make it work...and I’ve watched these guys do just that.

There is a reward for them and for the consumers of this material. For them, they can plant the flag on this hill, congratulate themselves for the discovery thus far, and make preparations for the next leg of the journey. For you, there is a wealth of knowledge compiled here by folks that earned it the old-fashioned way. And knowing what I do about these guys, they are happy to be the Sherpas for you on your Sentinel journey. Enjoy!

Jason S. Rader,

Director of Network and Cloud Security at Insight

Contributors

About the authors

Richard Diver has over 25 years’ international experience in technology with a deep technical background in cloud security, identity management, and information security. He works at Insight as the lead for Cloud Security Architecture, working with top partners across the industry to deliver comprehensive cloud security solutions. Any spare time he gets is usually spent with his family.

I would like to thank my loving family for allowing me to take time out to write this book, especially encouraging me during those times of procrastination!

Thank you to Gary for taking on this challenge with enthusiasm and passion, I could not have done this without your expertise. Also, to the experts that have helped to improve this book, including Ashwin Patil, Rod Trent, Dean Gross, Casey Tuohey, and Brandon Huckeba.

Gary Bushey is an Azure security expert with over 25 years of IT experience. He got his start early on when he helped his fifth-grade math teacher with their programming homework and worked all one summer to be able to afford his first computer, a Commodore 64. When he sold his first program, an apartment management system, at 14 he was hooked. During his career, he has worked as a developer, consultant, trainer, and architect. When not spending time in front of a computer, you can find him hiking in the woods, taking pictures, or just picking a direction and finding out what is around the next corner.

First and foremost, I need to thank my parents. They may not understand exactly what I do but they have supported me from allowing me to hook up my Commodore 64 to the color TV to faking interest when I talk about what I do. I’d also like to thank all the people at Microsoft that have helped me. Finally, I need to thank Lora Lake for supporting me through all my crazy decisions that have led to this point and for adopting me as her brother.

About the reviewers

Rod Trent, a community professional, keynoter, and evangelist, is a Cyber PFE for Microsoft and Azure Sentinel SME who spends his entire work life educating customers on how to implement, use, and maintain Azure Sentinel. Rod works with the largest Azure Sentinel implementations in the world. Some may remember Rod from his pre-Microsoft life, where he owned and operated some very significant communities dedicated to IT management and security, ran technology-focused editorial teams, and managed some large and popular technology conferences. When he’s not evangelizing Azure Sentinel and digging into KQL queries, he spends time with his wife of 30 years, Megan, and his four wonderful kids, Alex, Rachel, Eric, and Elly.

Ashwin Patil currently works as Senior Program Manager for Microsoft Threat Intelligence Center (MSTIC) and has over 10 years of experience entirely focused on security monitoring and incident response, defending enterprise networks. In his current role, he primarily works on threat hunting, detection research in KQL (Kusto query language) for Azure Sentinel, and developing Jupyter notebooks written in Python/R to do threat hunting and investigation across a variety of cloud and on-premise security event log data sources. He has a bachelor’s degree in computer engineering and is also certified with various SANS certifications, such as GCIA, GCFE, and GCIH, in the field of digital forensics and incident response (DFIR).

Packt is searching for authors like you

If you’re interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Table of Contents

Learn Azure Sentinel2

Why subscribe?3

Foreword

Contributors

About the authors6

About the reviewers7

Packt is searching for authors like you7

Preface

Who this book is forix

What this book coversix

To get the most out of this bookxi

Download the color imagesxi

Conventions usedxi

Get in touchxii

Reviewsxii

Section 1: Design and Implementation

Chapter 1: Getting Started with Azure Sentinel

The current cloud security landscape16

Cloud security reference framework17

SOC platform components20

Mapping the SOC architecture22

Log management and data sources22

Operations platforms23

Threat intelligence and threat hunting25

SOC mapping summary26

Security solution integrations26

Cloud platform integrations28

Integrating with AWS28

Integrating with Google Cloud Platform (GCP)28

Integrating with Microsoft Azure29

Private infrastructure integrations30

Service pricing for Azure Sentinel31

Scenario mapping34

Step 1 – Define the new scenarios34

Step 2 – Explain the purpose35

Step 3 – The kill-chain stage35

Step 4 – Which solution will do detection?36

Step 5 – What actions will occur instantly?36

Step 6 – Severity and output37

Step 7 – What action should the analyst take?37

Summary38

Questions38

Further reading39

Chapter 2: Azure Monitor – Log Analytics

Technical requirements42

Introduction to Azure Monitor Log Analytics43

Planning a workspace46

Creating a workspace using the portal47

Creating a workspace using PowerShell or the CLI49

Exploring the Overview page56

Managing the permissions of the workspace57

Enabling Azure Sentinel58

Exploring the Azure Sentinel Overview page61

The header bar62

The summary bar62

The Events and alerts over time section62

The Recent incidents section62

The Data source anomalies section62

The Potential malicious events section62

The Democratize ML for your SecOps section63

Connecting your first data source63

Obtaining information from Azure virtual machines63

Advanced settings for Log Analytics66

Connected Sources67

The Data option68

Computer Groups69

Summary73

Questions73

Further reading73

Section 2: Data Connectors, Management, and Queries

Chapter 3: Managing and Collecting Data

Choosing data that matters78

Understanding connectors80

Native connections – service to service81

Direct connections – service to service82

API connections82

Agent-based83

Configuring Azure Sentinel connectors85

Configuring Log Analytics storage options92

Calculating the cost of data ingestion and retention94

Reviewing alternative storage options96

Questions97

Further reading98

Chapter 4: Integrating Threat Intelligence

Introduction to TI100

Understanding STIX and TAXII102

Choosing the right intel feeds for your needs103

Implementing TI connectors104

Enabling the data connector104

Registering an app in Azure AD106

Configuring the MineMeld threat intelligence feed110

Confirming the data is being ingestedforuse by Azure Sentinel116

Summary118

Questions118

Further reading119

Chapter 5: Using the Kusto Query Language (KQL)

Running KQL queries122

Introduction to KQL commands124

Tabular operators125

Query statement140

Scalar functions140

String operators142

Summary143

Questions144

Further reading144

Chapter 6: Azure Sentinel Logs and Writing Queries

An introduction to the Azure Sentinel Logs page146

Navigating through the Logs page146

The page header148

The Tables pane156

The Filter pane159

The KQL code window161

The results window166

Learn more173

Writing a query173

The billable data ingested174

Map view of logins175

Other useful logs176

Summary177

Questions178

Further reading178

Section 3: Security Threat Hunting

Chapter 7: Creating Analytic Rules

An introduction to Azure Sentinel Analytics182

Types of analytic rules182

Navigating through the Analytics home page183

Creating a rule from a rule template191

Creating a new rule using the wizard192

Managing analytic rules205

Summary206

Questions207

Chapter 8: Introducing Workbooks

An overview of the Workbooks page210

The workbook header211

The Templates view212

Workbook detail view212

Missing required data types213

Workbook detail view (continued)213

Saved template buttons214

Walking through an existing workbook216

Creating workbooks218

Creating a workbook using a template218

Creating a new workbook from scratch219

Editing a workbook221

Advanced editing224

Managing workbooks225

Workbook step types227

Text229

Query229

Metric234

Parameters234

Links/tabs240

Advanced settings244

Summary249

Questions250

Further reading250

Chapter 9: Incident Management

Using the Azure Sentinel Incidents page252

The header bar252

The summary bar253

The search and filtering section253

Incident listing255

Incident details pane256

Using the Actions button261

Exploring the full details page262

The Alerts tab263

The Bookmarks tab265

The Entities tab266

The Comments tab266

Investigating an incident267

Showing related alerts268

The Timeline button270

The Info button271

The Entities button272

The Help button272

Questions273

Further reading274

Chapter 10: Threat Hunting in Azure Sentinel

Introducing the Azure Sentinel Hunting page276

The header bar276

The summary bar277

The hunting queries list277

Hunting query details pane279

Working with Azure Sentinel Hunting queries281

Adding a new query281

Editing a query282

Cloning a query282

Deleting a query283

Working with Livestream283

Working with bookmarks285

Creating a bookmark286

Viewing bookmarks288

Associating a bookmark with an incident290

Using Azure Sentinel Notebooks293

The header bar294

The summary bar294

The notebook details pane295

Performing a hunt297

Develop premise298

Determine data299

Plan hunt300

Execute investigation 300

Respond300

Monitor301

Improve301

Summary302

Questions302

Further reading303

Section 4: Integration and Automation

Chapter 11: Creating Playbooks and Logic Apps

Introduction to Azure Sentinel playbooks308

Playbook pricing309

Overview of the Azure Sentinel connector309

Exploring the Playbooks page311

The header bar312

The summary bar312

Logic app listing313

Logic app settings page313

The menu bar314

The header bar315

The essentials section316

The summary section316

The Runs history section317

Creating a new playbook318

Using the Logic Apps Designer page319

The Logic Apps Designer header bar321

The Logic App Designer workflow editor section322

Creating a simple Azure Sentinel playbook323

Summary330

Questions330

Further reading331

Chapter 12: ServiceNow Integration

Overview of Azure Sentinel alerts 334

Overview of IT Service Management (ITSM)335

Logging in to ServiceNow336

Cloning an existing logic app337

Modifying the playbook340

Additional incident information343

Adding dynamic content346

Adding an expression349

Summary352

Questions352

Further reading352

Section 5: Operational Guidance

Chapter 13:Operational Tasks for Azure Sentinel

Dividing SOC duties356

SOC engineers356

SOC analysts357

Operational tasks for SOC engineers357

Daily tasks357

Weekly tasks358

Ad hoc tasks358

Operational tasks for SOC analysts359

Daily tasks359

Weekly tasks359

Monthly tasks360

Ad hoc tasks360

Summary361

Questions361

Chapter 14: Constant Learning and Community Contribution

Official resources from Microsoft364

Official documentation364

Tech community – blogs364

Tech community – forum365

Feature requests366

LinkedIn groups367

Other resources367

Resources for SOC operations368

MITRE ATT&CK® framework368

National Institute of Standards for Technology (NIST)368

GitHub for Azure Sentinel369

GitHub for community contribution370

Kusto Query Language (KQL)370

Jupyter Notebook371

Azure Logic Apps372

Summary373

Assessments

Chapter 1375

Chapter 2375

Chapter 3376

Chapter 4376

Chapter 5377

Chapter 6378

Chapter 7378

Chapter 8378

Chapter 9379

Chapter 10379

Chapter 11380

Chapter 12380

Chapter 13381

Other Books You May Enjoy

Leave a review - let other readers know what you think385

Preface

Microsoft’s launch of Azure Sentinel is a major step forward for Security Information and Event Management (SIEM) solutions. As the first completely cloud-first SIEM in the marketplace, Azure Sentinel allows you to collect and query data from Azure, on-premises systems, and other cloud systems.

This book provides you with the guidance you need in order to create, configure, and use Azure Sentinel in your environment.

Who this book is for

This book is for anyone who wants to learn about Azure Sentinel. If you need to install, configure, or use Azure Sentinel, this book is for you.

What this book covers

Chapter 1, Getting Started with Azure Sentinel, will give an overview of Azure Sentinel, including coverage of the current cloud landscape, the cloud security reference framework, Security Operations Center (SOC) platform components, and how to map the architecture. You will also learn about integrating on-premises infrastructure into Azure Sentinel as well as how Azure Sentinel is priced.

Chapter 2, Azure Monitor – Log Analytics, will cover Azure Monitor Log Analytics, including planning your Log Analytics instance, how to create a new instance, and how to attach an instance to Azure Sentinel. You will also learn about the advanced settings for Log Analytics and about the Azure Sentinel overview page.

Chapter 3, Data Collection and Management, will explain how to determine what data you need to ingest into Azure Sentinel and how to connect to various data sources to get that information. You will also learn how to adjust data retention plans and how data retention is priced.

Chapter 4, Integrating Threat Intelligence, will introduce you to threat intelligence and how to ingest different threat intelligence feeds into Azure Sentinel.

Chapter 5, Using Kusto Query Language (KQL), will discuss Kusto Query Language (KQL) and will explain out how to write your own queries.

Chapter 6, Azure Sentinel Logs and Writing Queries, will introduce you to Azure Sentinel’s Logs page and will teach you how to use it to start writing your KQL queries against the data you have ingested.

Chapter 7, Creating Analytic Rules, will teach you how to create analytic rules that will search for anomalies in your environment. It will discuss analytic rule templates and how you can use them to create your own rules as well as how to create them from scratch.

Chapter 8, Introducing Workbooks, will cover Azure Sentinel’s workbook page, workbook templates, and how you can create a workbook from a template or from scratch.

Chapter 9, Incident Management, will explain how to manage incidents that your analytic rules create. You will learn about the incident page, how to view an incident’s full details, and how to start investigating an incident using Azure Sentinel’s Investigate GUI interface.

Chapter 10, Threat Hunting in Azure Sentinel, will introduce you to Azure Sentinel’s Hunting page, which will allow you to start your threat hunting activities. It will also briefly discuss Azure Notebook, which is Azure’s hosted Jupyter resource. There will also be a discussion of the steps needed to perform your investigation.

Chapter 11, Creating Playbooks and Logic Apps, will introduce you to Azure Sentinel’s playbooks and explain how they relate to Logic Apps. You will learn about the logic app Azure Sentinel connector and go through a walk-through about creating your own playbook.

Chapter 12, ServiceNow Integration, will provide an introduction to Information Technology Service Management (ITSM), the ServiceNow application, and how to create a simple Azure Sentinel playbook to create a new ticket in ServiceNow using information from your Azure Sentinel incident.

Chapter 13, Operational Tasks for Azure Sentinel, will cover the steps needed to keep your Azure Sentinel instance running smoothly. The steps will be broken up between your SOC analytics and your SOC engineers, as each have different aspects of Azure Sentinel that they will be responsible for.

Chapter 14, Constant Learning and Community Contributions, contains a list of various places you can go to continuing learning about Azure Sentinel and its supporting resources, including Logic Apps, Jupyter Notebook, KQL, and Fusion.

To get the most out of this book

We recommend that you have access to an Azure environment where you have the rights to create your own Azure Sentinel environment. Prior experience of using the Azure portal would also be beneficial.

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://static.packt-cdn.com/downloads/9781838980924_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “Mount the downloaded WebStorm-10*.dmg disk image file as another disk in your system.”

A block of code is set as follows:

html, body, #map {

height: 100%;

margin: 0;

padding: 0

}

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

[default]

exten => s,1,Dial(Zap/1|30)

exten => s,2,Voicemail(u100)

exten => s,102,Voicemail(b100)

exten => i,1,Voicemail(s0)

Any command-line input or output is written as follows:

$ mkdir css

$ cd css

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: “Select System info from the Administration panel.”

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packt.com.

Section 1: Design and Implementation

In this section, you will gain an overview of Azure Sentinel, including the current cloud landscape, the cloud security reference framework, Security Operations Center (SOC) platform components, and how to map the architecture. You will also learn about Azure Monitor Log Analytics, including how to plan your Log Analytics instance, how to create a new instance, and how to attach an instance to Azure Sentinel.

The following chapters are included in this section:

Chapter 1, Getting Started with Azure SentinelChapter 2, Azure Monitor – Log Analytics

Chapter 1: Getting Started with Azure Sentinel

Welcome to the first chapter in this book about Azure Sentinel. To understand why this solution was developed, and how best to use it in your organization, we need to explore the cloud security landscape and understand each of the components that may feed data into or extract insights out of this system. We also need to gain a baseline understanding of what a strong Security Operations Center (SOC) architecture looks like, and how Azure Sentinel is going to help to build the foundations for a cost-effective and highly automated cloud security platform.

In this chapter, we will cover the following topics:

The current cloud security landscapeCloud security reference frameworkSOC platform componentsMapping the SOC architectureSecurity solution integrationsCloud platform integrationsPrivate infrastructure integrationsService pricing for Azure SentinelScenario mapping

The current cloud security landscape

To understand your security architecture requirements, you must first ensure you have a solid understanding of the IT environment that you are trying to protect. Before deploying any new security solutions, there is a need to map out the solutions that are currently deployed and how they protect each area of the IT environment. The following list provides the major components of any modern IT environment:

Identity for authentication and authorization of access to systems.Networks to gain access to internal resources and the internet.Storage and compute in the data center for internal applications and sensitive information.End user devices and the applications they use to interact with the data.And in some environments, you can include Industrial Control Systems (ICS) and the (IoT).

When we start to look at the threats and vulnerabilities for these components, we quickly find ourselves deep in the alphabet soup of problems and solutions:

Figure 1.1 – The alphabet soup of cyber security

This is by no means an exhaustive list of the potential acronyms available. Understanding these acronyms is the first hurdle; matching them to the appropriate solutions and ensuring they are well deployed is another challenge all together (a table of these acronyms can be found in the appendix of this book).

Cloud security reference framework

To assist with the discovery and mapping of current security solutions, we developed the cloud security reference framework. The following diagram is a section of this framework that provides the technical mapping components, and you can use this to carry out a mapping of your own environment:

Figure 1.2 – Technical mapping components – cloud security reference framework

Each of these the 12 components is described in the following with some examples of the type of solutions to consider for integration with Azure Sentinel and the rest of your security architecture:

Security Operations Center: At a high level, it includes the following technologies and procedures: log management and Security Incident and Event Monitoring (SIEM), Security Orchestration and Automated Response (SOAR), vulnerability management, threat intelligence, incident response, and intrusion prevention/detection. This component is further explored in the, Mapping the SOC Architecture section later in this chapter.Productivity Services: This component covers any solution currently in use to protect the business productivity services that your end users rely on for their day to day work. This may include email protections, SharePoint Online, OneDrive for Business, Box, Dropbox, Google Apps, and Salesforce. Many more will come in future, and most of these should be managed through the Cloud Access Security Broker (CASB) solutionIdentity and Access Management: Identities are one of the most important entities to track. Once an attacker gains access to your environment, their main priority is to find the most sensitive accounts and use them to exploit the systems further. In fact, identity is usually one of the first footholds into your IT environment, usually through a successful phishing attack.Client Endpoint Management: This component covers a wide range of endpoints, from desktops and laptops to mobile devices and kiosk systems, all of which should be protected by specialized solutions such as End Point Detection and Response (EDR), Mobile Device Management (MDM), and Mobile Application Management (MAM) solutions to ensure protection from advanced and persistent threats against the operating systems and applications. This component also includes secure printing, managing peripherals, and any other device that an end user may interact with, such as the future of virtual reality/augmentation devices.Cloud Access Security Broker: This component has been around for several years and is finally becoming a mainstay of the modern cloud security infrastructure due to the increase adoption of cloud services. The CASB is run as a cloud solution that can ingest log data from SaaS applications and firewalls and will apply its own threat detection and prevention solutions. Information coming from the CASB will be consumed by the SIEM solution to add to the overall picture of what is happening across your diverse IT environment.Perimeter Network: One of the most advanced components, when it comes to cyber security, must be the perimeter network. This used to be the first line of defense, and for some companies it still is the only line of defense. That is changing now, and we need to be aware of the multitude of options available, from external facing advanced firewalls, web proxy servers, and application gateways to virtual private networking solutions and secure DNS. This component will also include protection services such as DDoS, Web Application Firewall, and Intrusion Protection/Detection services. IoT and Industrial Control Systems: Industrial Control Systems (ICS) are usually operated and maintained in isolation from the corporate environment, known as the Information Technology/Operational Technology divide (IT/OT divide). These are highly bespoke and runs systems that may have existed for decades and are not easily updated or replaced.

The IoT is different yet similar; in these systems, there are lots of small headless devices that collect data and control critical business functions without working on the same network. Some of these devices can be smart to enable automation; others are single use (vibration and temperature sensors). The volume and velocity of data that can be collected from these systems can be very high. If useful information can be gained from the data, then consider filtering the information before ingesting into Azure Sentinel for analysis and short- or long-term retention.

Private Cloud Infrastructure: This may be hosted in local server rooms, a specially designed data center, or hosted with a third-party provider. The technologies involved in this component will include storage, networks, internal firewalls, and physical and virtual servers. The data center has been the mainstay of many companies for the last 2-3 decades, but most are now transforming into a hybrid solution, combining the best of cloud (public) and on-premises (private) solutions. The key consideration here is how much of the log data can you collect and transfer to the cloud for Azure Monitor ingestion. We will cover the data connectors more in Chapter 3, Data Collection and Management.

Active Directory is a key solution that should also be included in this component. It will be extended to public cloud infrastructure (component 09) and addressed in the Privileged Access Management section (component 10). The best defense for Active Directory is to deploy the Azure Advanced Threat Protection (Azure ATP) solution, which Microsoft has developed to specifically protect Active Directory domain controllers.

Public Cloud Infrastructure: These solutions are now a mainstay of most modern IT environments, beginning either as an expansion of existing on-premises virtualized server workloads, a disaster recovery solution, or an isolated environment created and maintained by the developers. A mature public cloud deployment will have many layers of governance and security embedded into the full life cycle of creation and operations. This component may include Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS) services; each public cloud service provider offers their own security protections that can be integrated with Azure Sentinel. Privileged Access Management: This is a critical component, not to be overlooked, especially gaining access to the SOC platform and associated tools. The Privileged Access Management (PAM) capability ensures all system-level access is highly governed, removing permissioned when not required, and making a record for every request for elevated access. Advanced solutions will ensure password rotation for service accounts, management of shared system accounts (including SaaS services such as Twitter and Facebook), and the rotation of passwords for the local administrator accounts on all computers and servers. For the SOC platform, consider implementing password vaults and session recording for evidence gathering.Cloud Workload Protection Platform: This component may also be known as a Cloud Security Posture Management (CSPM), depending on the view of the solution developed. This is a relatively new area for cloud security and is still maturing.

Whatever they are labelled as, these solutions are addressing the same problem: how do you know that your workloads are configured correctly across a hybrid environment? This component will include any DevOps tools implemented to orchestrate the deployment and ongoing configuration management of solutions deployed to private and public cloud platforms. You can also include solutions that will scan for, and potentially enforce, configuration compliance with multiple regulatory and industry standard frameworks.

Information Security: This component is critical to securing data at rest and in transit, regardless of the storage: endpoint, portable, or cloud storage. This component is important to cover secure collaboration, digital rights management, securing email (in conjunction with component 02, Productivity Services), scanning for regulated data and other sensitive information.

The Cloud Security Reference Framework is meant to be a guide as to what services are needed to secure your cloud implementation. In the next section, we will look at the SOC in more detail.

SOC platform components

As described earlier, the SOC platform includes a range of technologies to assist with the routine and reactive procedures carried out by various teams. Each of these solutions should help the SOC analysts to perform their duties at the most efficient level to ensure a high degree of protection, detection, and remediation.

The core components of the SOC include log management and Security Incident and Event Monitoring (SIEM), Security Orchestration and Automated Response (SOAR), Vulnerability Management, Threat Intelligence, and Incident Response. All of these components are addressed by the deployment of Azure Sentinel. Additional solutions will be required, and integrated, for other SOC platform capabilities such as Intrusion Prevention/Detection, integrity monitoring, and disaster recovery:

Deploying a SOC using Azure Sentinel comprises the following components:

Azure Monitor for data collection and analysis. This was originally created to ensure a cloud-scale log management solution for both cloud-based and physical data-center-based workloads. Once the data is collected, a range of solutions can then be applied to analyze the data for health, performance, and security considerations. Some solutions were created by Microsoft, and others created by partners.Azure Sentinel was developed to address the need for a cloud-native solution as an alternative, or as a complimentary solution, to the existing SIEM solutions that have become a mainstay of security and compliance over the last decade. The popularity of cloud services provides some key advantages, including reduced cost of storage, rapid scale compute, automated service maintenance, and continuous improvement as Microsoft creates new capabilities based on customer and partner feedback.

One of the immediate benefits of deploying Azure Sentinel is the rapid enablement without the need for costly investment in the supporting infrastructure, such as servers, storage, and complex licensing. The Azure Sentinel service is charged based on data consumption, per-gigabyte per month. This allows the initial deployment to start small and grow as needed until full scale deployment and maturity can be achieved.

Ongoing maintenance is also simplified as there are no servers to maintain or licenses to renew. You will want to ensure regular optimization of the solution by reviewing the data ingestion and retention for relevance and suitability. This will keep costs reasonable and improve the quality of data used for threat hunting.

Logic Apps provides integrations with a vast array of enterprise solutions, ensuring workflows are connected across the multiple cloud platforms and in existing on-premises solutions. While this is initially an optional component, it will become a core part of the integration and automation (SOAR) capabilities of the platform.

Logic Apps is a standards-based solution that provides a robust set of capabilities, however there are third-party SOAR solutions available if you don’t want to engineer your own automation solutions.

Mapping the SOC architecture

To implement a cohesive technical solution for your SOC platform, you need to ensure the following components are reviewed and thoroughly implemented. This is best done on a routine basis and backed up by regularly testing the strength of each capability using penetration testing experts that will provide feedback and guidance to help to improve any weaknesses.

Log management and data sources

The first component of a SOC platform is the gathering and storing of log data from a diverse range of systems and services across your IT environment. This is where you need to have careful planning to ensure you are collecting and retaining the most appropriate data. Some key considerations we can borrow from other big data guidance is listed here:

Variety: You need to ensure you have data feeds from multiple sources to gain visibility across the spectrum of hardware and software solutions across your organization.Volume: Too large a volume and you could face some hefty fees for the analysis and ongoing storage, too small and you could miss some important events that may lead to a successful breach.Velocity: Collecting real-time data is critical to reducing response times, but it is also important that the data is being processed and analyzed in real time too.Value/Veracity: The quality of data is important to understand meaning; too much noise will hamper investigations.Validity: The accuracy and source of truth must be verified to ensure that the right decisions can be made. Volatility: How long is the data useful for? Not all data needs to be retained long term; once analyzed, some data can be dropped quickly.Vulnerability: Some data is more sensitive than others, and when collected and correlated together in one place, can become an extremely valuable data source to a would-be attacker.Visualization: Human interpretation of data requires some level of visualization. Understanding how you will show this information to the relevant audience is a key requirement for reporting.

Azure Sentinel provides a range of data connectors to ensure all types of data can be ingested and analyzed. Securing Azure Monitor will be covered in Chapter 2, Azure Monitor – Log Analytics and connector details will be available in Chapter 3, Data Collection and Management.

Operations platforms

Traditionally a SIEM was used to look at all log data and reason over it, looking for any potential threats across a diverse range of technologies. Today there are multiple platforms available that carry out similar functionality to the SIEM, except they are designed with specific focus on a particular area of expertise. Each platform may carry out its own log collection and analysis, provide specific threat intelligence and vulnerability scanning, and make use of machine learning algorithms to detect changes in user and system behavior patterns.

The following solutions each have a range of capabilities built in to collect and analyze logs, carry out immediate remediations, and report their findings to the SIEM solution for further investigation:

Identity and Access Management (IAM): The IAM solution may be made up of multiple solutions, combined to ensure the full life cycle management of identities from creation to destruction. The IAM system should include governance actions such as approvals, attestation, and automated cleanup of group and permission membership. IAM also covers the capability of implementing multi-factor authentication: a method of challenging the sign-in process to provide more than a simple combination of user ID and password. All actions carried out by administrators, as well as user-driven activities, should be recorded and reported to the SIEM for context.

Modern IAM solutions will also include built-in user behavior analytics to detect changes in baseline patterns, suspicious activities, and the potential of insider-threat risks. These systems are also integrated with a CASB solution to provide session-based authentication controls, which is the ability to apply further restrictions if the intent changes, or access to higher sensitivity actions are required. Finally, every organization should implement privileged access management solutions to control the access to sensitive systems and services.

Endpoint Detection and Response (EDR): Going beyond anti-virus and anti-malware, a modern endpoint protection solution will include the ability to detect and respond to advanced threats as they occur. Detection will be based not only on signature-based known threats, but also on patterns of behavior and integrated threat intelligence. Detection expands from a single machine to complete visibility across all endpoints in the organization, both on the network and roaming across the internet.

Response capabilities will include the ability to isolate the machine from the network, to prevent further spread of malicious activities, while retaining evidence for forensic analysis and provide remote access to the investigators. The response may also trigger other actions across integrated systems, such as mailbox actions to remove threats that executed via email or removing access to specific files on the network to prevent further execution of the malicious code.

Cloud Access Security Broker (CASB): A CASB is now a critical component in any cloud-based security architecture. With the ability to ingest logs from network firewalls and proxy servers, as well as connecting to multiple cloud services, the CASB has become the first point of collation for many user activities across the network, both on-premises and when directly connected to the internet. This also prevents the need to ingest these logs directly into the SIEM (saving on costs), unless there is a need to directly query these logs instead of taking the information parsed by the CASB.

A CASB will come with many connectors for deep integration into cloud services, as well as connection to the IAM system to help to govern access to other cloud services (via SSO) acting as a reverse-proxy and enforcing session-based controls. The CASB will also provide many detection rule templates to deploy immediately, as well as providing the ability to define custom rules for an almost infinite set of use cases unique to your organization. The response capabilities of the CASB are dependent on your specific integrations with the relevant cloud services; these can include the ability to restrict or revoke access to cloud services, prevent the upload or download of documents, or hide specific documents from the view of others.

Cloud Workload Protection Platform (CWPP): The CWPP may also be known as a Cloud Security Posture Management (CSPM) solution. Either of these will provide a unique capability of scanning and continually monitoring systems to ensure they meet compliance and governance requirements. This solution provides a centralized method for vulnerability scanning and carrying out continuous audits across multiple cloud services (such as Amazon Web Services (AWS) and Azure) while also centralizing the policies and remediation actions.

Today there are several dedicated platforms for CWPP and CSPM, each with their own specialist solutions to the problem, but we predict this will become a capability that merges with the CASB platforms to provide a single solution for this purpose.

When these solutions are deployed, it is one less capability that we need the SIEM to provide; instead, it can take a feed from the service to understand the potential risk and provide an integration point for remediation actions.

Next Generation Firewall (NGFW): Firewalls have been the backbone of network security since the 1980s and remain a core component for segmentation and isolation of internal networks, as well as acting as the front door for many internet-facing services. With NGFW, not only do you get all of the benefits of previous firewall technologies, but now you can carry out deep packet inspection for the application layer security and integrated intrusion detection/prevention systems. The deployment of NGFW solutions will also assist with the detection and remediation of malware and advanced threats on the network, preventing the spread to more hosts and network-based systems.

As you can see from these examples, the need to deploy a SIEM to do all of the work of centrally collecting and analyzing logs is in the past. With each of these advanced solutions deployed to manage their specific area of expertise, the SIEM focus changes to look for common patterns across the solutions as well as monitoring those systems that are not covered by these individual solutions. With Azure Sentinel as the SIEM, it will also act as the SOAR: enabling a coordinated response to threats across each of these individual solutions, preventing the need to reengineer them all each time there is a change in requirements for alerting, reporting, and response.

Threat intelligence and threat hunting

Threat intelligence adds additional context to the log data collected. Knowing what to look for in the logs and how serious the events may be, requires a combination of skills and the ongoing intelligence feed from a range of experts that are deep in the field of cybercrime research. Much of this work is being augmented by Artificial Intelligence (AI) platforms; however, a human touch is always required to add that gut-feeling element that many detectives and police offices will tell you they get from working their own investigations in law enforcement.

SOC mapping summary

The following diagram provides a summary of the multiple components that come together to help to make up the SOC architecture, with some additional thoughts when implementing each one:

Figure 1.3 – SOC mapping summary

The solution works best when there is a rich source of log data streaming into the log management solution, tied in with data feeds coming from threat intel and vulnerability scans and databases. This information is used for discovery and threat hunting and may indicate any issues with configuration drift. The core solutions of the SOC operations include the SIEM, CASB, and EDR, amongst others; each with their own End User Behavior Analytics (EUBA) and SOAR capabilities. Integrating these solutions is a critical step in minimizing the noise and working toward improving the speed to response. The outcome should be the ability to report accurately on the current risk profile, compliance status, and clearly communicate in situations that require immediate response and accurate data.

Security solution integrations

Azure Sentinel is designed to work with multiple security solutions, not just those that are developed by Microsoft.

At the most basic level, log collection and analysis are possible from any system that can transmit their logs via the Syslog collectors. More detailed logs are available from those that connect via the CEF standard and servers that share Window Event logs. The preferred method, however, is to have direct integration via APIs to enable a two-way communication and help to manage the integrated solutions. More details of these options and covered in Chapter 3, Data Collection and Management.

Common Event Format (CEF)

CEF is an industry standard format applied to Syslog messages, used by most security vendors to ensure commonality between platforms. Azure Sentinel provides integrations to easily run analytics and queries across CEF data. For a full list of Azure Sentinel CEF source configurations, review the article at: https://techcommunity.microsoft.com/t5/Azure-Sentinel/Azure-Sentinel-Syslog-CEF-and-other-3rd-party-connectors-grand/ba-p/803891.

Microsoft is continually developing the integration options. At the time of writing, the list of integrated third-party solution providers includes the following:

AWSBarracudaCheckpointCiscoCitrix Systems Inc.CyberArkExtraHop NetworksF5 NetworksFortinetOne Identity LLC.Palo Alto NetworksSymantecTrendMicroZscaler

As you can see from this list, there are many of the top security vendors already available directly in the portal. Azure Sentinel provides the ability to connect to a range of security data sources with built-in connectors, ingest the logs data, and display using pre-defined dashboards.

Cloud platform integrations

One of the key reasons you might be planning to deploy Azure Sentinel is to manage the security for your cloud platform deployments. Instead of sending logs from the cloud provider to an on-premises SIEM solution, you will likely want to keep that data off your local network, to save on bandwidth usage and storage costs.

Let’s take a look at how some of these platforms can be integrated with Azure Sentinel.

Integrating with AWS

AWS provides