Microsoft Defender for Endpoint in Depth E-Book

Paul Huijbregts

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 522

Veröffentlichungsjahr: 2023

Beschreibung

With all organizational data and trade secrets being digitized, the threat of data compromise, unauthorized access, and cyberattacks has increased exponentially. Microsoft Defender for Endpoint (MDE) is a market-leading cross-platform endpoint security solution that enables you to prevent, detect, investigate, and respond to threats. MDE helps strengthen the security posture of your organization.
This book starts with a history of the product and a primer on its various features. From prevention to attack surface reduction, detection, and response, you’ll learn about the features, their applicability, common misconceptions, and caveats. After planning, preparation, deployment, and configuration toward successful implementation, you’ll be taken through a day in the life of a security analyst working with the product. You’ll uncover common issues, techniques, and tools used for troubleshooting along with answers to some of the most common challenges cybersecurity professionals face. Finally, the book will wrap up with a reference guide with tips and tricks to maintain a strong cybersecurity posture.
By the end of the book, you’ll have a deep understanding of Microsoft Defender for Endpoint and be well equipped to keep your organization safe from different forms of cyber threats.

Leseprobe

Microsoft Defender for Endpoint in Depth

Take any organization's endpoint security to the next level

Paul Huijbregts

Joe Anich

Justen Graves

BIRMINGHAM—MUMBAI

Microsoft Defender for Endpoint in Depth

Copyright © 2023 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Mohd Riyan Khan

Publishing Product Manager: Mohd Riyan Khan

Senior Editor: Romy Dias

Technical Editor: Shruthi Shetty

Copy Editor: Safis Editing

Project Coordinator: Ashwin Kharwa

Proofreader: Safis Editing

Indexer: Hemangini Bari

Production Designer: Prashant Ghare

Marketing Coordinator: Ankita Bhonsle

First published: January 2023

Production reference: 1020223

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80461-546-1

www.packtpub.com

I would like to dedicate this book to everyone that has supported not only this book but also my personal journey in moving from the Netherlands across the world in the middle of a pandemic, to welcome me as a part of this amazing international MDE team – thanks for keeping a spot open for me.

– Paul Huijbregts

To my father; enjoy retirement this year, you deserve it! Love you.

– Joe Anich

To Bryan Shaffer, without whom I might’ve been a pharmacist, and Adam Kerby, without whom I’d have had to research my own electronics.

– Justen Graves

Contributors

About the authors

With almost 20 years of industry experience and relevant certifications, Paul Huijbregts has a long history of working with customers across the world leveraging his passion for (Microsoft) security solutions – and being brutally honest about them.

After joining Microsoft in 2016 and engaging regularly with Defender for Endpoint teams, Paul moved to Redmond (together with his wife and kids) to join them and become a product manager – in the middle of the pandemic (October 2020). Here, he is on what is called the “Platforms” team, working on solutions across operating systems and environments, focusing primarily on server endpoints and security management. His motto is: “I drink beer and I know Microsoft security things.”

I would like to thank my wife and kids for giving me the space and time (and the beer money) required to keep writing. In addition, big thanks to the infosec community for their continued support, my peers, and most of all some of the excellent subject matter experts that have been working with the product for much, much longer than I have – thanks for your passion, dedication, and entertaining this crazy PM that was asking lots of questions over lunch or coffee.

Joe Anich has 15 years of experience in the IT industry ranging from endpoint management with a focus on SCCM and Intune to endpoint security and incident response. Currently working on Microsoft’s Detection and Response Team (DART), he works closely with customers during critical moments. Working in incident response has given Joe insight into SOC operations and how to help teams around the world improve their security posture as a whole. Outside of work, Joe enjoys running around the house with his 2-year-old son playing “chase me.” Fun fact: During the late 90s, Joe could be found at the roller-skating rink most Friday nights, gliding around the rink with a super rope in hand, maybe in JNCOs or Lee Pipes, vibing to 90s hip hop.

I want to thank my beautiful wife, Katie, for running the household during my chaotic work schedule and yet still allowing me to pursue my passion to write this book in whatever hours were left of the day. My success comes from your willingness to support me. Thanks for all you do, and for being the best mother little Z could ask for.

Justen Graves is a security engineer with 14 years of IT experience. Most of his career has been focused on endpoint enablement and security, with the last 4 years spent at Microsoft. Currently working in Microsoft’s Cyber Defense Operations Center, their internal SOC, he uses tools such as Microsoft Defender for Endpoint every day to defend corporate Microsoft from attack.

Justen has a BS in cybersecurity and an MBA. He holds many industry certifications, including CISSP, PMP, and GSEC, and several Microsoft certifications, including Azure Solutions Architect Expert and Enterprise Administrator Expert. Starting his career at Walmart and managing to never relocate, he resides in Northwest Arkansas with his wife and three children.

I want to thank my beautiful wife, Paula, for all the support and compassion while I struggled through this book... immediately after an MBA. She and my children, Andrew, Sloane, and Ember, as well as my mom, Sharon, were incredibly supportive and patient with me throughout this time and I truly couldn’t have done it without them.

I’d also like to thank the DSR SOC team and leadership for their support, as well as my Microsoft family for all the knowledge shared with me along the way. It would take pages to list all the fantastic people I’ve encountered at Microsoft. I do want to show explicit appreciation to Joe and Paul for letting me come along on this journey, without whose perseverance this book never would have happened.

About the reviewers

Ian Hoyle has worked in the IT field for over 30 years, since the inception of the internet, in Australia, as a principal architect at the world’s largest mining company, and more recently at Microsoft, holding a number of technical roles, currently as a senior security technical specialist. His interest in IT security was triggered by a visit to Israel in 2016 for the internal launch of what was then called Windows Defender Advanced Threat Protection.

He received two BSc (Hons) degrees in theoretical physics and geophysics too long ago (!!) and then went on to receive a Ph.D. in geophysics. Like so many people in engineering and science, he has ended up in IT and in the security field, which he loves.

I’d like to thank the authors and the publisher for being invited to act as a reviewer of this book. It has been a lot of work but a lot of fun, so thanks!

Kshitij Kumar is a director for the Microsoft Detection and Response Team (DART). Over the course of his career, he has specialized in the forensic analysis and triage of endpoints (Windows, Linux, and macOS devices) as well as Azure environments to perform incident response investigations at scale. Throughout his time working in DART and previous roles with the CrowdStrike Services team, Kshitij has supported hundreds of customers facing advanced adversaries. He has spoken at the Mandiant mWISE conference as well as Black Hat Arsenal, sharing tools and hunting methodologies with his peers in the spirit of collaboration and contribution.

Special thanks

Editorial Reviewer: Holly Burmaster

Content contributors

Attack Surface Reduction: Sujit Magar

Network Protection: Alex Schuldberg

ZEEK Integration with MDE: Elad Soloman and Inbar Rotem

Cloud-Delivered Protection/BaFS: Matt McCormack and Mady Marinescu

X-Plat: Tudor Dobrila, Srinivas Koripella, and John Nix

History, AV: Mady Marinescu and Tudor Coserea

History, EDR: Michael Shalev and Heike Ritter

History, Microsoft Threat Experts/Defender Experts: Tommy Blizzard (MTE), Brian Hooper (DEX), Chris Riggs (DEX-H), and Rani Lofstrom

Device Control: Yuji Aoiki

Web Protection: Thomas Doucette

Security Operations and Advanced Hunting: Carlo Garza, Emily Hacker, Maxwell Peterson, Chris Smith, and Joshua Woods

Troubleshooting: Yong Rhee

Table of Contents

Preface

Part 1: Unpacking Microsoft Defender for Endpoint

1

A Brief History of Microsoft Defender for Endpoint

It all started in Romania…

The early days of antimalware

At the Forefront

A cloud was born

Making sense of it

Rapid innovation

Expanding coverage

Defender everywhere

Microsoft Defender experts

Milestone 1 – Microsoft Threat Experts

Milestone 2 – growing and scaling

Milestone 3 – Microsoft Defender Experts

Summary

2

Exploring Next-Generation Protection

What is next-generation protection?

Breaking down client-side protection

Client-side engines

RTP

Security intelligence

Scan types

Running modes

Exclusions

Expanding on cloud-delivered protection

Cloud-based engines

Automatic sample submissions

BAFS

Dynamic security intelligence

Block levels

Tamper protection

Web protection

Leveraging SmartScreen and Network Protection clients together

Device control

Reporting

Summary

3

Introduction to Attack Surface Reduction

What is attack surface reduction?

Examining ASR rules

The philosophy behind ASR rules

Rule categories and descriptions

Operating modes

Exclusions

Analyzing ASR telemetry using AH

Network protection layers and controls

Custom indicators

Operating modes

CFA ransomware mitigations

Operating modes

Story from the field

Exploit protection for advanced mitigations

Summary

4

Understanding Endpoint Detection and Response

Clarifying the difference between EDR and XDR

Digging into the components of EDR

Telemetry components

How telemetry is gathered

Zeek integration

Understanding alerts and incidents

How alerts and incidents are generated

Alerts overview

Incidents overview

Reviewing entities and actions

Files

Other entities

Submitting files to Microsoft

Action center

Exploring enhanced features

Threat analytics

Advanced hunting

Microsoft Defender Experts

Summary

Part 2: Operationalizing and Integrating the Products

5

Planning and Preparing for Deployment

Architecting a deployment framework

Understanding personas

Leadership

IT admins

Security admin

Security operations

Gathering data and initial planning

Defining scope

Performing discovery

Analyzing the results

Planning your deployment

Creating buckets

Taking a gradual approach

Selecting your deployment method

Understanding security operations needs

Creating a backout plan

Some key considerations per feature

Adoption order

Next-generation protection

Attack surface reduction

Endpoint detection and response

Other platforms

Summary

6

Considerations for Deployment and Configuration

Operating system specifics and prerequisites

Understanding monitoring agents

Supported operating systems

Operating system specifics

Prerequisites

Configuration options for the portal

General options

Licenses

Email notifications

Auto remediation

Permissions

APIs

Rules

Configuration management

Device management

Network assessments

Selecting your deployment methodology

Onboarding packages and installers

Group policy

Intune

Microsoft Defender for Cloud

Other deployment methods

Configuration management considerations

Shell options

Group policy

Mobile Device Management (Intune)

Microsoft Endpoint Configuration Manager

Security management for Microsoft Defender for Endpoint

Summary

7

Managing and Maintaining the Security Posture

Performing production readiness checks

Considerations for connectivity

Enabling Defender Antivirus capabilities

Attack surface reduction

Endpoint detection and response

Server-specific settings

Staying up to date

Windows

Linux and macOS

Gradual rollout

Maintaining security posture through continuous discovery and health monitoring

Sensor health and operating system

Intune reports

ConfigMgr reports

Getting started with vulnerability management

Dashboard

Security recommendations

Remediation

Inventories

Weaknesses

Event timeline

Summary

Part 3: Operations and Troubleshooting

8

Establishing Security Operations

Getting started with security operations

Portal familiarization

Security operations structure

Understanding attacks

The Cyber Kill Chain as a framework

MITRE ATT&CK™ framework

Case study – defining a modern attack

Triage and investigation

Antimalware detections and remediations

Considering alert verbiage

Managing incidents

Performing initial triage

Moving into investigation and analysis

Responding to threats

Files and processes

URLs and IP addresses

Device response actions

Putting it into practice

Threat hunting

Go hunt

Further investigation and threat hunting

Creating custom detection rules

Summary

9

Troubleshooting Common Issues

Ensuring the health of the operating system

Windows

Linux

macOS

Checking connectivity

Connectivity quick checks and common issues

Client analyzer

Capturing network packets using Netmon

Overcoming onboarding issues

Troubleshooting onboarding issues

MMA versus the new unified agent

Custom indicators

Web content filtering

Resolving policy enablement

Checking settings

Addressing system performance issues

Windows

Linux performance

macOS performance

Navigating exclusion types to resolve conflicting products

Submitting a false positive

Exclusions versus indicators

Understanding your update sources

Comparing files

Bonus – troubleshooting book recommendations

Summary

10

Reference Guide, Tips, and Tricks

Useful commands for use in daily operations

PowerShell reference

MpCmdRun

macOS/Linux

Tips and tricks from the experts

Online resources

Reference tables

Processes

ASR rules

Settings

Logs and other useful output

Useful logs

Summary

Index

Other Books You May Enjoy

Part 1: Unpacking Microsoft Defender for Endpoint

In this part, you will learn about the history of the product and will then be provided with a primer for each aspect of the product areas. You will gain a deeper understanding of its features, their applicability, and how they can benefit the security posture.

The following chapters will be covered in this section:

Chapter 1, A Brief History of Microsoft Defender for EndpointChapter 2, Exploring Next-Generation ProtectionChapter 3, Introduction to Attack Surface ReductionChapter 4, Understanding Endpoint Detection and Response

1

A Brief History of Microsoft Defender for Endpoint

This brief history captures, at a very high level, the evolution of Microsoft’s endpoint security solutions—a journey that has, at the time of writing, gone on for nearly a quarter of a century. By no means should it be seen as complete; however, a lot can be learned about a product by understanding how and why it became what it is.

It all started in Romania…

…at a company called GeCAD. Established in 1992 by Radu Georgescu, GeCAD originally focused on creating computer-aided design (CAD) software. In 1994, however, it reached out to Costin Raiu about distributing a commercial version of a virus scanner he had been distributing for free. Raiu had gained interest in viruses after a virus called BadSectors.3428 infected his school as a youth. He spent that evening writing his first successful cleaner utility to help remediate this virus, the whole time worried someone else would beat him to it. Afterward, he got requests from his friends to reverse-engineer other viruses and create cleaner tools for them as well. Eventually, this led to Raiu developing and freely distributing a full-fledged antivirus scanner called Mscan. Once acquired by GeCAD, the first antivirus software produced was named RAV (short for RSN Antivirus, though the name behind the acronym was later changed to Reliable Antivirus) and sold commercially.

Partnered with Raiu at GeCAD on the RAV development project was Mady Marinescu, and in the early days, the rest of the team was mostly comprised of recent university graduates writing virus definitions at a small kitchen table. In 1998, Raiu moved on to a new opportunity at Kaspersky Lab just a year after it was established, most likely due to becoming friends with Eugene Kaspersky over virus definition conversations online. That same year, GeCAD shifted focus heavily to (email server) security. It offered antispam and content filtering for Exchange but also for other common email platforms such as Sendmail and qmail. Development on RAV continued by Mady and team, and though it was considered a cross-platform product, development at GeCAD was primarily focused on meeting the growing security needs of Linux users. This is ironic because, in 2003, the RAV technology and its developers were acquired by Microsoft.

Cold snack

Note that in the late 90s, the focus of security solutions was mostly on viruses. Malware and spyware became popular later, around the year 2000.

The early days of antimalware

In 2004, Microsoft bought another company, called GIANT AntiSpyWare, which was based in New York. Its technology, focused on antispyware, was merged into the antivirus product that was acquired through the GeCAD acquisition. A key technology called SpyNet (for which you can still find references in the Windows registry) eventually evolved into Microsoft Active Protection Service (MAPS), which, in turn, is the foundation for cloud-delivered protection.

For Windows XP and Windows Vista, Microsoft then published Windows Live OneCare. This was a paid consumer offering that included a variety of capabilities, including antimalware, anti-phishing, and a firewall, and it included real-time protection.

The Defender brand started life on Windows XP, and eventually shipped with Windows 7 as an antispyware solution, initially porting over the product that was acquired with GIANT. Early on, it was revamped into a unified code base to replace the internals; the engine was now also capable of providing antivirus/antimalware if provided with the right signatures. Customers that wanted to upgrade from Defender to full antimalware protection could download and install Microsoft Security Essentials (MSE). The user interface for this was the first project based out of the Israel Development Center (ILDC). It was the equivalent of Forefront Endpoint Protection (FEP)—but for consumers.

Cold snack

You may also remember an ActiveX component called Windows Live Safety Scanner, which offered on-demand scans without requiring any installation. After a few standalone tools that were released for specific outbreaks, such as Blaster and Sasser, Microsoft started regularly publishing the Malicious Software Removal Tool (MSRT) – essentially, an antimalware engine with a limited set of signatures. The Windows Live Safety Scanner later evolved into Microsoft Safety Scanner/Microsoft Emergency Response Tool (MSERT), bringing the full Defender signature set.

In 2008, the company Komoku was acquired. It focused on rootkit detection by statically analyzing the running state of a system, with the purpose of flagging rootkits by finding anomalies in the kernel. This rootkit detection was then added to the Forefront product.

At the Forefront

The Forefront family was Microsoft’s first step toward establishing a suite of security solutions: combining primarily existing products under the Forefront flag such as Threat Management Gateway, Unified Access Gateway, and FEP. The latter was Microsoft’s first commercial endpoint protection solution that used the same engine that was, by now, the foundation of Windows Live Defender/MSE. FEP 2007 (and later, 2010) was then adopted by System Center to become part of the System Center Configuration Manager product; it was later rebranded as System Center Endpoint Protection (SCEP). This brought endpoint protection management and deployment together with a broader set of capabilities for managing and maintaining operating systems.

Cold snack

SCEP even provided a basic antimalware agent for macOS and Linux. If you had the right license, you would go to the Volume Licensing Service Center (VLSC) to download the installation packages. These were later deprecated and left a gap until Microsoft decided to build new solutions under the Microsoft Defender Advanced Threat Protection (ATP) brand.

In 2012, Windows 8 was the first Windows version to ship with what is the foundation of the full, modern Defender as you know it in Windows 10. The Windows Defender name was brought back. It could still be brought under management via System Center (Configuration Manager) Endpoint Protection. The Endpoint Protection role inside modern-day Microsoft Configuration Manager deployment (now in the Microsoft Intune family) continues to allow management of endpoint protection on Microsoft Endpoint Manager (MEM)-supported operating systems, regardless of which client components are installed.

Cold snack

Starting with Windows 8, because Windows Defender was installed and enabled by default, the automatic detection and disablement of third-party antimalware was introduced: see running modes for more information on how this affects the effective running mode of Windows Defender Antivirus (Defender Antivirus).

A cloud was born

Shortly after, between 2013 and 2015, the Windows Defender team started using the Windows telemetry collection pipeline to start streaming Defender AV telemetry. Soon after, they added telemetry from SCEP and MSRT (which, by then, were deployed on over a billion devices) to a data lake. This data lake was hosted on what can be considered an internal cloud (a precursor of Microsoft Azure) alongside Bing telemetry, and the raw telemetry was cooked to generate processed entity profiles including file, process, and network. This enabled querying vast volumes of data to identify all occurrences of a given entity in a performant manner. The team also applied a real-time streaming analytics engine called Stream Insights to the incoming telemetry. This allowed them to perform real-time malware detection, creating one of the foundations for what is now called cloud-delivered protection—a major milestone in the evolution of Defender Antivirus to a true machine learning (ML)-powered, next-generation endpoint protection solution.

Around 2015, cloud operations for the product were moved to Microsoft’s ILDC, where today, Sense, the endpoint detection sensor in the Microsoft Defender for Endpoint (MDE) product is developed. Before Sense, SCEP could, in fact, act as an endpoint detection and response (EDR) sensor, but required very aggressive cloud communication. Though this resulted in a heavyweight solution due to having to scan before sending telemetry, it allowed Microsoft to develop the backend for Sense mentioned previously.

Cold snack

Profiles, or event types, introduced through the data lake effort can be found today inside MDE. As an early adopter of Microsoft’s Cosmos NoSQL database, Defender Antivirus’s data lake efforts greatly stimulated the development of EDR until its official release in 2017—it remains in use today to continue to support the staggering worldwide scale needed to protect hundreds of millions of machines. In fact, billions of requests are served daily, likely making the Defender cloud the largest-scale security solution on the planet today.

One of the key goals of establishing a data lake was to provide the ability to perform behavioral analysis to deal with malware that was specifically designed to avoid detection; emulation, a technique to simulate execution, can only go so far in collecting the signals needed to come to a verdict. A way to detect malware that was designed with obfuscation in mind was needed, which shifted the focus to the execution phase into post-breach, away from physical attributes and toward behavioral detection.

The telemetry gathered in the data lake was augmented to include process information from the antivirus, and events from Event Tracing for Windows (ETW), to create profiles for files, network connections, and processes. Then, these were matched against indicators of attack (IoAs).

Cold snack

Microsoft’s security operations center (SOC), the Cyber Defense Operations Center (CDOC), was one of the earliest adopters of what was then called the IOC Storyboard, an Excel file that allowed them to leverage the telemetry to perform pivoting across entities/profiles, and hunt across the data. This extremely popular workbook was quickly adopted by other blue teams inside Microsoft. Today, Microsoft’s digital security division, covering everything from internal IT to security for customer-facing services such as Azure and Office 365, remains one of the biggest users of MDE and is a heavy driver of further product development.

Making sense of it

As the limitations of ETW were reached, and needed an agent that used less bandwidth and fewer machine resources, it became clear what the EDR product should be. Project Seville was started; Sense (which is the name of the EDR sensor) was born. The existing cooked data was used to continue development, and collaboration with the Microsoft blue teams intensified to define more scenarios. To overcome the limitations of ETW, Sense was built into the operating system (Windows 10), and kernel and memory sensors were added as part of operating system development, giving Microsoft Defender ATP deeper optics than ever before.

The following screenshot shows the cloud user interface that was built to replace the Excel workbook that was widely used by internal Microsoft defenders:

Figure 1.1 – Cloud interface that replaced the previously used Excel workbook

Closer to what people may know today, which is what we see in the following screenshot, was version 2:

Figure 1.2 – Second version of the Defender dashboard

Some elements in the current Microsoft 365 Defender portal still bear some resemblance, but the overall experience is vastly different.

Rapid innovation

Since its initial launch in 2016, Microsoft Defender ATP has seen a non-stop progression of new features across prevention, detection, and response capabilities—even expanding into new product categories such as threat vulnerability management, which requires little or no scanning as it uses existing device inventory data.

In December 2017, Defender Antivirus switched to a monthly update model for the product. This allowed for a more rapid release cadence for new features, fixes, and capabilities as releases were no longer tied to Windows. The first version of this monthly update started with 4.12. Windows Server 2016, and simultaneously the first Redstone release of Windows 10 (RS1), shipped with a version starting with 4.10: the same version the latest SCEP client has today, and the reason you need to update the operating system and the antimalware platform to get to the latest versions, which currently start with 4.18.

Windows 10/2016 shipped with new core capabilities, including Exploit Protection, the integration of which was known as the Enhanced Mitigation Experience Toolkit, (EMET), which was a standalone piece of software for earlier Windows versions. The monthly update model facilitated the release of features such as attack surface reduction rules and network protection and really helped to accelerate the evolution of Windows Defender toward an elaborate, feature-rich set of endpoint protection capabilities.

Cold snack

The first monthly updates had a version number starting with 4.12. In 2018, the current versioning format was established, and platform versions started following the 4.18.YYMM format. The engine has been packaged together with definition files since around 2005, and its versioning scheme is the same across all products containing the engine today.

Expanding coverage

At first, partner integrations were the only way to extend coverage to non-Windows operating systems (macOS, Linux, and mobile). These partner integrations leveraged a cloud-to-cloud connection where telemetry was forwarded so that a machine page could be created.

Due to market demand and the evolving threat landscape, in the fall of 2018, Microsoft started working on a new security product for macOS. Microsoft rapidly developed a solution with initially only antimalware capabilities delivered by an off-the-shelf engine (augmented with RTP, manageability, quarantine, and a user interface) and made it generally available in June 2019; later that year, EDR was added to the feature set.

Following the successful release of MDE on macOS, the focus switched to Linux. The general availability of Microsoft Defender ATP for Linux was announced in June 2020. As with macOS, it initially only contained antimalware functionality, with EDR capabilities following later in the same year. Next up were Android and iOS, both released in 2020.

At the same time, work continued to develop a newer, more enhanced engine that was more capable of evolving along with the threat landscape. This not only provides more efficient protection delivered by significant optimization, but it is also very similar to the Windows antimalware engine, allowing developers and researchers to cross-develop for all platforms at the same time; a shared core set of security intelligence automatically provides Windows malware coverage on Linux and macOS. The similarities are no coincidence: as you can read at the start of the chapter, the original team built security solutions primarily for Linux.

Defender everywhere

We started our journey with Defender Antivirus and its predecessors. It is now a product that is protecting hundreds of millions of devices across the world, top scoring in independent AV tests. It sits at the core of the prevention capabilities inside MDE—on Windows, macOS, and Linux, as well as Android and iOS. With attack surface reduction innovations and the expansion to a feature-rich EDR that is continuously battle-tested inside one of the largest solutions and cloud providers in the world (Microsoft), acclaimed by independent testing providers such as MITRE, you have a truly impressive set of security capabilities at your disposal.

Cold snack

MDE is also integrated into other products/suites, including Microsoft Defender for Cloud. Today, it also forms the foundation and an integral part of Microsoft’s extended detection and response (XDR) Microsoft 365 Defender, initially defining the genre by aggressively pursuing cross-suite integration across identities, cloud apps, email, data, and—of course—endpoints. In addition, many other Microsoft cloud services (including other security solutions) use Defender components for endpoint security and also behind the scenes.

Microsoft Defender experts

From early in the development of MDE, or as it was first called, Windows Defender Advanced Threat Protection (ATP), Microsoft’s research team partnered with MSTIC to produce one-pagers that would be linked in your portal to alerts that could be attributed to known actors (another example of a collaboration with MSTIC is the capability known as Threat Analytics), focusing on stages in the kill chain identifying lateral movement, ransomware, and network activity to profile them.

This capability led to a lot of interest from Microsoft’s customers, with a lot of questions about how Microsoft could inform them of trends they were seeing. While Microsoft was able to detect on a global scale through analytics based on anonymous data points and using insights from attacks launched against Microsoft and its cloud services, this was not enough to generate alerts that depended on relevant contextual information. The true value would come from a more managed detection and response (MDR) approach, where just like any MDR service, the team would need to be granted access to actual data from customer environments. Of course, privacy boundaries were in place that could not (and would not) be crossed, and so meeting this customer request required careful navigation of the privacy and compliance impact of creating a service that would interface the collective knowledge of Microsoft’s world-class research team with the context of customer’s MDE data.

In December 2017, the team started engaging with large customers to figure out the right balance between providing a much-requested service and observing the right level of confidentiality needed. Agreements were drafted and refined to ensure they would meet customers’ compliance requirements, and an early pilot program provided much-needed inputs toward how the service could be shaped, to not just serve specific large customers but also to scale and grow with demand.

Initially, this pilot involved monitoring the alert queue and wrapping context around it (such as which malware families were considered riskier). This led to deeper reports at first. Then, moving to a more hands-off approach, the journey continued to find a balance between engaging daily and intensively versus only occasionally or based on specific criticality. Finetuning further with customers, a balanced and appropriate level of detail was found in the targeted attack notifications (TANs, now called Endpoint Attack Notificationsor EANs).

At first, Microsoft’s hunters had to create manual queries to find new signals (among billions) and then evaluate global results for techniques that they were trying to find. Through capturing incidents and learning from them, the set of queries and manual effort grew rapidly. This led to the need for tooling: a platform to store queries and run them, requiring low latency to facilitate timely detections. With the success of the pilot, an investment was made to scale out the team and the tools.

Cold snack

Working through the challenges of building the service, the Microsoft Threat Experts effort also laid the groundwork for much-used features such as Incidents, Threat Analytics, and even Advanced Hunting.

Milestone 1 – Microsoft Threat Experts

Taking the now matured concept to the product and getting more evidence that there was a strong need for customers to be aware of lurking, critical threats in their environment, at RSA in May 2019, the Microsoft Threat Experts (MTE): Targeted Attack Notification (TAN, later EAN) service was launched, as a lightweight addition to Microsoft Defender for Endpoint, into General Availability. This was free of charge for customers that opted into it.

In October 2019, Experts on Demand was added as a premium (paid) capability to support customers that needed to follow up on alerts or TANS/EANs and needed help, providing a trusted path for organizations to leverage additional expertise in dealing with advanced attacks.

Microsoft Defender for Endpoint, through integration with other security services such as (at the time) Office 365 Advanced Threat Protection, Microsoft Cloud App Security, and Azure Advanced Threat Protection, became a part of the larger suite of products called Microsoft Threat Protection (which then evolved into Microsoft 365 Defender, Microsoft’s XDR solution).

This led to an increasing demand for MTE to cover these other security services, an expansion of their scope. Based on this customer feedback, the MTE team started incubating this idea around 2020, beginning by hunting across the full suite as opposed to only endpoint data.

The other strong feedback was that a lot of customers needed more help to manage everything within Microsoft Threat Protection – dealing with the workloads, alerts, incidents, and threats daily.

Milestone 2 – growing and scaling

With the increasing number of customers using Microsoft Defender for Endpoint and the Microsoft Threat Experts service, scaling became a very important topic. Investments were made into systems that could help more quickly surface and analyze potential threats at a very large scale, leveraging machine learning. Most importantly, it provided accurate prioritization to identify the most serious threats.

The large-scale automation in the hunting systems, combined with the increased demand for help from customers, opened the path for the development of managed security services. This led to an incubation effort to investigate what would be the best way to build and provide the required services.

Milestone 3 – Microsoft Defender Experts

In 2022, at RSA, Microsoft launched Microsoft Security Experts, a new service category containing the now further evolved Microsoft Threat Experts capabilities:

Microsoft Defender Experts for Hunting: This service is an evolution of MTEs EAN’s, now covering all of Microsoft 365 Defender – providing a new type of targeted attack notification called Defender Experts Notification (DEN) as an add-on to the productMicrosoft Defender Experts for XDR (extended detection and response): This new service adds managed detection and response to the full scope of Microsoft 365 Defender, meaning that Microsoft analysts will monitor and respond to your incidents alongside existing customer teams and automation

Cold snack

Experts on Demand became a core component of these larger services, allowing you to request the help of an expert, in context, from any threat in the Microsoft 365 Defender portal.

Finally, under the name of Microsoft Security Services for Enterprise, Microsoft now offers comprehensive Managed Security Services Provider (MSSP) services combining hunting, detection, and response for both Microsoft’s XDR as well as SIEM; in addition, delivering practice modernization, onboarding, and incident response across the enterprise environment.

Summary

The history in this chapter highlights the drastic evolution of the product from antispyware to a critical SOC tool, to a full endpoint prevention, detection, and response suite, and provides key insights into the strategy behind it, including the evolution of Microsoft Defender Experts. This sets the stage for the following chapters, starting with—just like Defender’s journey—core prevention capabilities.

2

Exploring Next-Generation Protection

In this chapter, we are going to cover the main components in the next-generation protection area of Microsoft Defender for Endpoint (MDE). There is a lot that can be covered here, and our aim is to fill some voids for some while heavily ramping others when it comes to what these products are and how they work. We’ll cover everything from the antivirus aspect of next-gen, how cloud-delivered protection fits into the fold, everything tamper protection has to offer, as well as web and device control. Where possible, we attempt to ensure concepts apply to most if not all operating systems.

As just mentioned, the chapter will be laid out in the following order:

What is next-generation protection?Breaking down client-side protectionExpanding on cloud-delivered protectionTamper protectionWeb protectionDevice control

What is next-generation protection?

Next-generation protection is the category of capabilities in MDE that focuses on prevention. Comprised of client-side, real-time antivirus protection combined with near-instant cloud-delivered detections of emerging threats, and shored up by dedicated product and protection updates, it helps protect against a variety of threats, including the following:

Virusesand trojansMalwareor spywareRootkits

What makes it next generation? I hear you ask. In the case of MDE, this is a culmination of several evolutions of traditional, signature-based, antimalware solutions, augmented by the power of cloud computing, and fed by extensive research efforts using machine learning (ML). Some key points along the way:

Transitioning from detection through static signatures (single threats), toward definitions (threat families), and offering more robust as well as less resource-intensive protection using heuristics (probability scoring)The introduction of client-side ML models, helping to identify and block malware that was never observed beforeBehavioral monitoring, using context to increase the confidence to incriminate specific binaries through observing a sequence of eventsCloud-based ML models, which serve to constantly support clients in making determinations, increasing precision, and helping to identify more emerging malwareRapid delivery of new definitionsBreadth of signal, leveraging inputs from a vast network of sensorsIn most cases, a simple request to the cloud helps to get a verdict on most malware if local models cannot make an accurate determinationAs a final resort, automatic sample submission is used as a fallback optionBlock at first sight (BAFS) can then even hold unlocking of the file on the endpoint until the cloud analysis pass has completed, preventing patient zero in many cases

Cold snack

Cloud-delivered protection processes billions of requests daily. It’s responsible for helping to protect every single cloud-enabled Windows device on the planet that is running Microsoft Defender Antivirus (Defender Antivirus).

To better understand how Defender Antivirus works, it helps to know what the core components are and what their job is. These are shown in the following table, with a brief description of each’s role in securing your endpoints:

Component

Description

Platform/product/app

This is the foundation that provides manageability interfaces (configuration), updateability, and delivery of the protection stack components. It gets updates through Microsoft Update (Windows), Microsoft Auto Update (MAU, macOS), the update repository (Linux), or the mobile app store (Android/iOS).

Antimalware engine

Leverages the drivers/components provided/installed by the platform or operating system to perform detection.

This gets updated through security intelligence update packages.

Security intelligence

This component is leveraged by the engine to help it identify malicious software and activities. Updates can occur as part of security intelligence updates, in full or as deltas.

User interface (client)

Provides a configuration and management interface, sometimes with limited reporting capabilities. Part of Windows, else part of the platform/app.

Sensors

Refers to any additional components, such as kernel sensors, needed for endpoint detection and response (EDR). Built into Windows or the platform/app used to enable it on non-Windows operating systems.

Table 2.1 – Defender Antivirus components

Cold snack

On Windows, you may have observed driver creation performed by msmpeng.exe. kernel support library driver (KSLDriver) (what you’re seeing as MpKsl*, where * is a wildcard placeholder for a random string of characters) is a driver that’s dynamically dropped/loaded by Defender to aid some tasks (anti-rootkit scanning, Intel’s threat detection technology (TDT), and so on). Defender uses this scheme, where this driver is dynamically dropped and named, to avoid name-squatting attacks.

The preceding general outline of Defender Antivirus components holds true for most operating systems (including Linux and macOS) and product versions where an MDE agent is available; mobile devices are the key exception to this, where everything is bundled into a single app, and security intelligence is primarily provided by cloud lookups.

Cold snack

In Windows 10, 1703, the Defender team released a new capability: parts of the Defender antimalware engine on Windows could now be virtualized (that is, run inside a sandbox). A major focal point for the team is preventing anyone from attacking or abusing core Defender Antivirus components. Understanding that Defender has high privileges that grant it access to the content it inspects, it opens the potential for attackers to abuse or compromise this dynamic. Sandboxing ensures that even if Defender becomes compromised in such a way, it is only within a single context that is isolated from the primary Defender instance running on the host operating system. At the time of writing, due to significant performance and compatibility challenges, only part of this technology is enabled by default today.

Now that we have a foundational understanding of Defender Antivirus, we’ll take a closer look at client-side Antivirus and how it delivers core real-time protection (RTP). Then, we’ll explore cloud-delivered protection and how it makes Defender Antivirus more dynamic and agile in an ever-evolving threat landscape. Finally, we’ll expand on tamper protection as mitigation for attempts to disable the protections outlined and touch on a few protection features that aren’t technically a part of the next-generation protection stack (but also don’t really have a proper home).

Breaking down client-side protection

Defender Antivirus actively protects your devices from the moment they are started. Running in the background, Antivirus scans suspicious binaries and alerts you the moment action is needed, sometimes even taking action on your behalf, and in either case notifying you so that you can investigate. In this section, we’ll break Defender Antivirus down into its component engines, gain an understanding of how that rolls up into RTP, clarify how it gets its security intelligence, and define different scan types, running modes, and exclusion options.

Client-side engines

As mentioned previously, both cloud-based and client-side engines work together to achieve next-generation protection. In this section, we’ll break down the different client-side engines to help illustrate how, when layered together, they create a holistic and dynamic protection stack on the client.

AMSI integration engine

The antimalware scan interface (AMSI) is an open Windows API that allows applications to request a memory buffer scan by an installed security product/Antivirus at runtime. While originally developed to make antivirus integration easy for application developers, today, AMSI has become best known for its integration with scripting engines such as PowerShell, JavaScript, and VBScript. Security products can use AMSI to capture the code a script is executing in memory so that it can be passed through detection logic, or it can be surfaced for review—within EDR products, for example.

Behavior monitoring engine

Designed to detect new and emerging threats or suspicious behavior, behavior monitoring looks for a pattern of events/activities matched against a definition to detect a variety of malicious behavior. When you get to Chapter 4, Understanding Endpoint Detection and Response, you’ll notice that EDR works in much the same way, but in this case, pattern recognition is more focused on preventing malware than detecting attackers.

Emulation engine

To thwart obfuscation techniques such as packed (obfuscation through encryption, compression, and so on) or polymorphic (has code to change its identifiable features on the fly to avoid detection) malware, this engine is used to emulate the execution of the malware and capture any calls or other identifiers that would be missed by static analysis methods.

Heuristics engine

In people, heuristics are cognitive shortcuts that we use to solve problems more quickly by leaning on what we know about the world, rather than by analyzing the problem in front of us deeply. This is especially effective when speed is more important than absolute accuracy. When it comes to endpoint security, heuristics work much the same way. The heuristics engine performs rule-based analysis, much like signature detections, but in this case, the rules are designed to identify characteristics of files that match known malware approaches or some threshold of similar source code. In this way, heuristic analysis can identify new or modified malware much better than static analysis.

ML engine

You may have thought that all the Defender Antivirus ML was happening in the cloud at this point, but there are also lightweight ML models employed client-side. For the data scientists out there, these are predominantly (if not exclusively) supervised ML (SML) models. The goal is to appropriately label, in a general sense, previously unseen entities (files, processes, and so on) as suspicious, malicious, or whichever label best suits the need. Then, logic can be applied for how to surface or handle that entity once it is labeled with a high level of confidence.

Memory scanning engine

Fairly straightforward, the memory scanning engine scans the memory address space associated with a given process. Much as with AMSI and other engines, the goal is to catch malicious code in a state where it’s unmasked/deobfuscated and can be clearly identified and reacted to.

Network engine

Equally straightforward, the network engine inspects network activity and tries to identify whether anything untoward is happening. Due to the integration of products, this engine is the reason RTP must be enabled for the network protection feature of attack surface reduction (ASR), covered in Chapter 3, Introduction to Attack Surface Reduction, to work.

Conclusion

In closing, the key takeaway here is that there are multiple components that make up the engine—many aspects of endpoint protection, including host intrusion prevention system (HIPS)-like mitigations, network threat defense, script scanning, and so on are all covered by the same product. In the end, the term antivirus is much too narrow to accurately describe the full scope of what the product is capable of.

Understanding all of these moving parts, you should now be starting to rationalize how all the pieces fit together to create one unified protection stack. Next, we’ll clearly define RTP and several of its critical features.

RTP

Also known as always-on protection, RTP identifies malware through active scanning, behavior monitoring, and heuristics. More loosely, it’s the term used to describe the set of technologies that are responsible for, well… being there even before you need it, as opposed to manually triggered or on-demand actions. You can think of it as another type of scan: one that is triggered automatically by any type of file access (on-access scanning), activity in memory (memory scanning), script execution (AMSI detection), and so on.

So, what does that mean? It means that Defender steps in and seizes an opportunity to scan what is going on. It uses drivers that sit in various places to be able to intercept what’s going on in the filesystem or memory, and then act on whatever is deemed to be malicious—think of it as existing in the blocking path of potentially unwanted activity. Being in the blocking path comes at a price; one that you should be willing to pay as it greatly reduces the chances of malware getting a foothold. In ye olden days, you had to kick off or schedule a scan, which is way too slow to catch malware in the act; today, you can rely on active, always-on protection to keep a steady eye out on your behalf. RTP is very often the number one suspect when it comes to performance impact. Setting exclusions is the primary mitigation in these cases. But why is this? Let us revisit the blocking path concept. Because the filter drivers sit in between—for example—the disk and the network stack, data flows through them to provide the opportunity for the engine to act on what it is observing. This observation, or scan, is parsed and matched against either definitions or other logic. Naturally, this introduces a slight delay. By itself or even in bulk, this is typically not a problem. However, applications that perform any very high-volume operation, such as repeatedly opening and closing a file or writing to it (think about the concept of a database file here), are at risk of continuously triggering scans, to the point performance starts degrading.

Cold snack

In many cases, when you look at running processes and see Defender is consuming 100% CPU time, what you are observing is an application repeating transactions, causing Defender to act on it by scanning these transactions—leading to extreme overhead. If you then decide to disable RTP, you are not solving the actual problem! The best path to resolution is to investigate which application and/or file is responsible and consider an exclusion. The behavior monitoring aspect of RTP can be thought of as an alternative approach to (traditional) HIPS capabilities. Instead of requiring the authoring of custom rulesets yourself, behavior monitoring can analyze sequences of events to identify malicious behavior that targets or attempts to exploit vulnerabilities in (often legacy) software. The intelligence is authored by various security research teams inside Microsoft using a diverse set of sources to help increase confidence that the activities should be blocked.

This not only takes away the burden of authoring and maintaining rulesets, but it also provides a dynamic framework where Microsoft is on point and continuously building and tweaking rules on your behalf.

IOAV and MOTW

IOfficeAntivirus (IOAV) is the Windows API COM interface used to trigger the scanning of any file that supports the MSOfficeAntivirus component category. That may sound complicated, but most notably, this interface is called by the Windows attachment manager any time a file download and save is attempted by a browser, an instant messenger program, or a mail client (using the IAttachmentExecute::Save interface). When called via the Windows attachment manager, two things happen. First, the file is scanned by any installed and registered antivirus software and may be deleted or altered as a result. Second, what’s called the mark of the web (MOTW) is applied to it. MOTW is simply a set of properties, known collectively as evidence, that are stored in the zone identifier alternate data stream (Zone.Identifier ADS) on a downloaded file. This evidence includes security zone information and relevant URLs. There are five zones defined by default:

Value Setting ------------------------------ 0    My Computer 1    Local Intranet Zone 2    Trusted sites Zone 3    Internet Zone 4    Restricted Sites Zone

To help illustrate what we’re talking about here, what follows is an example of using PowerShell to check the mark of the web manually, as both the Get-Content and Get-Item cmdlets in PowerShell have a -Stream parameter. For this example, we used a browser-downloaded copy of a popular timelining tool created by host forensic tool savant, Eric Zimmerman (@ericzimmermanon Twitter):

PS C:\> Get-Content .\TimelineExplorer.zip -Stream Zone.Identifier [ZoneTransfer] ZoneId=3 ReferrerUrl=https://ericzimmerman.github.io/ HostUrl=https://f001.backblazeb2.com/file/EricZimmermanTools/net6/TimelineExplorer.zip

Note the ZoneId=3, indicating the file was downloaded from the internet. Attachment manager, and subsequently IOAV, are also called by File Explorer when a file with a MOTW is opened or executed. This is a crucial element to BaFS, as it will trigger the flow for it.

A few other programs of note that leverage MOTW are Microsoft Defender SmartScreen (SmartScreen) and Microsoft Office (Office) applications. SmartScreen is built into Windows and so has your back, even if you’re not using Defender Antivirus. It checks any files with a MOTW against a known good list (often referred to as an allowlist), and if the file isn’t present there, it notifies you that the file is unknown and prevents it from running unless you insist it should. It also checks visited web pages against a known list of malicious sites (often referred to as a blocklist