Building a Next-Gen SOC with IBM QRadar E-Book

Ashish M Kothekar

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 266

Veröffentlichungsjahr: 2023

Beschreibung

This comprehensive guide to QRadar will help you build an efficient security operations center (SOC) for threat hunting and need-to-know software updates, as well as understand compliance and reporting and how IBM QRadar stores network data in real time.
The book begins with a quick introduction to QRadar components and architecture, teaching you the different ways of deploying QRadar. You’ll grasp the importance of being aware of the major and minor upgrades in software and learn how to scale, upgrade, and maintain QRadar. Once you gain a detailed understanding of QRadar and how its environment is built, the chapters will take you through the features and how they can be tailored to meet specifi c business requirements. You’ll also explore events, flows, and searches with the help of examples. As you advance, you’ll familiarize yourself with predefined QRadar applications and extensions that successfully mine data and find out how to integrate AI in threat management with confidence. Toward the end of this book, you’ll create different types of apps in QRadar, troubleshoot and maintain them, and recognize the current security challenges and address them through QRadar XDR.
By the end of this book, you’ll be able to apply IBM QRadar SOC’s prescriptive practices and leverage its capabilities to build a very efficient SOC in your enterprise.

Leseprobe

Building a Next-Gen SOC with IBM QRadar

Accelerate your security operations and detect cyber threats effectively

Ashish M Kothekar

BIRMINGHAM—MUMBAI

Building a Next-Gen SOC with IBM QRadar

Copyright © 2023 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Pavan Ramchandani

Publishing Product Manager: Prachi Sawant

Senior Editor: Romy Dias

Technical Editor: Arjun Varma

Copy Editor: Safis Editing

Project Coordinator: Ashwin Kharwa

Proofreader: Safis Editing

Indexer: Pratik Shirodkar

Production Designer: Prashant Ghare

Marketing Coordinator: Marylou Dmello

First published: June 2023

Production reference: 1310523

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80107-602-9

www.packtpub.com

I strongly believe in the power of manifestation. This book is the manifestation of my utmost effort to take up complex subjects, simplify them, and present them to a larger audience.

Foreword

IBM QRadar has been the leader in the security market segment for more than a decade. IBM QRadar is an aggregator that encompasses security products, ingests data from them, correlates the data, and uses artificial intelligence and machine learning to provide insights, alerts, and reports for practitioners and stakeholders. These include analysts, security administrators, government agencies, law enforcement, and executive management. This book gets into the foundational concepts on which QRadar is built, explaining the architecture, services, and different components. It details how QRadar can be used by stakeholders for different purposes, covering QRadar apps, which are customized solutions built by IBM for different security products. It ends with informative crafted chapters on the author’s experience with QRadar and tips for getting the most out of QRadar for your organization’s security.

I have known Ashish for over half a decade and closely worked with him, where he has been instrumental in designing different security solutions, including IBM QRadar and IBM Storage products. He has authored many IBM Redpapers in this area. This book provides an end-to-end view and knowledge of QRadar, including insights into its usage to secure your organization. I definitely recommend it for practitioners across industry verticals.

Sandeep Patil

IBM STSM (IBM Storage CTO Office), IBM Master Inventor, WW Prolific Inventor

Contributors

About the author

Ashish M Kothekar is currently working on the SWAT team for IBM Security products. He has more than 16 years of experience working with IBM. He is the subject matter expert (SME) for IBM products in the threat management segment, including IBM QRadar and Cloud Pak solutions for security. He is the author of many IBM Redpapers written on the collaboration of IBM Security and IBM Storage products. He is an avid tech blogger and writes on various security use cases.

I would like to take this opportunity to thank Yogesh Talekar, my manager, for his consent and for all his help in fast-tracking the formalities required to start this book. Yogesh is more than just a manager; he is also a friend, guide, and philosopher.

The technical reviewers—Boudhyayan Chakrabarty (Bob), Sam Yiu, and Ankit Rai—have helped me dive deep into the technical aspects of QRadar features. Special shout-out to Bob, who has been there through thick and thin in this journey.

My family has stood by me and been a source of encouragement throughout this journey. I would like to sincerely thank them. And to my late grandmother, without whom many things would not have been possible, including this book.

About the reviewers

Samson Yiu is a cybersecurity engineer whose qualifications include a degree in computer science; CISSP, IBM Security Certified Consultant, ITIL, MCSE, and postgraduate certificates in cybersecurity; and detailed knowledge of security architectures and best practices. He has 12 years of experience in the design, implementation, and support of solutions protecting networks, systems, and information assets for Fortune 500 companies throughout Asia. His major strength is in troubleshooting failed deployments and bad architectural implementations. He has served as the SME for QRadar certification exams and is currently engaged as an active NSW cybersecurity ambassador. This is the sixth book that he has been involved in.

I wish to thank my wife, Jessie, and my kids, Edison, Vivian, and Rochelle, for allowing me the time to improve myself to study and write. Cybersecurity as a job did not exist when I graduated from university so I wish those who have chosen this vocation the best of luck in the future as this is an awesome career with infinite learning.

Ankit Rai has more than 10 years’ experience in cybersecurity and has assisted the CISOs, VPs, and heads of cybersecurity departments at various multinational corporations in procurement and doing PoC of the security tools, as well as performing end-to-end deployment, maintenance, and support for various cybersecurity solutions. Ankit Rai is a successful professional who built the first ever fully functional SOC for one of the largest small finance banks. He has a large community across social networking sites and loves to speak about cybersecurity in colleges and universities.

My heart goes out to my family, who made me the man I am today, and to my managers, who always promoted my urge to learn and my unquenchable curiosity. I’d also like to thank my life partner, Monika, who has been with me through thick and thin, night and day, always.

Lastly, I’d like to thank the community who welcomed me with open arms when I was still learning the ropes of information security. Thank you!

Boudhayan Chakrabarty (Bob) is an executive architect in the IBM Security Elite team specializing in threat management. With over 16 years of hands-on experience, Bob has also been a part of many international deployments of security compliance and security intelligence solutions, wherein he contributed starting from the RFP phase to the proof of concept phase to the ultimate deployment and training of the customer. He is a regular speaker on cybersecurity at different conferences and has also authored multiple books and publications. He has also run many enablement and training courses on security intelligence and compliance products. He is an SME for Security Intelligence and Compliance certifications.

I would like to take this opportunity and thank everyone who had supported me in this journey, especially all my mentors. Special mention to Yogesh Talekar who has been a source of inspiration and a guiding light for me always. He is the reason for whatever I have achieved so far. I would also like to thank my life Maitreyee who has always stood up for me and by me. She sustained me in ways I could never imagine. Thank you for everything.

Table of Contents

Preface

Part 1: Understanding Different QRadar Components and Architecture

1

QRadar Components

Understanding the QRadar Console

Tomcat

Hostcontext

Hostservices

Exploring event data

Event Processor

Event Collector

Exploring flow data

Flow Processor

Flow Collector

Getting to know the Data Node

Other QRadar components

QRadar Network Insights

QRadar Incident Forensics

QRadar Packet Capture

QRadar Vulnerability Manager

QRadar Risk Manager

QRadar App Host

Summary

2

How QRadar Components Fit Together

All-in-one deployment

Distributed deployment

QRadar Incident Forensics (QRIF)

QRadar Risk Manager

Summary

3

Managing QRadar Deployments

Understanding different types of QRadar deployments

QRadar appliances

QRadar installed on virtual machines

QRadar on bare-metal servers

QRadar installation on cloud solutions

QRadar Community Edition

Installing QRadar

Upgrading QRadar deployments

Upgrading the QRadar version

Upgrading QRadar appliance firmware

Scaling QRadar deployments

Scaling by adding data nodes

Scaling by adding processors

Scaling by adding collectors

Scaling by adding CPU and memory on QRadar appliances

Licensing

Summary

Part 2: QRadar Features and Deployment

4

Integrating Logs and Flows in QRadar

Exploring protocols and DSMs

How to transfer data from applications to QRadar

How to parse or make sense of data that is received

Services involved in the integration of an event log

Understanding flows and types of flows

Internal flow sources

External flow sources

Superflows and their types

Getting to know DLC

Summary

5

Leaving No Data Behind

Understanding queues and buffers

Persistent queues

In-memory queues and disk buffers

Getting to know DSM Editor

Summary

6

QRadar Searches

How do searches work?

Services involved in a QRadar search

Different types of QRadar searches

Default searches

Customized searches

Searches using quick filter

Data accumulation

QRadar search tuning

Indexing and index management

The sequence of filters used in a query

Creating a MADV

Summary

7

QRadar Rules and Offenses

Different types of QRadar rules

Understanding the Rule Wizard

Rule name

Rule systems

Rule conditions

Rule actions

Rule responses

Using reference data in rules

Building blocks

What is historical rule correlation?

Offense generation and management

Summary

Part 3: Understanding QRadar Apps, Extensions, and Their Deployment

8

The Insider Threat – Detection and Mitigation

Insider threats – detection and mitigation challenges

What is UBA?

Setting up QRadar UBA

Installing QRadar UBA

Importing users into QRadar UBA

QRadar UBA internals

How does QRadar UBA work?

Understanding the UBA dashboard

Integration with the ML app

UBA application tuning

Understanding the QRadar NTA app

Summary

9

Integrating AI into Threat Management

QRadar Assistant app – a quick overview

QRadar Advisor with Watson

QRAW integration with IBM QRadar SOAR

QRadar Use Case Manager app

Summary

10

Re-Designing User Experience

QRadar Analyst Workflow

Exploring the Search view

Offenses view

QRadar Pulse dashboard app

QRadar Experience Center

Creating your own app

Summary

11

WinCollect – the Agent for Windows

Understanding WinCollect

The types of WinCollect agents

Managed WinCollect agents

Standalone WinCollect agents

Tuning WinCollect

Log source confirguation

XPath query configuration

Summary

12

Troubleshooting QRadar

Exploring log source and flow integration issues

Autoupdate issues

Log source configuration issues

Flow integration issues

Understanding QRadar deployment issues

Investigating QRadar app issues

Exploring QRadar performance issues

QRadar FAQs answered

Query 1

Query 2

Query 3

Query 4

Query 5

Query 6

Query 7

Query 8

Query 9

Query 10

Query 11

A next-generation QRadar sneak peek

Summary

Further reading

Index

Other Books You May Enjoyd

Preface

This book is a complete guide to planning, deploying, and managing a QRadar environment for any security operation center. It focuses on the intricacies of each component in QRadar and how they can be deployed to achieve the desired result. You will find the best practices to implement huge deployments in QRadar. This book describes how QRadar apps should be used as added features as well as how to develop customized QRadar apps.

Who this book is for

This book is for security analysts, system administrators, and security architects, as well as executive management to help them understand the concepts and features of QRadar. The book includes real-world examples that will help incident management teams handle security incidents and plan for cybersecurity attacks.

What this book covers

Chapter 1, QRadar Components, explains all the QRadar components, what the different QRadar services are, and which services run on which components. This chapter will help you understand how QRadar is designed and how different components provide different functionalities.

Chapter 2, How QRadar Components Fit Together, looks at the QRadar console, which is the central component around which other components fit together; depending on the requirement, other QRadar components can be added to the console. Also, we will explain in detail what different types of deployments exist – namely, all-in-one deployment and distributed deployment.

Chapter 3, Managing QRadar Deployments, deals with installing, upgrading, and scaling QRadar as and when required. We also discuss licensing requirements in QRadar.

Chapter 4, Integrating Logs and Flows in QRadar, discusses the practical aspects of ingesting data in QRadar. There are various ways in which different types of events and flow data are ingested, which are described in detail in this chapter.

Chapter 5, Leaving No Data Behind, explores how data is handled by QRadar. The majority of the shortcomings when working with QRadar occur while ingesting data. We will also discuss the DSM Editor, a tool to ingest data that is not supported out of the box.

Chapter 6, QRadar Searches, discusses how searches work and how they can be tuned in QRadar. SIEM is only as efficient as the searches performed on it. We will also discuss the different types of searches in QRadar and how data accumulation works in it.

Chapter 7, QRadar Rules and Offenses, delves into one of the most fundamental aspects of QRadar, which is rules and offenses. We will discuss the different types of rules, how to run rules for historical data called historical correlation, how offenses are generated, and finally, how to fine-tune and manage rules and offenses.

Chapter 8, The Insider Threat – Detection and Mitigation, examines how UBA can be used to detect an insider threat in your organization. IBM has a public portal where apps are published, which can be downloaded and installed on QRadar. Some of these apps are created by IBM, while other vendors have come up with apps for their own applications. IBM UBA is one such app developed by IBM for insider threat management.

Chapter 9, Integrating AI into Threat Management, discusses three QRadar apps – the QRadar Assistant app, QRadar Advisor for Watson, and QRadar Use Case Manager. We will also discuss the practical use of these apps.

Chapter 10, Re-Designing User Experience, explores how to use apps to improve the user experience. IBM QRadar needed an overhaul when it came to user experience. Hence, IBM devised apps such as IBM QRadar Pulse and IBM Analyst Workflow to change the way QRadar can be managed, which we will look at in this chapter.

Chapter 11, WinCollect – the Agent for Windows, focuses on how to install, manage, upgrade, and fine-tune Wincollect agents, one of many in-built features from IBM QRadar. Wincollect is an agent for the Windows operating system and collects events from Windows machines. It can also poll events from other Windows machines where it is not installed and send them to QRadar.

Chapter 12, Troubleshooting QRadar, examines the pain points and solutions to many of the issues in QRadar, based on years of experience working with it. There are tips and tricks as well as a list of frequently asked questions about QRadar. This chapter should help you become a pro user of QRadar.

To get the most out of this book

Software/hardware covered in the book

Operating system requirements

QRadar components and services

Windows, macOS, or Linux – for browsing and logging in to a command-line interface

Wincollect

RHEL – for QRadar

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the color images

We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://packt.link/PtEjQ.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “The log source name as defined here would be [email protected], where cali_ips could be the device type and the source address could be the source IP picked up from the event payload.”

A block of code is set as follows:

html, body, #map { height: 100%; margin: 0; padding: 0 }

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

[default] exten => s,1,Dial(Zap/1|30) exten => s,2,Voicemail(u100) exten => s,102,Voicemail(b100) exten => i,1,Voicemail(s0)

Any command-line input or output is written as follows:

$ mkdir css $ cd css

Bold: Indicates a new term, an important word, or words that you see on screen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “For this feature, you need to enable it in the Configuration tab of the DSM Editor.”

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share your thoughts

Once you’ve read Building a Next-Gen SOC with IBM QRadar, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781801076029

Submit your proof of purchaseThat’s it! We’ll send your free PDF and other benefits to your email directly

Part 1: Understanding Different QRadar Components and Architecture

In this part, we work on the fundamentals of QRadar by discussing different types of components and how they fit together. We also consider the different types of deployments and how to manage, scale, and upgrade them.

This part has the following chapters:

Chapter 1, QRadar ComponentsChapter 2, How QRadar Components Fit TogetherChapter 3, Managing QRadar Deployments

1

QRadar Components

We live in a digital age in which the paradigms of security have changed. In the past, wars were fought on battlefields. Now, digital space is where the security of a nation-state, an enterprise, or an individual is threatened. Gartner predicts that by 2025, cyber attackers will use weaponized technology to harm or kill humans. Earlier, cyberattacks were restricted to things such as denial of services, information theft, and ransomware.

These cyberattacks have a heavy financial toll (billions of dollars), cause disruption in production, cause intellectual property to be stolen, and eventually, the brand reputation is tarnished. This is a never-ending battle in this digital age. Security vendors have come up with hundreds of security products and solutions to counter these cyberattacks. IBM has been at the forefront and is leading the security space with top-of-the-line products and solutions.

To understand the impact of a cyberattack, we just have to look a few years back at what happened with Ashley Madison. Ashley Madison was a dating app for those who were married, and the slogan they used to advertise then was “Life is short. Have an affair.” Not surprisingly, the service had 37 million subscribers.

And then the unthinkable happened for the subscribers of the site. Ashley Madison used the weakest password encryption algorithm, and it was easily hacked. A hacker group called the Impact Group gave Ashley Madison 30 days to pay a ransom. As Ashley Madison did not pay, on the 30th day, they released about 60 GB of data with the names, email addresses, credit card numbers, and other details of the subscribers on the dark net. Soon, the media and the crooks started looking for famous personalities to hold them for ransom. The hack soon became public knowledge, leading to a large number of breakups, divorces, and even suicides. The financial implications of such breaches are unaccountable. The site and the brand of Ashley Madison were damaged permanently.

The point that needs to be understood from this scenario is that security breaches can cost lives, and hence any organization (whether it be a dating website, a bank, or a telecom company) needs to be on top of its game when it comes to security.

IBM QRadar is a solution suite that provides enhanced threat intelligence and insights into cyberattacks. These insights help organizations automate responses to threats and also help in devising new strategies to counter cyberattacks. An organization uses hundreds of enterprise solutions and security products from different vendors, such as firewalls, Endpoint Detection Response (EDR), Intrusion Prevention System (IPS), Data Loss Prevention (DLP), and so on. IBM QRadar seamlessly integrates with all these products, consumes all the security data from them, and provides security alerts or insights that are actionable.

In this book, we will learn more about how to build your next-generation Security Operations Center (SOC) using the IBM QRadar solution suite. To understand IBM QRadar and how it functions, it is important to understand the different components. We call all these different QRadar components managed hosts (apart from the Console).

In this chapter, we will discuss various QRadar services for each component, which should be a good starting point to design the architecture for your SOC. As per different requirements, different components can be used in the deployment. Various aspects such as deployment types, scaling, upgrades, and licensing are discussed in corresponding chapters. In this chapter, however, we’re going to cover the following main topics:

Understanding the QRadar ConsoleExploring event dataExploring flow dataGetting to know the Data NodeInvestigating QRadar components

Understanding the QRadar Console

The Console is the brain of QRadar and is the single indispensable component of QRadar. It can collect and process data and throw alerts based on the rules. This is the primary job of the Console. Other components (described later) are mostly used to scale these functionalities in one form or another. Now, let us look at the three major services running on the Console and understand them.

Tomcat

The primary utility of this service is for displaying the User Interface (UI) of QRadar. The QRadar UI can be accessed by typing the IP address or the hostname (if it can be resolved) in the browser.

If the tomcat service is down on QRadar, you will not be able to load the QRadar UI. It maintains the user sessions, active sessions, and current users – all those who have logged in to QRadar UI. It also plays a part in authenticating users, whether it is local authentication, Lightweight Directory Access Protocol (LDAP) authentication, or any other type of authentication. It is a multithreaded service that also deals with the export of data from the QRadar UI. The status of the tomcat service can be checked using a simple command:

systemctl status tomcat

We will cover troubleshooting tricks and tips for tomcat in the final chapter.

Important note

The Tomcat service is only available on the QRadar Console.

Hostcontext

When the hostcontext service is started, it triggers many other services with it. All the functionalities of QRadar are dependent on the hostcontext service. This service is part of all the QRadar managed hosts, unlike the tomcat service. The hostcontext service is responsible for replicating the deployment changes from the Console to other managed hosts.

The following is the list of services triggered because of hostcontext:

ecs-ec-ingress: The specialty of this service from the following services is that even if the hostcontext service is stopped from the command line or if the hostcontext service crashes, ecs-ec-ingress keeps running and collecting events from Log Sources. If the ecs-ec-ingress service is stopped, there are two ways of starting:Restarting the hostcontext serviceStarting the ecs-ec-ingressservice separatelyecs-ec: The primary function of this service is to parse (map) the incoming events/flows. This service converts the events into a form that QRadar understands. The event is mapped to its event name by this service. For example, Linux OS has sent an authentication event to QRadar that there is an invalid user named testdev trying to log in via SSH.

The payload of such an event would look like this:

"Apr  10 18:26:40 servername sshd[26388]: input_userauth_request: invalid user testdev "

QRadar needs to make sense of this payload. This is called parsing. Then, QRadar needs to map this event to an appropriate event name, which is called event mapping. The event will be parsed as follows:

Time: 6:26:40 p.m.Date: 10th AprilEvent Name: Authentication FailureServer Details: servername

So, the two important functions of ecs-ec are as follows:

Event parsingEvent mappingecs-ep: Once the events are parsed and mapped, they need to be processed. Rules are provided as a part of the initial installation. These rules can further be customized as per the security use cases of the organization. ecs-ep is responsible for matching each incoming event against all the enabled rules. If the rule conditions are fulfilled based on the incoming event/events, offenses (security alerts) can be triggered based on the rule action and rule response (defined).

For example, we could have a rule to trigger an offense if we receive an event called Authentication Failure from Linux OS after 6 p.m. In such a case, looking at the previous event in the example, an offense will be generated.

ecs-ep is also responsible for offense management in terms of the following:

Offense creationRenaming offensesAttaching events to triggered offensesOffenses in a dormant conditionOffense closureOffense deletionQFlow: This service is responsible for collecting flows in QRadar. Flows are network packet information collected in a specific format over a period of time.Accumulator: This service is responsible for creating global views in QRadar, which can be used for dashboards, reports, and so on.Ariel proxy: This service is responsible for relaying the search queries to the appropriate managed hosts.Ariel query: This service queries ariel databases across all managed hosts based on the query run on the Console.

There are many other services, and we will discuss them in detail when introducing the concepts related to them.

Hostservices

This service is also part of the Console and other managed hosts. Typically, QRadar has two types of databases:

Ariel, which is not managed by hostservices (it stores event/flow data)Postgres, which is managed by hostservices (it stores configuration)

An ariel database has security data that is collected, and processed by hostcontext. Postgres has the configuration details of QRadar. This is managed by the hostservices service. Postgres is a relational database management system (RDBMS) that has multiple tables containing information on QRadar deployment, configuration, and settings. The Postgres database is replicated on different managed hosts using the hostcontext service. The Postgres database can be queried, if required, using the psql command line. We will discuss this in detail when we talk about optimization and tuning QRadar.

The brain behind QRadar is the Console, and the other components act as auxiliary parts of the system helping the Console perform the functions in a better way. Before jumping on to other QRadar components, let's first discuss and understand event data and flow data.

Exploring event data

Every organization that plans on building a SOC has hundreds and thousands of applications, servers, and endpoints that it would like to monitor. Each of these applications or servers has security and audit logs. These security and audit logs are what we call event data. QRadar supports multiple protocols such as Syslog, JDBC, and UDP multiline. It also supports product-specific protocols such as the Akamai Kona REST API protocol and the CISCO NSEL protocol. Using these protocols, QRadar can either pull data from Log Sources or we can configure Log Sources to send data to QRadar.

The data sent is in the form of events that are parsed and mapped to certain event names. The events can be viewed from the QRadar UI from the Log Activity tab. There are multiple query options that can be used.

The following figure shows a screenshot of the Log Activity tab with filters applied:

Figure 1.1 – Log Activity tab

Here, we refer to Data Source as Log Source, and they can beused interchangeably.

Event Processor

We previously learned that the Console is the brain of QRadar. However, there is a limit to the amount of data that can be collected and processed by the Console. This is where the Event Processor comes into the picture. The Console can delegate the work of collecting and processing event data to the Event Processor.

The main services running on the Event Processor are as follows:

hostcontext: There are multiple services that are triggered when the hostcontext service is started. On each managed host (Event Processor), there is a configuration file named /opt/qradar/conf/nva.hostcontext.conf. It has a parameter called 'COMPONENT_PROCESSES'. Based on the values, the services are started when hostcontext is started. The main services that are part of the hostcontext service on the Event Processor are as follows:ecs-ec-ingressecs-ececs-epAriel query serverhostservices: This is like what we have seen on the Console.

You will see that there is no ariel proxy service on the Event Processor. This is because the ariel proxy service is only on the Console. When we search for data on the Console, the ariel proxy service sends the query request to the Event Processor. This request is accepted and worked on by the ariel query server service.

The ariel query server queries the ariel database, which is on the Event Processor, and sends the resultant data back to the Console, where it is shown in Log Activity.

Important note

We will be using the term managed host for any QRadar component that can be managed from the Console – for example, Event Processor, Flow Processor, Event Collector, and so on. There are very few QRadar components that are not managed by the Console. We will discuss them later.

An Event Processor can collect data using ecs-ec-ingress, parse the data using ecs-ec, and process the data using ecs-ep. You will notice that there is an ariel database in each Event Processor, which means that the event data is stored locally. Only when the data is searched is the resultant data sent to the Console for display. Over a period, the resultant data on the Console is removed as per configured policies

As the collection, parsing, and processing of data are done