Ghidra Software Reverse Engineering for Beginners E-Book

A. P. David

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 262

Veröffentlichungsjahr: 2021

Beschreibung

Ghidra, an open source software reverse engineering (SRE) framework created by the NSA research directorate, enables users to analyze compiled code on any platform, whether Linux, Windows, or macOS. This book is a starting point for developers interested in leveraging Ghidra to create patches and extend tool capabilities to meet their cybersecurity needs.
You'll begin by installing Ghidra and exploring its features, and gradually learn how to automate reverse engineering tasks using Ghidra plug-ins. You’ll then see how to set up an environment to perform malware analysis using Ghidra and how to use it in the headless mode. As you progress, you’ll use Ghidra scripting to automate the task of identifying vulnerabilities in executable binaries. The book also covers advanced topics such as developing Ghidra plug-ins, developing your own GUI, incorporating new process architectures if needed, and contributing to the Ghidra project.
By the end of this Ghidra book, you’ll have developed the skills you need to harness the power of Ghidra for analyzing and avoiding potential vulnerabilities in code and networks.

Leseprobe

Ghidra Software Reverse Engineering for Beginners

Analyze, identify, and avoid malicious code and potential threats in your networks and systems

A. P. David

BIRMINGHAM—MUMBAI

Ghidra Software Reverse Engineering for Beginners

Copyright © 2020 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Commissioning Editor: Vijin Boricha

Acquisition Editor: Meeta Rajani

Senior Editor: Arun Nadar

Content Development Editor: Romy Dias

Technical Editor: Aurobindo Kar

Copy Editor: Safis Editing

Project Coordinator: Neil Dmello

Proofreader: Safis Editing

Indexer: Priyanka Dhadke

Production Designer: Shankar Kalbhor

First published: December 2020

Production reference: 1101220

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80020-797-4

www.packt.com

To my son, Santiago. I love you, Santi! This book is dedicated only to you.

– A. P. David

Packt.com

Subscribe to our online digital library for full access to over 7,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe?

Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionalsImprove your learning with Skill Plans built especially for youGet a free eBook or video every monthFully searchable for easy access to vital informationCopy and paste, print, and bookmark content

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.

At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

Contributors

About the author

A. P. David is a senior malware analyst and reverse engineer. He has more than 7 years of experience in IT, having worked on his own antivirus product, and later as a malware analyst and reverse engineer. He started working for a company mostly reverse engineering banking malware and helping to automate the process. After that, he joined the critical malware department of an antivirus company. He is currently working as a security researcher at the Galician Research and Development Center in Advanced Telecommunications(GRADIANT) while doing a malware-related PhD. Apart from that, he has also hunted vulnerabilities for some relevant companies in his free time, including Microsoft's Windows 10 and National Security Agency's Ghidra project.

I want to thank my son, Santiago, for being with me and giving the support I've needed to write this book even while the COVID-19 global pandemic was raging around us. Thanks to my family for the help, but special thanks to my parents: Feliciano and María José. The whole Packt editing team has helped this author immensely, but I'd like to give special thanks to Romy Dias, who edited most of my work, and Vaidehi Sawant for the great project management.

About the reviewer

Elad Shapira is head of research at Panorays, where he specializes in mimicking hackers' behavior by exploring new hacking techniques and vectors. Prior to Panorays, Elad served as the Mobile Security Research Team leader at AVG Technologies. Elad is a recognized speaker, having presented at various hacking conferences including Recon and BlueHat. He teaches at Afeka Academic College of Engineering and assists in directing local hacking competitions. Elad is also interested in hardware hacking, low-level development, playing Capture the Flag, and making and breaking things.

I would like to thank my dad, a man who could face whatever challenges life threw at him, for all his love, dedication, and endless support. Dad, you are my man. I love and admire you from the core of my heart. I am proud to be your son.

Packt is searching for authors like you

If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Table of Contents

Preface

Section 1: Introduction to Ghidra

Chapter 1: Getting Started with Ghidra

Technical requirements

WikiLeaks Vault 7

NSA release

Ghidra versus IDA and many other competitors

Ghidra overview

Installing Ghidra

Overview of Ghidra's features

Summary

Questions

Chapter 2: Automating RE Tasks with Ghidra Scripts

Technical requirements

Using and adapting existing scripts

The script class

Script development

Summary

Questions

Chapter 3: Ghidra Debug Mode

Technical requirements

Setting up the Ghidra development environment

Overviewing the software requirements

Installing the Java JDK

Installing the Eclipse IDE

Installing PyDev

Installing GhidraDev

Debugging the Ghidra code and Ghidra scripts

Debugging Ghidra scripts from Eclipse

Debugging any Ghidra component from Eclipse

Ghidra RCE vulnerability

Explaining the Ghidra RCE vulnerability

Exploiting the Ghidra RCE vulnerability

Fixing the Ghidra RCE vulnerability

Looking for vulnerable computers

Summary

Questions

Further reading

Chapter 4: Using Ghidra Extensions

Technical requirements

Installing existing Ghidra extensions

Analyzing the code of the Sample Table Provider plugin

Understanding the Ghidra extension skeleton

Analyzers

Filesystems

Plugins

Exporters

Loaders

Developing a Ghidra extension

Summary

Questions

Further reading

Section 2: Reverse Engineering

Chapter 5: Reversing Malware Using Ghidra

Technical requirements

Setting up the environment

Looking for malware indicators

Looking for strings

Intelligence information and external sources

Checking import functions

Dissecting interesting malware sample parts

The entry point function

Analyzing the 0x00453340 function

Analyzing the 0x00453C10 function

Analyzing the 0x0046EA60 function

Analyzing the 0x0046BEB0 function

Analyzing the 0x0046E3A0 function

By analyzing this function, we notice that the pipe is used for some kind of synchronization. The CreateThread API function receives as parameters the function to execute as a thread and an argument to pass to the function; so, when a thread creation appears, we have to analyze a new function – in this case, lpStartAddress_00449049:

Analyzing the 0x004559B0 function

Analyzing the 0x004554E0 function

Analyzing the 0x0046C860 function

Analyzing the 0x0046A100 function

Summary

Questions

Further reading

Chapter 6: Scripting Malware Analysis

Technical requirements

Using the Ghidra scripting API

Writing scripts using the Java programming language

Writing scripts using the Python programming language

Deobfuscating malware samples using scripts

The delta offset

Translating API hashes to addresses

Deobfuscating the hash table using Ghidra scripting

Improving the scripting results

Summary

Questions

Further reading

Chapter 7: Using Ghidra Headless Analyzer

Technical requirements

Why use headless mode?

Creating and populating projects

Performing analysis on imported or existing binaries

Running non-GUI scripts in a project

Summary

Questions

Further reading

Chapter 8: Auditing Program Binaries

Technical requirements

Understanding memory corruption vulnerabilities

Understanding the stack

Stack-based buffer overflow

Understanding the heap

Heap-based buffer overflow

Format strings

Finding vulnerabilities using Ghidra

Exploiting a simple stack-based buffer overflow

Summary

Questions

Further reading

Chapter 9: Scripting Binary Audits

Technical requirements

Looking for vulnerable functions

Retrieving unsafe C/C++ functions from the symbols table

Decompiling the program using scripting

Looking for sscanf callers

Enumerating caller functions

Analyzing the caller function using PCode

PCode versus assembly language

Retrieving PCode and analyzing it

Using the same PCode-based script in multiple architectures

Summary

Questions

Further reading

Section 3: Extending Ghidra

Chapter 10: Developing Ghidra Plugins

Technical requirements

Overview of existing plugins

Plugins included with the Ghidra distribution

Third-party plugins

The Ghidra plugin skeleton

The plugin documentation

Writing the plugin code

The provider for a plugin

Developing a Ghidra plugin

Documenting the plugin

Implementing the plugin class

Implementing the provider

Summary

Questions

Further reading

Chapter 11: Incorporating New Binary Formats

Technical requirements

Understanding the difference between raw binaries and formatted binaries

Understanding raw binaries

Understanding formatted binaries

Developing a Ghidra loader

The old-style DOS executable (MZ) parser

The old-style DOS executable (MZ) loader

Understanding filesystem loaders

FileSystem Resource Locator

Summary

Questions

Further reading

Chapter 12: Analyzing Processor Modules

Technical requirements

Understanding the existing Ghidra processor modules

Overviewing the Ghidra processor module skeleton

Setting up the processor module development environment

Creating a processor module skeleton

Developing Ghidra processors

Documenting processors

Identifying functions and code using patterns

Specifying the language and its variants

Summary

Questions

Further reading

Chapter 13: Contributing to the Ghidra Community

Technical requirements

Overviewing the Ghidra project

The Ghidra community

Exploring contributions

Understanding legal aspects

Submitting a bug report

Suggesting new features

Submitting questions

Submitting a pull request to the Ghidra project

Summary

Questions

Further reading

Chapter 14: Extending Ghidra for Advanced Reverse Engineering

Technical requirements

Learning the basics of advanced reverse engineering

Learning about symbolic execution

Learning about SMT solvers

Learning about concolic execution

Using Ghidra for advanced reverse engineering

Adding symbolic execution capabilities to Ghidra with AngryGhidra

Converting from PCode into LLVM with pcode-to-llvm

Summary

Questions

Further reading

Assessments

Other Books You May Enjoy

Section 1: Introduction to Ghidra

This section aims to introduce you to Ghidra and its history, the project structure, extension development, scripts, and, as it is open source, how to contribute.

This section contains the following chapters:

Chapter 1, Getting Started with GhidraChapter 2, Automating RE Tasks with Ghidra ScriptsChapter 3, Ghidra Debug ModeChapter 4, Using Ghidra Extensions

Chapter 1: Getting Started with Ghidra

In this introductory chapter, we will provide an overview of Ghidra in some respects. Before starting, it would be convenient to know how to acquire and install the program. This is obviously something simple and trivial if you want to install a release version of the program. But I guess you probably want to know this program in depth. In which case, I can tell you in advance that it is possible to compile the program by yourself from the source code.

Since the source code of Ghidra is available and ready to be modified and extended, you will probably also be interested in knowing how it is structured, what kind of pieces of code exist, and so on. This is a great opportunity to discover the enormous possibilities that Ghidra offers us.

It is also interesting to review the main functionalities of Ghidra from the point of view of a reverse engineer. This will arouse your interest in this tool since it has its own peculiarities, and this is precisely the most interesting thing about Ghidra.

In this chapter, we're going to cover the following main topics:

WikiLeaks Vault 7Ghidra versus IDA and many other competitorsGhidra overview

Technical requirements

The GitHub repository containing all the necessary code for this chapter can be found at the following link:

https://github.com/PacktPublishing/Ghidra-Software-Reverse-Engineering-for-Beginners

Check out the following link to see the Code in Action video: https://bit.ly/3qD1Atm

WikiLeaks Vault 7

On March 7, 2017, WikiLeaks started to leak Vault 7, which became the biggest leak of confidential documents on the US Central Intelligence Agency (CIA). This leak included secret cyber-weapons and spying techniques divided into 24 parts, named Year Zero, Dark Matter, Marble, Grasshopper, HIVE, Weeping Angel, Scribbles, Archimedes, AfterMidnight and Assassin, Athena, Pandemic, Cherry Blossom, Brutal Kangaroo, Elsa, OutlawCountry, BothanSpy, Highrise, UCL/Raytheon, Imperial, Dumbo, CouchPotato, ExpressLane, Angelfire, and Protego.

While Michael Vincent Hayden, the director of the CIA between 2006 and 2009 and director of the NSA between 1999 and 2005, as the spokesperson, did not confirm or deny the authenticity of this enormous leak, some NSA intelligence officials anonymously did leak the material.

The existence of Ghidra was leaked in the first part of Vault 7: Year Zero. This first part consists of a huge leak of documents and files stolen from the CIA's Center for Cyber Intelligence in Langley, Virginia. The leak's content is about the CIA's malware arsenal, zero-day weaponized exploits, and how Apple's iPhone, Google's Android, devices Microsoft's Windows devices, and even Samsung TVs are turned into covert microphones.

Ghidra was referenced three times in this leak (https://wikileaks.org/ciav7p1/cms/index.html), showing things such as how to install it, a step-by-step tutorial (with screenshots) of how to perform a manual analysis of a 64-bit kernel cache by using Ghidra, and the latest Ghidra version available at the time, which was Ghidra 7.0.2.

NSA release

As announced during RSA Conference 2019 in San Francisco, Rob Joyce, senior advisor for cybersecurity at NSA, explained the unique capabilities and features of Ghidra during a session called Get your free NSA reverse engineering tool, and Ghidra program binaries were also published.

During this session, some features were explained:

Team collaboration on a single project featureThe capabilities to extend and scale GhidraThe generic processor model, also known as SLEIGHThe two working modes: interactive and non-GUIThe powerful analysis features of Ghidra

Finally, on April 4, 2019, the NSA released the source code of Ghidra on GitHub (https://github.com/NationalSecurityAgency/ghidra), as well as on the Ghidra website, where you can download Ghidra release versions that are ready to use: https://ghidra-sre.org. The first version of Ghidra that was available on this website was Ghidra 9.0. Ghidra's website is probably not available to visitors outside the US; if this is the case, you can access it by using a VPN or an online proxy such as HideMyAss (https://www.hidemyass.com/).

Unfortunately for the NSA, a few hours later, the first Ghidra vulnerability was published by Matthew Hickey, also known as @hackerfantastic, at 1:20 AM, March 6, 2019. He said the following via Twitter:

Ghidra opens up JDWP in debug mode listening on port 18001, you can use it to execute code remotely (Man facepalming). to fix change line 150 of support/launch.sh from * to 127.0.0.1 https://github.com/hackerhouse-opensource/exploits/blob/master/jdwp-exploit.txt.

Then, a lot of suspicions about the NSA and Ghidra arose. However, taking into account the cyber-espionage capabilities of the NSA, do you think the NSA needs to include a backdoor in its own software in order to hack its users?

Obviously, no. They don't need to do this because they already have cyber-weapons for that.

You can feel comfortable when using Ghidra; probably, the NSA only wanted to do something honorable to improve its own image and, since Ghidra's existence was leaked by WikiLeaks, what better way to do that than to publish it at RSA Conference and release it as open source?

Ghidra versus IDA and many other competitors

Even if you have already mastered a powerful reverse engineering framework, such as IDA, Binary Ninja, or Radare2, there are good reasons to start learning Ghidra.

No single reverse engineering framework is the ultimate one. Each reverse engineering framework has its own strengths and weaknesses. Some of them are even incomparable to each other because they were conceived with different philosophies (for instance, GUI-based frameworks versus command line-based frameworks).

On the other hand, you will see how those products are competing with and learning from each other all the time. For instance, IDA Pro 7.3 incorporated the undo feature, which was previously made available by its competitor, Ghidra.

In the following screenshot, you can see the epic and full-of-humor @GHIDRA_RE official Twitter account's response to IDA Pro's undo feature:

Figure 1.1 – IDA Pro 7.3 added an undo feature to compete with Ghidra

Differences between frameworks are susceptible to change due to the competition, but we can mention some current strengths of Ghidra:

It is open source and free (including its decompiler).It supports a lot of architectures (which maybe the framework you are using does not support yet).It can load multiple binaries at the same time in a project. This feature allows you to easily apply operations over many related binaries (for example, an executable binary and its libraries).It allows collaborative reverse engineering by design.It supports big firmware images (1 GB+) without problems.It has awesome documentation that includes examples and courses.It allows version tracking of binaries, allowing you to match functions and data and their markup between different versions of the binary.

In conclusion, it is recommended to learn as many frameworks as possible to know and take advantage of each one. In this sense, Ghidra is a powerful framework that you must know.

Ghidra overview

In a similar way as happened at RSA Conference, we will provide a Ghidra overview in order to present the tool and its capabilities. You will soon realize how powerful Ghidra is and why this tool is not simply another open source reverse engineering framework.

At the time of writing this book, the latest available version of Ghidra is 9.1.2, which can be downloaded from the official website mentioned in the previous section of this chapter.

Installing Ghidra

It is recommended to download the latest version of Ghidra (https://ghidra-sre.org/) by clicking on the red Download Ghidra v9.1.2 button, but if you want to download older versions, then you need to click on Releases:

Figure 1.2 – Downloading Ghidra from the official website

After downloading the Ghidra archive file (ghidra_9.1.2_PUBLIC_20200212.zip) and decompressing it, you will see the following file structure:

Figure 1.3 – The Ghidra 9.1.2 structure after it is decompressed

The content can be described as follows (source: https://ghidra-sre.org/InstallationGuide.html):

docs: Ghidra documentation and some extremely useful resources, such as learning Ghidra courses for all levels, cheatsheets, and a step-by-step installation guideExtensions: Optional Ghidra extensions allowing you to improve its functionality and integrate it with other toolsGhidra: The Ghidra program itselfGPL: Standalone GPL support programslicenses: Contains licenses used by Ghidraserver: Contains files related to Ghidra Server installation and administrationsupport: Allows you to run Ghidra in advanced modes and control how it launches, including launching it to be debuggedghidraRun: The script used to launch Ghidra on Linux and iOSghidraRun.bat: Batch script allowing you to launch Ghidra on WindowsLICENSE: Ghidra license file

In addition to downloading a release version of Ghidra (which is precompiled), you can compile the program on your own, as will be explained in the next section.

Compiling Ghidra on your own

If you want to compile Ghidra on your own, then you can download the source code from the following URL: https://github.com/NationalSecurityAgency/ghidra.

You can then build it using Gradle by running the following command:

gradle --init-script gradle/support/fetchDependencies.gradle init

gradle buildGhidra

gradle eclipse

gradle buildNatives_win64

gradle buildNatives_linux64

gradle buildNatives_osx64

gradle sleighCompile

gradle eclipse -PeclipsePDE

gradle prepDev

This will produce a compressed file containing the compiled version of Ghidra:

/ghidra/build/dist/ghidra_*.zip

Before starting Ghidra, make sure your computer meets the following requirements:

4 GB RAM1 GB storage (for installing Ghidra binaries)Dual monitors strongly recommended

Since Ghidra is written in Java, if it is executed before installing the Java 11 64-bit runtime and development kit, some of the following error messages could be displayed:

When Java is not installed, you will see the following:

"Java runtime not found..."

When the Java Development Kit (JDK) is missing, you will see the following:

Figure 1.4 – Missing JDK error

Therefore, if you get any of those messages, please download the JDK from one of the following sources:

https://adoptopenjdk.net/releases.html?variant=openjdk11&jvmVariant=hotspothttps://docs.aws.amazon.com/corretto/latest/corretto-11-ug/downloads-list.html

How to solve installation issues

Ghidra's step-by-step installation guide, including known issues, can be found in Ghidra's documentation directory at docs\InstallationGuide.html.

It is also available online at the following link: https://ghidra-sre.org/InstallationGuide.html.

Note that you can report new issues you find in Ghidra through the following link: https://github.com/NationalSecurityAgency/ghidra/issues.

After installing Ghidra, you will be able to launch it using ghidraRun on Linux and iOS and ghidraRun.bat on Windows.

Overview of Ghidra's features

In this section, we will look at an overview of some fundamental Ghidra features in order to understand the overall functionality of the program. It is also a good starting point to get familiar with it.

Creating a new Ghidra project

As you will notice, differently than other reverse engineering tools, Ghidra doesn't work with files directly. Instead, Ghidra works with projects. Let's create a new project by clicking on File | New Project…. You can also do this faster by pressing the Ctrl + N hotkey (the complete list of Ghidra hotkeys is available at https://ghidra-sre.org/CheatSheet.html and also in Ghidra's documentation directory):

Figure 1.5 – Creating a new Ghidra project

Furthermore, projects can be non-shared or shared projects. Since we want to analyze a hello world program without collaboration with other reverse engineers, we will choose Non-Shared Project, and then click on the Next>> button. Then, the program asks us to choose a project name (hello world) and where to store it:

Figure 1.6 – Choosing a project name and directory

The project is composed of a hello world.gpr file and a hello world.rep folder:

Figure 1.7 – Ghidra project structure

A Ghidra project (the *.gpr file) can only be opened by a single user. Therefore, if you try to open the same project twice at the same time, the concurrency lock implemented using the hello world.lock and hello world.lock~ files will prevent you from doing so, as shown in the following screenshot:

Figure 1.8 – Ghidra's project locked

In the next section, we will cover how to add binary files to our project.

Importing files to a Ghidra project

We can start to add files to our hello world project. In order to analyze an extremely simple application with Ghidra, we will compile the following hello world program (hello_world.c) written in the C programming language:

#include <stdio.h>

int main(){

printf("Hello world.");

}

We use the following command to compile it:

C:\Users\virusito\Desktop\hello_world> gcc.exe hello_world.c

C:\Users\virusito\>\

Let's analyze the resulting Microsoft Windows Portable Executable file: hello_world.exe.

Let's import our hello world.exe file to the project; to do that, we have to go to File | Import file. Alternatively, we can press the I key:

Figure 1.9 – Importing a file to the Ghidra project

Ghidra automatically identified the hello_world.exe program as an x86 Portable Executable binary for 32-bit architectures. As it was successfully recognized, we can click OK to continue. After importing it, you will see a summary of the file:

Figure 1.10 – Ghidra project file import result summary

By double-clicking the hello_world.exe file or clicking on the green Ghidra icon of Tool Chest, the file will be opened and loaded by Ghidra:

Figure 1.11 – A Ghidra project containing a Portable Executable file

After importing files into your project, you can start to reverse engineer them. This is a cool feature of Ghidra, allowing you to import more than one file into a single project, because you can apply some operation (for example, search) over multiple files (for example, an executable binary and its dependencies). In the next section, we will see how to analyze those files using Ghidra.

Performing and configuring Ghidra analysis

You will be asked whether to analyze the file, and you probably want to answer Yes to this because the analysis operation recognizes functions, parameters, strings, and more. Usually, you will want to let Ghidra get this information for you. A lot of analysis configuration options do exist. You can see a description of every option by clicking on it; the description is displayed in the upper-right Description section:

Figure 1.12 – File analysis options

Let's click on Analyze to perform the analysis of the file. Then, you will see the Ghidra CodeBrowser window. Don't worry if you forget to analyze something; you can reanalyze the program later (go to the Analysis tab and then Auto Analyze 'hello_world.exe'…).

Exploring Ghidra CodeBrowser

Ghidra CodeBrowser has, by default, a pretty well-chosen distribution of dock windows, as shown in the following screenshot: