Applied Deep Learning on Graphs E-Book

Lakshya Khandelwal

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 339

Veröffentlichungsjahr: 2024

Beschreibung

With their combined expertise spanning cutting-edge AI product development at industry giants such as Walmart, Adobe, Samsung, and Arista Networks, Lakshya and Subhajoy provide real-world insights into the transformative world of graph neural networks (GNNs).
This book demystifies GNNs, guiding you from foundational concepts to advanced techniques and real-world applications. You’ll see how graph data structures power today’s interconnected world, why specialized deep learning approaches are essential, and how to address challenges with existing methods. You’ll start by dissecting early graph representation techniques such as DeepWalk and node2vec. From there, the book takes you through popular GNN architectures, covering graph convolutional and attention networks, autoencoder models, LLMs, and technologies such as retrieval augmented generation on graph data. With a strong theoretical grounding, you’ll seamlessly navigate practical implementations, mastering the critical topics of scalability, interpretability, and application domains such as NLP, recommendations, and computer vision.
By the end of this book, you’ll have mastered the underlying ideas and practical coding skills needed to innovate beyond current methods and gained strategic insights into the future of GNN technologies.

Leseprobe

Applied Deep Learning on Graphs

Leverage graph data for business applications using specialized deep learning architectures

Lakshya Khandelwal

Subhajoy Das

Applied Deep Learning on Graphs

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Niranjan Naikwadi

Publishing Product Manager: Yasir Ali Khan

Book Project Manager: Aparna Nair

Senior Editor: Paridhi Agarwal

Technical Editor: Seemanjay Ameriya

Copy Editor: Safis Editing

Proofreader: Paridhi Agarwal

Indexer: Manju Arasan

Production Designers: Jyoti Kadam and Alishon Falcon

DevRel Marketing Coordinator: Vinishka Kalra

First published: December 2024

Production reference: 1121224

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK

ISBN 978-1-83588-596-3

www.packtpub.com

To my beloved parents, Ravi and Preeti, whose sacrifices helped me become who I am today. My deepest gratitude to my wife, Deblina, whose unwavering support and patience made those countless late nights of writing possible. Special thanks to my brother, Rahul, whose guidance has been my compass through the years, and Arushi, my sister-in-law, along with my precious eight-month-old niece, Nehal, whose smiles and laughter brought joy during this challenging journey. Your collective love, support, and encouragement made this book a reality.

– Lakshya Khandelwal

Contributors

About the authors

Lakshya Khandelwal holds a bachelor’s and master’s degree from IIT Kanpur in mathematics and computer science and has 8+ years of experience in building scalable machine learning products for multiple tech giants. He has worked as a lead ML engineer with Samsung, building natural language intelligence for the very first version of Bixby. He has also worked as a data scientist with Adobe, developing search bid optimization solutions as part of the advertising cloud suite for major enterprises across the globe. In addition, he has led natural language and forecasting initiatives at Walmart, building next-generation AI products for millions of customers. Lakshya currently leads AI for AirMDR, building agentic AI for the cybersecurity domain.

Subhajoy Das is a staff data scientist with 7 years of experience under his belt. He graduated from IIT Kharagpur with a bachelor’s and master’s degree in mathematics and computing. Since then, he has worked in organizations at varying stages of growth: from fast-growing e-commerce start-ups such as Meesho to behemoths such as Adobe. He has driven several pivotal features in every company he has worked in, including building an end-to-end recommendation system for the Meesho app and curating interesting advertising using reinforcement learning-based optimizations in Adobe Advertising. He is currently working at Arista Networks, building AI-driven apps that are responsible for the cybersecurity of several Fortune 500 companies.

About the reviewers

Sumit Dahiya is a seasoned cybersecurity specialist and solution architect with a focus on cloud security, identity and access management, and digital transformation. With more than 18 years of experience, he spearheads extensive security projects and digital transformation initiatives and is renowned for developing industry-leading solutions and leading multinational teams. He has experience with safe system architecture, microservices, and open-source technologies. Sumit mentors people in the fields of architecture and cybersecurity and has contributed to several papers and conferences. He wants to express his gratitude to his mentors, family, and friends for their constant encouragement and support along his journey.

Humashankar Vellathur Jaganathan is the principal engineering manager at CGI and a BCS Fellow. He is an esteemed mentor and key strategic adviser for Hubspot and Lucid Software; as a leader, he possesses a unique ability to strategize and think on his feet. His publications include “Mortgage-based securities data hybrid encryption for financial data analysis” in the International Journal of Electronic Security and Digital Forensics.

I would like to extend special thanks to my mentor, Prakash Murugesan, a distinguished engineer at Verizon, and Imran Ur Rehman, a senior project manager at Capgemini, for their expert guidance and valuable input.

Ashish Kumar is an AI and data science innovator with over 8 years of experience, specializing in scalable, real-time AI solutions. He holds an integrated MTech in mathematics and computing from IIT Delhi (2016). Ashish’s groundbreaking work includes the development of a bidding algorithm for low-impression keywords within Adobe Advertising Cloud, for which he earned a U.S. patent. Recently, Ashish mastered large language models, successfully delivering a project for profile generation. He has served as a judge in Microsoft’s hackathon, further demonstrating his expertise and leadership in AI. His work is marked by a proven ability to drive impactful, innovative solutions across complex, high-stakes applications.

Table of Contents

Preface

Part 1: Foundations of Graph Learning

1

Introduction to Graph Learning

Do we need graphs?

A case study

Formalizing graphs

Definition and semantics

Types and properties of graphs

Directed graphs

Bipartite graphs

Connected graphs

Weighted graphs

Subgraphs

Centrality

Community structure

Isomorphism

Graph data structures

Adjacency matrix

Adjacency list

Traditional graph-based solutions

Searching

Partitioning

Path optimization

The need for representation learning

GNNs and the need for a separate vertical

Summary

2

Graph Learning in the Real World

Node-level learning

Node classification

Node regression

Node clustering

Node anomaly detection

Edge-level learning

Edge classification

Edge regression

Edge clustering

Edge outlier detection

Graph-level learning

Graph-level representations

Real-world applications

Recommender systems

Knowledge graphs

Entity and relationship embeddings

Semantic similarity and entity resolution

Some other applications

Summary

3

Graph Representation Learning

Representation learning – what is it?

Graph representation learning

A framework for graph learning

DeepWalk

Random walk – the what and the why

Estimating the node embeddings

Node2Vec

Graph traversal approaches

Finalizing the random walk strategy

Node2Vec versus DeepWalk

Limitations of shallow encodings

Summary

Part 2: Advanced Graph Learning Techniques

4

Deep Learning Models for Graphs

Technical requirements

Message passing in graphs

Decoding GNNs

GCNs

Using GCNs for different graph tasks

GraphSAGE

GATs

Attention networks

Attention coefficients computation

Aggregation of neighbor features

Multi-head attention

Stacking GAT layers

Summary

5

Graph Deep Learning Challenges

Data-related challenges

Heterogeneity in graph structures

Dynamic and evolving graphs

Noisy and incomplete graph data

Model architecture challenges

Capturing long-range dependencies

Depth limitation in GNNs

Over-smoothing and over-squashing

Balancing local and global information

Facing a model architecture challenge – an example

Computational challenges

Scalability issues for large graphs

Memory constraints in graph processing

Parallel and distributed computing for graphs

Task-specific challenges

Node classification in imbalanced graphs

Link prediction in sparse graphs

Graph generation and reconstruction

Graph matching and alignment

Interpretability and explainability

Explaining GNN decisions

Visualizing graph embeddings

Summary

6

Harnessing Large Language Models for Graph Learning

Understanding LLMs

Textual data in graphs

Leveraging InstructGLM

LLMs for graph learning

LLMs as enhancers

LLMs as predictors

Integrating RAG with graph learning

Advantages of graph RAG (GRAG) approaches

Challenges in integrating LLMs with graph learning

Summary

Part 3: Practical Applications and Implementation

7

Graph Deep Learning in Practice

Setting up the environment

Creating the graph dataset

Node classification – predicting student interests

Link prediction – recommending new friendships

Summary

8

Graph Deep Learning for Natural Language Processing

Graph structures in NLP

Importance of graph representations in language

Types of linguistic graphs

Graph-based text summarization

Extractive summarization using graph centrality

Abstractive summarization with graph-to-sequence models

Information extraction (IE) using GNNs

Event extraction

Open IE

Advantages of GNN-based IE

Graph-based dialogue systems

Dialogue state tracking with GNNs

Graph-enhanced response generation

Knowledge-grounded conversations using graphs

Graph-based dialogue policy learning

Summary

9

Building Recommendation Systems Using Graph Deep Learning

Fundamentals of recommendation systems

Types of recommendation systems

Key metrics and evaluation

Graph structures in recommendation systems

User-item interaction graphs

Incorporating side information

Temporal graphs

Multi-relational graphs

Graph-based recommendation models

MF with graph regularization

Graph neural network models

Training graph deep learning models

Data preprocessing

Model training techniques

Scalability and optimization

Mini-batch training with neighborhood sampling

Distributed training

Explainability in graph-based recommendations

Attention mechanisms for interpretability

Path-based explanations

The cold start problem

Graph embedding transfer

Content-based feature integration

Summary

10

Graph Deep Learning for Computer Vision

Traditional CV approaches versus graph-based approaches

Graph construction for visual data

Pixel-level graphs

Superpixel-based graphs

Object-level graphs

Scene graphs

Comparing different graph construction methods

GNNs for image classification

Graph convolutional networks for image data

Attention mechanisms in graph-based image classification

Hierarchical graph representations for multi-scale feature learning

GNNs versus CNNs

Object detection and segmentation using GNNs

Graph-based object proposal generation

Instance segmentation with GNNs

Panoptic segmentation using graph-structured outputs

Multi-modal learning with GNNs

Integrating visual and textual information using graphs

Cross-modal retrieval using graph-based representations

GNNs for visual-language navigation

Limitations and next steps

Scalability issues in large-scale visual datasets

Efficient graph construction and updating for real-time applications

Integrating graph-based methods with other deep learning approaches

New applications and research opportunities

Summary

Part 4: Future Directions

11

Emerging Applications

Biology and healthcare

Protein-protein interaction networks

Drug discovery and development

Disease prediction and progression modeling

Brain connectomics analysis

Genomics and gene regulatory networks

Social network analysis

Community detection

Influence propagation modeling

User behavior prediction

Fake news detection

Financial services

Fraud detection in transaction networks

Credit risk assessment

Stock market prediction using company relationship graphs

Anti-money laundering systems

Personalized financial recommendations

Systemic risk assessment

Cybersecurity

Why graphs for cybersecurity?

Network intrusion detection

Malware detection

Energy systems

Graph representation of energy systems

Load forecasting

Fault detection and localization

Optimal power flow

Renewable energy forecasting

Energy storage management

Vulnerability assessment

IoT

Device interaction modeling

Anomaly detection in sensor networks

Predictive maintenance

Smart home applications

Legal governance and compliance

Knowledge graph construction for legal and regulatory data

Automated compliance monitoring and risk assessment

Legal document analysis and contract management

Regulatory intelligence and policy impact assessment

Summary

12

The Future of Graph Learning

Emerging trends and directions

Scalability and efficiency

Interpretability and explainability

Dynamic and temporal graphs

Heterogeneous and multi-modal graphs

Advanced architectures and techniques

Graph transformers and attention mechanisms

Graph generative models

Few-shot and zero-shot learning on graphs

Reinforcement learning on graphs

Integration with other AI domains

Graph learning and LLMs

Federated graph learning

Quantum GNNs

Potential breakthroughs and long-term vision

Artificial general intelligence and graphs

Neuromorphic computing with graphs

Graph learning in the Metaverse

Interdisciplinary applications

Summary

Index

Other Books You May Enjoy

Part 1: Foundations of Graph Learning

In the first part of the book, you will get an overview of the fundamental concepts of graph learning, including basic definitions, real-world applications, and core representation techniques. You will learn about the essential building blocks needed to understand graph-based machine learning, practical use cases across industries, and various methods for representing graph data in machine learning contexts.

This part has the following chapters:

Chapter 1, Introduction to Graph LearningChapter 2, Graph Learning in the Real WorldChapter 3, Graph Representation Learning