Smart Public Safety Video Surveillance System E-Book

Abhishek Djeachandrane

• Herausgeber: John Wiley & Sons
• Kategorie: Fachliteratur
• Serie: ISTE Invoiced
• Sprache: Englisch

Seitenzahl: 286

Veröffentlichungsjahr: 2025

Beschreibung

In smart cities, video surveillance is essential for public safety, evolving beyond simple camera installations and centralized monitoring due to the overwhelming amount of footage that challenges human operators. To enhance anomaly detection, experts have developed sophisticated computer vision techniques that classify events as normal or abnormal.

Smart Public Safety Video Surveillance System explores an end-to-end urban video surveillance system, which aims to address asymmetric threats through three key strategies: firstly, it employs a corrective signal called “task-specific QoE” that considers contextual factors; secondly, it utilizes machine learningdriven predictive systems and a method known as "similarity-based meta-reinforcement learning" for effective anomaly detection; and thirdly, it advocates for "zero-touch" self-management systems based on autonomous computing. This holistic approach ensures rapid adaptation and situational awareness, effectively meeting the demands of modern businesses and enhancing overall safety in dynamic urban environments.

Leseprobe

Table of Contents

Cover

Table of Contents

Dedication Page

Title Page

Copyright Page

Preface

List of Acronyms

Introduction

I.1. General context

I.2. Challenges

I.3. Objectives

I.4. Problematics and contributions

I.5. Structure of the book

1 Literature Review on an End-to-End Video Surveillance System for Public Safety

1.1. General description: human threats in urban areas and abnormal situation detection

1.2. Analytics for video surveillance

1.3. System architecture for video surveillance

1.4. Analytics and architecture: studies and reflections

1.5. Challenges

1.6. Conclusion

2 A Development Platform for Integration and Testing

2.1. Introduction

2.2. Proposed framework – QoE-driven SA-centric DSS

2.3. Use case – Airbus DS SLC’s target market

2.4. Conclusion

3 A Multi-Criteria Enriched Corrective Signal with Endogenous, Exogenous and Human Factors

3.1. Context

3.2. Problem statement

3.3. Proposals

3.4. Conclusion

4 A Situational Awareness-centric Predictive System for Anomaly Detection

4.1. Context

4.2. Baseline

4.3. Problem statement

4.4. Proposals

4.5. Conclusion

5 Towards an Autonomic Intelligent Video Surveillance System

5.1. Context

5.2. Problem statement

5.3. Proposals

5.4. Conclusion

Conclusions and Perspectives

C.1. Summary of contributions

C.2. Perspectives and future work

References

Index

Other titles from ISTE in Networks and Telecommunications

End User License Agreement

List of Tables

Introduction

Table I.1. NYPD September crime statistics

Chapter 1

Table 1.1. Comparative study of handcrafted methods in crowd scene analysis

Table 1.2. Comparative study of deep learning methods in crowd scene analysis

Table 1.3. Comparative study of learning scheme and model in crowd scene analy...

Table 1.4. Comparative study of feature modeling in crowd scene analysis

Table 1.5. Comparative study of methods in traffic analysis

Table 1.6. Comparative study of methods in environment scene analysis

Table 1.7. Comparative study of methods in individual behavior scene analysis

Table 1.8. Methods for terrorism detection in file names, audio and image anal...

Table 1.9. Comparative study of datasets

Table 1.10. Comparative study of anomalies in datasets

Table 1.11. Comparison study of feature extractors

Table 1.12. Comparison study of anomaly detection framework

Table 1.13. Comparative study of extended cognitive modules

Table 1.14. Existing studies on cross-related work between QoE and video surve...

Table 1.15. Existing studies on cross-related work between computing infrastru...

Table 1.16. Referred attacks on video surveillance system

Table 1.17. Two cases of visual data poisoning on AI-based video analytics

Table 1.18. Studies on video quality impact on video surveillance analytics ta...

Table 1.19. Study on task-specific QoE-oriented optimization for a video surve...

Chapter 2

Table 2.1. Study on the application field and input data in data enrichment

Table 2.2. Study on data storage and representation in data enrichment

Table 2.3. Study on the enrichment process in data enrichment

Table 2.4. Study of knowledge extraction from data in data enrichment

Table 2.5. Comparative study of self-* properties and tasks for autonomic comp...

Table 2.6. Comparative study on autonomic computing in video surveillance

Chapter 3

Table 3.1. Anomaly detection ground truth in reinforcement learning

Table 3.2. Image degradation impact assessed by QoP and traditional QoE in MNI...

Chapter 4

Table 4.1. Comparison study of feature extraction between 3D-CNN and LSTM and ...

Table 4.2. HMM experiment over distribution type and number of hidden nodes

Table 4.3. The pros and cons of our approach

Table 4.4. Inference and learning times of the Actor-Critic

Table 4.5. Actor-Critic experiment results

Table 4.6. Methods comparison on UCF-Crime with our proposed Actor-Critic meth...

Table 4.7. Comparison study between our interpretable and adaptive model

Table 4.8. RL algorithm comparison study with and without SOM

Table 4.9. Methods comparison on UCF-Crime with our similarity-based meta-RL m...

Table 4.10. Ablation study on proposed methods

Chapter 5

Table 5.1. Accuracy and loss values of the first method

Table 5.2. Model output on real samples of the first method

Table 5.3. P-values of three tests on different levels of noise injected in a ...

Table 5.4. P-values of three tests on different sample sizes of original data

Table 5.5. P-values of three tests to compare the full original dataset with d...

Table 5.6. Data histograms of original data and noisy data of factor 0.11

Table 5.7. Gaussian distribution fitted to original data and noisy data of fac...

Table 5.8. Data histograms of original data with increasing sample size

Table 5.9. Accuracy and loss values of the first method

Table 5.10. Event-based learning control with two statistical tests on episodi...

Table 5.11. Model output on real samples of the third method

Table 5.12. Accuracy and user request comparative study of event-based and tim...

Table 5.13. Accuracy and user request comparative study of event-based and tim...

Table 5.14. Accuracy and user request comparative study of event-based and tim...

Table 5.15. Accuracy and user request comparative study of event-based and tim...

List of Illustrations

Introduction

Figure I.1. Voluntary blows and injuries among individuals aged 15 or older (V...

Figure I.2. Recorded instances of sexual violence

Figure I.3. Recorded instances of vehicle theft

Figure I.4. Total reported crimes in the state of New York

Chapter 1

Figure 1.1. Urban area-specific direct human threats

Figure 1.2. Human threats in urban areas from cyber and physical space

Figure 1.3. Data-driven method framework

Figure 1.4. Challenges in video surveillance

Chapter 2

Figure 2.1. High-level view of our proposed system regulated by QoE

Figure 2.2. Detailed view of our proposed SA-centric DSS framework

Figure 2.3. Components of SA

Figure 2.4. Endsley’s model of SA

Figure 2.5. Human behavior analysis tasks – classification

Figure 2.6. Application use in control room.

Chapter 3

Figure 3.1. Relationships between data, knowledge and information

Figure 3.2. End-to-end smart video surveillance assessment

Figure 3.3. Quality of Prediction with an evolving ground truth built on human...

Figure 3.4. Three approaches for ground truth gathering.

Figure 3.5. HCI and vision-based QoE computing system workflow

Figure 3.6. Mouse tracker system output

Figure 3.7. Eye tracker system output

Figure 3.8. Vision-based affective computing system output for satisfied and u...

Figure 3.9. Traditional quality of experience with no reference video quality ...

Figure 3.10. Lowest and highest VIIDEO score as traditional QoE indicator.

Figure 3.11. Traditional QoE-VIIDEO score tracking in explosion.

Figure 3.12. System output – augmented QoE for content perception

Figure 3.13. System output – augmented QoE for video integrity.

Figure 3.14. Image degradation by blurry noise on MNIST dataset’s number 5...

Chapter 4

Figure 4.1. Baseline framework

Figure 4.2. Reinforcement learning as a multi-instance learning framework: tra...

Figure 4.3. Comparison between C3D and LSTM/CNN feature extractor

Figure 4.4. HMM statistical model

Figure 4.5. HMMs anomaly detection framework

Figure 4.6. HMM and SOM-based anomaly detection framework

Figure 4.7. U-matrix and activation map of SOM with 100 neurons and 10,000 neu...

Figure 4.8. Activation map comparison between abnormal and normal sets

Figure 4.9. Activation map of abnormal instances

Figure 4.10. Best HMM model AUC for frame-level anomaly detection

Figure 4.11. Our intelligent agent architecture inspired by a common utility-b...

Figure 4.12. RL for anomaly detection on surveillance videos

Figure 4.13. Actor-Critic loss and reward curves

Figure 4.14. RL and SOM-based anomaly detection framework

Figure 4.15. RL and SOM-based anomaly detection ROC curve

Figure 4.16. Flow diagram of the proposed anomaly detection and anticipation a...

Figure 4.17. N-step reinforcement learning in an MDP environment

Figure 4.18. Distance metric learning with triplet loss

Figure 4.19. Multi-task learning: reinforcement and similarity learning

Figure 4.20. Learning curve with average cumulative rewards

Figure 4.21. 2D visualization of embeddings on testing data with t-SNE before ...

Figure 4.22. Novel similarity-based meta-RL framework ROC curve – AUC

Chapter 5

Figure 5.1. User-oriented self-configuring IVS

Figure 5.2. Accuracy and loss curves of the first method

Figure 5.3. Confusion matrix of the first method

Figure 5.4. User-centric self-protecting IVS

Figure 5.5. Data deterioration due to salt and pepper noise levels ranging fro...

Figure 5.6. Boxplot comparative study of different levels of noise injected in...

Figure 5.7. Boxplot comparative study of different sample size of original dat...

Figure 5.8. Comparative study of boxplots for the full original dataset and di...

Figure 5.9. Accuracy and loss curves of the second method

Figure 5.10. Confusion matrix of the second method

Figure 5.11. Learning model accuracy on normal and noisy data over time

Figure 5.12. Event-based learning control with two statistical tests on episod...

Figure 5.13. User-centric self-aware and self-adaptive DSS

Figure 5.14. Accuracy and loss curves of the third method

Figure 5.15. Confusion matrix: first, fire detection and second, video quality...

Figure 5.16. Comparative study of event-based and time-based methods on noisy ...

Figure 5.17. Two experiments under identical conditions of adaptive models

Figure 5.18. Sample of noise level of 0.4 applied to the system

Guide

Cover Page

Table of Contents

Dedication

Title Page

Copyright Page

Preface

List of Acronyms

Introduction

Begin Reading

Conclusions and Perspectives

Index

Other titles from ISTE in Networks and Telecommunications

Wiley End User License Agreement

Pages

ii

iii

iv

ix

x

xi

xii

xiii

xiv

xv

xvi

xvii

xviii

xix

xx

xxi

xxii

xxiii

xxiv

xxv

xxvi

xvii

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

To my beloved family, Jean, Soumady, Archana, Vijayalakshmi, and Pecto, with all my love and gratitude.

Abhishek Djeachandrane

To all my family, my wife, with all my love and gratitude.

Serge Delmas

To my beloved wife, my wonderful children, and my dear family – your love fills my life with joy and purpose. I am endlessly grateful for each of you.

Said Hoceini

To my grown-up daughter, Ikram, who has just married and whom I wish full happiness in her new life. No matter how you grow, you will always be my beloved little girl.

Abdelhamid Mellouk

New Generation Networks Set

coordinated by Abdelhamid Mellouk

Volume 5

Smart Public Safety Video Surveillance System

Innovative Technologies for Homeland Security and Mission-Critical Operations

Abhishek Djeachandrane

Said Hoceini

Serge Delmas

Abdelhamid Mellouk

First published 2025 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.

Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the undermentioned address:

ISTE Ltd

27-37 St George’s Road

London SW19 4EU

UK

www.iste.co.uk

John Wiley & Sons, Inc.

111 River Street

Hoboken, NJ 07030

USA

www.wiley.com

© ISTE Ltd 2025The rights of Abhishek Djeachandrane, Said Hoceini, Serge Delmas and Abdelhamid Mellouk to be identified as the authors of this work have been asserted by them in accordance with the Copyright, Designs and Patents Act 1988.

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s), contributor(s) or editor(s) and do not necessarily reflect the views of ISTE Group.

Library of Congress Control Number: 2025934394

British Library Cataloguing-in-Publication DataA CIP record for this book is available from the British LibraryISBN 978-1-83669-054-2

Preface

In an era marked by rapid technological advancements and an increasing emphasis on public safety, the integration of intelligent systems into urban environments has become essential. The development of Smart Public Safety Video Surveillance Systems (SPSVSS) represents a significant step toward enhancing community security, empowering law enforcement officials to respond proactively to incidents while upholding civil liberties.

The journey presented in this book, the result of a collaboration between AIRBUS Defence and Space and UPEC (Paris XII University), stems from rigorous research and innovative engineering, focused on creating systems that intelligently analyze data in real time to monitor public spaces. Traditional surveillance technologies are insufficient for addressing the complexities of modern urban settings, highlighting the need for intelligent systems that leverage artificial intelligence, machine learning, and data analytics.

This work aims to bridge theoretical advancements with practical applications in public safety in order to develop advanced surveillance systems that adapt to their environments and provide critical insights for security personnel. Throughout this book, readers will encounter a comprehensive analysis of methodologies, challenges, and achievements.

A part of this this book was initially based on the work done in the framework of the Abhishek Djeachandrane PhD thesis, under the direction of Pr. Abdelhamid Mellouk. It was subsequently developed into a book to facilitate understanding of the technical aspects and the broader implications of surveillance technology. The findings originate from extensive lab work, collaborative discussions, experiments, and data analysis, helping readers understand key concepts and methodologies related to SPSVSS and stimulating dialogue about future advancements in the field.

To give a complete bibliography and a historical account of the research that led to the present form of the subject would be impossible. It is thus inevitable that some topics have been treated in less detail than others. The choices made reflect in part personal taste and expertise, and in part a preference for very promising research and recent developments in the field of technologies-based public safety, highlighting the ongoing need for research and development in this vital area.

This book is a start, but also leaves many questions unanswered. We hope that it will inspire a new generation of investigators and investigations.

The authors hope you will enjoy reading this book and get many helpful ideas and overviews for your own study.

Abhishek DJEACHANDRANE

Said HOCEINI

Serge DELMAS

Abdelhamid MELLOUK

April 2025

List of Acronyms

2D:

Two-dimensional

3D:

Three-dimensional

3GPP:

The 3rd Generation Partnership Project

5G:

Fifth Generation Technology Standard

6G:

Sixth Generation Technology Standard

AC:

Autonomic Computing

A-C:

Actor-Critic

ACC:

Accuracy

AdaBoost:

Adaptive Boosting

ADC:

Analog-to-Digital Converter

AI:

Artificial Intelligence

API:

Application Programming Interface

AUC:

Area-Under-The-Curve

BCE:

Binary Cross-Entropy

BMU:

Best Matching Unit

C3D:

Convolutional 3D/3D ConvNets

CCTV:

Closed-circuit television

CE:

Cross-Entropy

CLT:

Central Limit Theorem

CNN:

Convolutional Neural Network

CS:

Camera Station

CVAE:

Convolutional Variational Auto-encoder

DAC:

Discriminant Analysis and Classification

DB:

Database

DDoS:

Distributed Denial-of-Service

DL:

Deep Learning

DMIL:

Deep Multi-Instance Learning

DRL:

Deep Reinforcement Learning

DSS:

Decision Support System

E2E:

End-to-End

ECG:

Electrocardiogram

EDA:

Estimation of Distribution Algorithm

ELM:

Extreme LearningMachine

EM:

Expectation–Maximization

FAA:

Federal Aviation Administration

FE:

Feature Extractor

FFNN:

Feed Forward Neural Network

FN:

False Negative

FR:

Full Reference

GB:

Gradient Boosting

GCN:

Graph Convolutional Network

GDPR:

General Data Protection Regulation

GMM:

Gaussian Mixture Model

GPS:

Global Positioning System

GRU:

Gated Recurrent Unit

HCAI:

Human-Centered Artificial Intelligence

HCI:

Human–Computer Interaction

HFST:

High-Frequency and Spatio-Temporal

HMM:

Hidden Markov Model

HOF:

Histograms of Optical Flow

HOFO:

Histograms of Optical Flow Orientation

HOG:

Histograms of Oriented Gradient

I3D:

Inflated 3D/Two-Stream Inflated 3D ConvNets

IFC:

Industry Foundation Classes

IoT:

Internet of Things

IP:

Internet Protocol

IQA:

Image Quality Assessment

IR:

Information Retrieval

IT:

Information Technology

IVS:

Intelligent Video Surveillance

KL:

Kullback–Leibler

KLT:

Kanade–Lucas–Tomasi

KNN:

K-Nearest Neighbors

LQ:

Luminance Quality

LSB:

Least Significant Bit

LST:

Local Spatio-Temporal

LSTM:

Long Short-Term Memory

MAB:

Multi-Armed Bandit

mAP:

mean Average Precision

MC:

Monte Carlo

MCX/

MCS:

Mission Critical Communications

MDP:

Markov Decision Process

MIL:

Multi-Instance Learning

ML:

Machine Learning

MLP:

Multi-Layer Perceptron

MOS:

Mean Opinion Score

MSE:

Mean Square Error

MTL:

Multi-Task Learning

MW:

Mann–Whitney

NIQE:

Natural Image Quality Evaluator

NN:

Neural Network

NR:

No-Reference

NYPD:

New York Police Department

OWL:

Web Ontology Language

PCA:

Principal Component Analysis

PoC:

Proof of Concept

PSNR:

Peak Signal-to-Noise Ratio

QoE:

Quality of Experience

QoS:

Quality of Service

R-CNN:

Region-Convolutional Neural Network

RDF:

Resource Description Framework

RF:

Random Forest

RNN:

Recurrent Neural Network

ROC:

Receiver Operating Characteristic

RoI:

Region of Interest

RPCA:

Region Principal Component Analysis

RTFM:

Robust Temporal Feature Magnitude

SA:

Situational Awareness

SL:

Similarity Learning

SOM:

Self-Organizing Map

SoTA:

State of The Art

SS:

Surveillance System

SSIM:

Structural Similarity Index Measure

ST:

Spatio-Temporal

STT:

Spatio-Temporal Texture

SVM:

Support Vector Machine

TD:

Time-Difference

TF-IDF:

Term Frequency-Inverse Document Frequency

TN:

True Negative

TSN:

Temporal Segment Network

tSNE:

t-distributed Stochastic Neighbor Embedding

UAV:

Unmanned Aerial Vehicle

VANET:

Vehicular Ad-hoc Network

VCA:

Video Content Analysis

vMOS:

video Mean Opinion Score

VQA:

Video Quality Assessment

WCAE:

Weighted Convolutional Autoencoder

WCN:

Wireless Camera Network

XGBoost:

Extreme Gradient Boosting

Introduction

“The world is a dangerous place to live.”

Albert Einstein

I.1. General context

In France, according to the French Ministry (2023), almost all recorded delinquency indicators were up in 2022 compared with the previous year. These increases follow those observed before the health crisis for homicides, intentional assault and battery, sexual violence and fraud recorded by the police and gendarmerie. For example, the number of recorded victims of intentional assault and battery (persons aged 15 or over) rose sharply in 2022 (+15%, after +12% in 2021).

Recorded crime indicators relating to non-violent theft from persons, burglary, vehicle theft, theft from vehicles and theft of vehicle accessories, which had fallen sharply during the health crisis, rose sharply in 2022.

According to statistics from the The French Central Directorate of the Judicial Police (2023), as shown in Figure I.1, the crime rate in France is on the rise compared to 2021. Against a backdrop of almost constant increases since 2008 (with a drop during the 2020 pandemic), however, with certain disparities, for example, vehicle theft is down as shown in Figure I.3, while physical and sexual violence are up as shown in Figure I.2.

Figure I.1.Voluntary blows and injuries among individuals aged 15 or older (VBI), and intra-family violence (IFV)

Figure I.2.Recorded instances of sexual violence

Figure I.3.Recorded instances of vehicle theft

Statistics for specific types of violence, such as sexual violence or vehicle theft, attest to the existence of violence, even if trends are evolving differently, as shown in these respective figures, which show developments between 2008 and 2022.

This question of national security concerns not only countries such as France but also some important cities such as the world’s largest city, New York, which is sounding the alarm.

For instance, according to the New York Police Department (2023), the apparent dips come after total major crimes spiked 22% last year, from 103,388 incidents in 2021 to 126,537 in 2022, as attested in Figure I.4 provided by the Office of the New York State Comptroller (2022) where the number of total crimes and total crime rates are rising for the first time in eight years.

Figure I.4.Total reported crimes in the state of New York

As shown in Table I.1 by amNewYork Metro (2022), that rise was driven by increases in six of the seven categories: rapes, robberies, felony assaults, burglaries, grand larcenies and car thefts, according to NYPD statistics.

Table I.1.NYPD September crime statistics

Category

2022 numbers

2021 numbers

Difference

Percentage change

Murder

39

51

↓ 12

↓ 23.5%

Rape

145

131

↑ 14

↑ 10.7%

Robbery

1,508

1,295

↑ 213

↑ 16.4%

Burglary

2,189

2,218

↓ 29

↓ 1.3%

Grand larceny

4,409

1,148

↑ 261

↑ 22.7%

Auto theft

1,215

3,753

↑ 799

↑ 21.3%

TOTAL

11,057

9,000

↑ 215

↑ 21.5%

Criminal activity has been a persistent issue for societies throughout history. It takes on many forms and can occur in many different ways. It is important for law enforcement to be aware of these changes and to adapt their tactics to effectively combat crime and protect the public. Advances in technology have provided new tools for authorities to use in this fight against criminality, and it is important to make use of these tools to enhance public safety.

Indeed, the rising crime rate in France is a concern not only for law enforcement agencies but also for solution providers such asAirbus. Airbus is a global leader in aeronautics, space and associated services. One of its program units, secure land communications (SLC), offers advanced communication and collaboration solutions to help customers collect, process and distribute relevant information. SLC’s portfolio caters to the needs of professionals in public safety, transportation, energy and utilities sectors. It includes networks, terminals, applications and services based on Tetra, Tetrapol and broadband technologies. As a European leader and a major player in the global market, SLC has customers in over 80 countries and employs approximately 1,150 people in 17 countries. Being a critical solution provider in the world, it is committed to meeting all the requirements set by final users.

Physical crime has been a long-standing concern for many communities. It involves offenses such as assault, robbery and homicide, and can have a significant impact on victims and their families. Law enforcement agencies need to be vigilant in their efforts to prevent these crimes from occurring, and to bring those responsible to justice. The use of technology can play an important role in this effort, providing tools such as surveillance cameras and forensic analysis to aid in investigations. Overall, it is crucial to take a proactive approach to combat physical crime and ensure the safety of the public.

Public safety refers to measures taken by governments and communities to protect citizens from harm and maintain order in society. This includes maintaining safe and secure communities, preventing and responding to crime and ensuring the safety of public spaces and infrastructure. Public safety is a critical component of smart cities as advanced technology and data analysis can be used to improve emergency response times, monitor crime trends and enhance overall safety for residents.

As mentioned, smart cities can potentially help to prevent crimes by implementing technologies such as surveillance cameras and predictive policing algorithms. These tools can aid in identifying and addressing criminal behavior in real time, thereby increasing public safety. Smart cities are urban areas that use advanced technology and data analysis to improve the quality of life for residents and enhance sustainability. This includes using sensors, data analytics and automation to improve transportation, energy usage, waste management and public safety. Smart cities are becoming increasingly popular around the world as more and more people move to urban areas and governments seek innovative solutions to urban challenges.

For public safety purposes, video surveillance is a crucial aspect of maintaining security and safety in various settings. Whether it is in public places such as malls, airports or train stations, or in private locations such as homes or businesses, video surveillance systems can provide an extra layer of protection and deter potential criminal activity. Video surveillance can help prevent theft, vandalism and other criminal acts. It can also aid in identifying suspects and providing evidence for investigations. In addition, video surveillance can help monitor employee behavior, improve workplace safety and ensure compliance with regulations and policies. Moreover, video surveillance can also be used for remote monitoring, allowing for real-time observation of events or incidents that require immediate attention. With advancements in technology, video surveillance systems can now incorporate features such as facial recognition, license plate recognition and motion detection, enhancing their effectiveness and accuracy.

Overall, video surveillance plays a vital role in maintaining safety and security, providing valuable insights and enabling quick responses to potential threats or incidents. For all the reasons mentioned, video surveillance is considered a critical service because it plays a crucial role in ensuring the safety and security of people and property. It can be used to monitor and detect potential threats, prevent crime and provide evidence in the event of an incident. In many cases, video surveillance systems are integrated with other security measures to create a comprehensive security solution. The reliability and effectiveness of these systems is paramount, as they can have a significant impact on public safety and the protection of valuable assets.

A general description of video surveillance will allow us to better identify principal challenges, by giving a general overview of video surveillance systems. In this area, the end devices in the architecture are considered the eye of the system; the communication media are designed as synapses to convey information and the analytics represent the brain of the system. Here, video surveillance will be defined by architecture and analysis, the former consisting of network architecture and computing infrastructure, the latter of all the analysis that can be produced from the information gathered.

In video surveillance architecture, network architecture could be defined as follows.

Analog surveillance systems: analog signal processing technology is the basis model for image transmission, exchange and recording using a short-distance coaxial cable and a long-distance transceiver optical fiber.

Digital surveillance systems: by introducing digital video recorders (DVR), digital image files are transmitted. This differs from analog video surveillance in these aspects: signal transmission, control and storage. Video encoding formats like MPEG-4 and H264 enable digital video surveillance systems to conduct image monitoring with remote transmission across all existing digital networks while utilizing low bandwidth.

Network surveillance systems: the network system is based on digital signal processing and involves network cameras or IP cameras, which is a type of digital video camera. The networking technique is used to realize signal transmission, exchange, control and video storage. All the components could be integrated into the system in a wireless or wired way to tackle many problems on this topic.

The key difference between these architectures is in transmission, exchange and storage. The main difference between digital and network surveillance systems is that digital surveillance systems (SS) unlock new capabilities of digital signal processing contrary to analogical signal, and network SS unlock new capabilities of processing through Internet protocol. In general, digital SS convert analogical signals to digital ones, while the network ones deal only with digital signals.

Computing infrastructure is where data from edge devices is processed and stored and could be defined as:

Cloud computing: cloud computing refers to the practice of accessing computing resources, such as servers, storage and applications, over the Internet instead of owning and maintaining them locally.

Fog computing: refers to a decentralized computing architecture that involves using localized devices to bring computing resources closer to the edge of the network, which can help reduce latency, improve security and lower bandwidth usage.

Edge computing: edge computing is a distributed computing paradigm that involves processing and storing data at or near the edge of the network, where the data is generated. It typically involves using more powerful devices.

Local computing: local computing usually refers to computing at one premise and is organized as a local area network.

The key difference between these infrastructures is in the localization of intelligence and computing power.

In video surveillance analytics, these tasks and goals could be defined as:

face detection or recognition;

human action detection or recognition;

object detection and tracking;

crowd counting;

video summarization;

abnormal event detection;

traffic road congestion/accident detection;

car plate recognition;

abnormal behavior detection.

One of the primary responsibilities of an automated surveillance system is to monitor the environment being surveilled through the use of cameras, identify and evaluate the observed situation efficiently, and subsequently alert appropriate personnel if necessary. All of this is done by its analytics components which could be modeled by:

traditional computer programs;

computer programs coupled to image processing techniques;

Artificial Intelligence (rule-based or data-driven) and its computer vision domain.

The key difference between these analytics solutions is in their information source and goal complexity.

On the one hand, the implementation of smart technologies such as surveillance cameras and predictive policing algorithms can potentially help reduce crime rates and increase public safety. On the other hand, concerns have been raised about the potential for these technologies to infringe on privacy and civil liberties governed by the GDPR in Europe, for example, as well as the potential for bias and discrimination in the use of predictive policing algorithms.

I.2. Challenges

With the evolution of technologies (exponential increase in data exchange capacities with 4G and now 5G, with 6G announced for the end of the 2020s, or the advent of the IoT phenomenon, which is still in its infancy), the multiplication of information sources and flows (open sources, social networks, IoT sources, etc.), and the availability of increasingly powerful and diverse terminals and accessories (smartphones, tablets, connected watches, cameras, augmented and virtual reality devices (e.g. glasses), biometric and environmental sensors, etc.), administrations, organizations and security agents (public security, civil security, private security) in the field and in command centers have, and will increasingly have, access to a considerable amount of information whose “intelligent” exploitation could enable disruptive advances in the accomplishment and effectiveness of their missions.

What is more, we need to bear in mind that the agents concerned may be operating in a wide variety of conditions (pedestrian or vehicular agents, or in a command center or crisis call management), in difficult or even hostile environments and in stressful situations (hands busy, driving, attention required on a critical task or potential danger).

The interaction of these professional users with real-time information flows in these constrained operational conditions therefore requires the use of innovative technologies such as Artificial Intelligence, providing them with, among other things, mission-specific information analysis and presentation and decision support capabilities, as well as operational efficiency.

I.3. Objectives

The field of application of Artificial Intelligence technologies to security missions, as outlined in the previous section, is vast. The work targeted by the book will be essentially related to the four themes developed below.

Augmented situational information.

Situational information is to be understood here in the sense of situational awareness, i.e. the perception and understanding of the spatio-temporal environment and the events taking place within it, and their possible projection into the future. It is an essential element in decision-making in a number of fields, notably for control, safety and rescue missions, whether in the public domain (public safety, civil security) or the private sector (transport, industry, essential services). As mentioned in section I.2, the evolution of technologies and information sources means that security and rescue professionals now have access to vast quantities of multimedia (data, voice, video), multi-source (IoT sensors, cameras, command centers, specialized applications, public networks, open sources, social networks, the web, the dark web, etc.), real-time and non-real-time data. The main objective is therefore to be able to use this data to build up “augmented” real-time (or near-real-time) situational information using Artificial Intelligence technologies in a way that is relevant and contextual to the mission and situation, integrated with mission control and execution tools.

Operational decision support.

Complementing (or coupled with) “augmented situational information” (see the relevant paragraph above), based on the same data and information, possibly supplemented by other sources of intelligence, drawing on business procedures and their historical data (often kept for legal reasons), real-time (or near-real-time) decision support functionalities using Artificial Intelligence technologies, relevant and contextual to the mission and situation, integrated with mission control and execution tools, would be a major added value for both command room personnel and agents deployed in the field.

Intelligent interaction.