Intelligent Data Mining and Analysis in Power and Energy Systems E-Book

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Serie: IEEE Press Series on Power Engineering
• Sprache: Englisch

Seitenzahl: 897

Veröffentlichungsjahr: 2022

Beschreibung

Intelligent Data Mining and Analysis in Power and Energy Systems A hands-on and current review of data mining and analysis and their applications to power and energy systems In Intelligent Data Mining and Analysis in Power and Energy Systems: Models and Applications for Smarter Efficient Power Systems, the editors assemble a team of distinguished engineers to deliver a practical and incisive review of cutting-edge information on data mining and intelligent data analysis models as they relate to power and energy systems. You'll find accessible descriptions of state-of-the-art advances in intelligent data mining and analysis and see how they drive innovation and evolution in the development of new technologies. The book combines perspectives from authors distributed around the world with expertise gained in academia and industry. It facilitates review work and identification of critical points in the research and offers insightful commentary on likely future developments in the field. It also provides: * A thorough introduction to data mining and analysis, including the foundations of data preparation and a review of various analysis models and methods * In-depth explorations of clustering, classification, and forecasting * Intensive discussions of machine learning applications in power and energy systems Perfect for power and energy systems designers, planners, operators, and consultants, Intelligent Data Mining and Analysis in Power and Energy Systems will also earn a place in the libraries of software developers, researchers, and students with an interest in data mining and analysis problems.

Leseprobe

Table of Contents

Cover

Title Page

Copyright

Dedication

About the Editors

List of Contributors

Foreword

Introduction

Part I: Data Mining and Analysis Fundamentals

1 Foundations

1.1 Data Mining: Why and What?

1.2 Data Mining into KDD

1.3 The Data Mining Process

1.4 Data Mining Task and Techniques

1.5 Data Mining Issues and Considerations

1.6 Summary

References

2 Data Mining and Analysis in Power and Energy Systems: An Introduction to Algorithms and Applications*

Acronyms

2.1 Introduction

2.2 Data Mining Technologies

2.3 Data Mining Applications in Power Systems

2.4 Discussion and Final Remarks

References

Note

3 Deep Learning in Intelligent Power and Energy Systems

3.1 Introduction

3.2 Deep Learning

3.3 Accomplishments, Limitations, and Challenges

3.4 Conclusions

References

Part II: Clustering

4 Data Mining Techniques Applied to Power Systems

4.1 Introduction

4.2 Data Mining Techniques

4.3 Data Mining Techniques Applied to Power Systems

4.4 Electrical Tariffs Design Based on Data Mining Techniques

4.5 Data Mining Contributions to Characterize Zonal Prices

4.6 Data Mining‐Based Methodology for Wind Forecasting

4.7 Final Remarks

References

5 Synchrophasor Data Analytics for Anomaly and Event Detection, Classification, and Localization

5.1 Introduction

5.2 Synchrophasor Data Quality Issues and Challenges

5.3 ML‐Based Anomaly Detection, Classification, and Localization (ADCL) Over Data Drifting Multivariate Synchrophasor Data Streams

5.4 Synchrophasor Data Anomaly and Event Detection, Localization, and Classification (SyncAED)

5.5 Test‐Bed and Test Cases

5.6 Results and Discussion

5.7 Summary

Acknowledgments

References

6 Clustering Methods for the Profiling of Electricity Consumers Owning Energy Storage System

6.1 Introduction

6.2 Methodology Definition

6.3 Clustering of Consumers with ESS

6.4 Conclusion

Acknowledgments

References

Part III: Classification

7 A Novel Framework for NTL Detection in Electric Distribution Systems

7.1 Introduction

7.2 Data Acquisition and Pre‐Processing

7.3 Feature Extraction

7.4 Classification Strategies

7.5 Evaluation

7.6 Experiments

7.7 Conclusion

References

8 Electricity Market Participation Profiles Classification for Decision Support in Market Negotiation

8.1 Introduction

8.2 Bilateral Negotiation

8.3 Decision Support for Bilateral Negotiations

8.4 Illustrative Results

8.5 Conclusion

References

9 Socio‐demographic, Economic, and Behavioral Analysis of Electric Vehicles

9.1 Introduction

9.2 Electric Vehicle Outlook

9.3 Data Mining Models for EVs

9.4 Conclusions

References

Part IV: Forecasting

10 A Multivariate Stochastic Spatiotemporal Wind Power Scenario Forecasting Model

10.1 Introduction

10.2 Generalized Dynamic Factor Model

10.3 Conclusion

References

11 Spatiotemporal Solar Irradiance and Temperature Data Predictive Estimation

11.1 Introduction

11.2 Virtual Weather Stations

11.3 Distributed Weather Forecasting

11.4 Results and Discussion

11.5 Summary

Acknowledgment

References

12 Application of Decomposition‐Based Hybrid Wind Power Forecasting in Isolated Power Systems with High Renewable Energy Penetration

12.1 Introduction

12.2 Decomposition Techniques

12.3 Decomposition‐Based Neural Network Forecasting

12.4 Forecast‐Based Dispatch in Isolated Power Systems

12.5 Case Studies

12.6 Conclusions and Discussions

References

Part V: Data Analysis

13 Harmonic Dynamic Response Study of Overhead Transmission Lines

13.1 Introduction to Methodology

13.2 Problem Formulation

13.3 Numerical Analysis

13.4 Conclusion

13.A Appendix

References

14 Evaluation of Shortest Path to Optimize Distribution Network Cost and Power Losses in Hilly Areas: A Case Study

14.1 Introduction

14.2 Design of Power Distribution Network

14.3 Digital Elevation Map

14.4 Placement of Generators and Load Centers

14.5 Single Line Diagram of 9‐Bus System

14.6 Finding Shortest Path Between Load/Generating Centers

14.7 Selection of Conductor Using Newton Raphson Method

14.8 Calculation of CO

2

Emission Cost Saving

14.9 Overall Cost Estimation of Distribution System

14.10 Sensitivity Analysis

14.11 Conclusion

References

15 Intelligent Approaches to Support Demand Response in Microgrid Planning

15.1 Introduction

15.2 Microgrid Planning

15.3 Demand Response in Microgrids

15.4 Intelligent Approaches to Support Demand Response

15.5 Conclusion

References

16 Socioeconomic Analysis of Renewable Energy Interventions: Developing Affordable Small‐scale Household Sustainable Technologies in Northern Uganda

16.1 Introduction

16.2 Renewable Energy Technologies

16.3 Methodology

16.4 Application of the Method

16.5 Case Study Results for Product Development

16.6 Cost–Benefit Analysis (CBA)

16.7 Conclusion

References

Note

Part VI: Other Machine Learning Applications

17 Non‐Intrusive Load Monitoring Using A Parallel Bidirectional Long Short‐Term Memory Model

17.1 Introduction

17.2 NILM System and Data Preprocessing

17.3 Proposed Method

17.4 Validation

17.5 Conclusion

References

Note

18 Reinforcement Learning for Intelligent Building Energy Management System Control

*

Chapter Objectives

18.1 Introduction

18.2 Reinforcement Learning

18.3 Applications of Deep Reinforcement Learning in Building Energy Management Systems Control

18.4 Challenges and Research Directions

18.5 Conclusions

References

Note

19 Federated Deep Learning Technique for Power and Energy Systems Data Analysis

Nomenclature

19.1 Introduction

19.2 Federated Learning (FL)

19.3 Power Systems Challenges and the Performance of Artificial Intelligence Techniques in It

19.4 Application of Federated Deep Learning in Power and Energy Systems

19.5 Conclusion

References

20 Data Mining and Machine Learning for Power System Monitoring, Understanding, and Impact Evaluation

20.1 Introduction

20.2 Power System Monitoring with Phasor Measurement Unit Data

20.3 Power System Mechanistic and Predictive Understanding

20.4 Characterization and Modelling of Weather and Power Extremes

20.5 Conclusion

References

Conclusions

Index

End User License Agreement

List of Tables

Chapter 2

Table 2.1 Selected load profiling tasks and algorithms in power systems.

Table 2.2 Selected forecasting tasks and algorithms in power systems.

Table 2.3 Selected FDD tasks and algorithms in power systems.

Table 2.4 Selected applications in diverse topics on power systems.

Chapter 3

Table 3.1 Inclusion criteria.

Table 3.2 Exclusion criteria.

Table 3.3 Summary of other existing work for regression problems concerning...

Table 3.4 Summary of other existing work for classification problems concer...

Table 3.5 Summary of other existing work for decision‐making problems conce...

Chapter 4

Table 4.1 Clustering validity indices.

Table 4.2 Normalized indices to characterize electricity customers' behavio...

Table 4.3 Results of the partitions selected by the validity indices.

Table 4.4 Classification indices results for the obtained clusters – annual...

Table 4.5 Rule set of the classification model – annual working days.

Table 4.6 Rule set classification for a real MV consumer.

Table 4.7 Estimated accuracy of the ANN simulations model.

Chapter 5

Table 5.1 Statistical parameter formulation for scoring PMUs.

Table 5.2 PMUNET simulation result with distributed deep autoencoder learni...

Table 5.3 Simulation result for PMUNET for event detection and localization...

Table 5.4 Simulation result for PMUNET for event classification.

Table 5.5 Data drift scenarios for IEEE 14 bus system.

Table 5.6 Performance of anomaly detector.

Table 5.7 Simulation result for SyncAED for event detection and localizatio...

Table 5.8 Simulation result for SyncAED for event classification.

Chapter 6

Table 6.1 Active community characterization.

Table 6.2 Partitional clustering – ESS per group (

k

‐means).

Table 6.3 Partitional clustering – ESS per group (PAM).

Table 6.4 Fuzzy clustering – Number of ESS per group (

c

‐means).

Table 6.5 Hierarchical clustering: group attribution.

Chapter 7

Table 7.1 Performance metrics for the highest value of

Q

obtained in the he...

Table 7.2 Performance comparison without the pre‐processing stage and using...

Table 7.3 Performance comparison with pre‐processing and using the proposed...

Chapter 8

Table 8.1 Clustering input values for all negotiation strategies.

Table 8.2 Buyer's strategies and their respective cluster.

Table 8.3 Training data used in the ANN and SVM with a maximum of 8 proposa...

Table 8.4 Counter‐strategies to use against buyers.

Table 8.5 Negotiation against buyer using Gluttonous + Anxious strategy.

Table 8.6 Strategies performance comparison against buyer using Gluttonous ...

Table 8.7 Negotiation against buyer using Gluttonous strategy.

Table 8.8 Strategies performance comparison against buyer using Gluttonous ...

Chapter 10

Table 10.1 Average performance of forecasting models.

Table 10.2 Comparison of computing time and cost in case 1.

Table 10.3 Comparison of computing time and loss in case 2.

Chapter 11

Table 11.1 Solar irradiance estimation results in W/m

2

.

Table 11.2 Temperature estimation results in °C.

Table 11.3 Solar irradiance prediction results in W/m

2

.

Table 11.4 Temperature prediction results in °C.

Table 11.5 MAE and RMSE of three regions for Case 3 predictions.

Chapter 12

Table 12.1 The 12‐month comparison of control strategies.

Chapter 13

Table 13.1 Comparison of basic conductor motions.

Table 13.2 Standards for various design aspects.

Table 13.3 Specifications of major accessories.

Table 13.4 Elastomeric bushing properties.

Table 13.5 Frequency modulation set given as input to simulation (frequency...

Table 13.6 Calculated Rayleigh damping and modal frequencies of the model....

Table 13.7 Phase lag for dampers placed in test span 250 m.

Table 13.8 Summary of displacements.

Chapter 14

Table 14.1 Coordinates of the selected villages and maximum load required a...

Table 14.2 Real and reactive power values on the buses at peak load (22nd h...

Table 14.3 Type of conductor and series impedance (Ω/km) [16].

Table 14.4 Distances between each section using DA and BBO.

Table 14.5 ACSR conductor specification of 3‐phase 4‐wire system [19].

Table 14.6 Emission of CO

2

from diesel generator for CCS, LFS, PSS, and onl...

Table 14.7 Emission of CO

2

in terms of cost saved using CCS, LFS, and PSS [...

Table 14.8 Cost of energy after optimal sizing [7], distribution network co...

Chapter 15

Table 15.1 Characteristics of BESSs and FCs for planning problem of microgr...

Chapter 16

Table 16.1 Alternative source of energy in Northern Uganda.

Table 16.2 Elements considered in the CBA.

Table 16.3 Modern bioenergy and small‐scale renewable energy for households...

Table 16.4 Firewood and charcoal saving from the use of alternative sources...

Table 16.5 Benefits for firewood saved.

Table 16.6 Benefits of reduced cooking time.

Table 16.7 Benefits to preserve the local forest resources.

Table 16.8 Health benefits.

Table 16.9 Greenhouse gas emissions reduction.

Table 16.10 Summary of benefits.

Table 16.11 Cost of intervention for biogas and solar cooker.

Table 16.12 Result of NPV and IRR.

Chapter 17

Table 17.1 Windows length and handpicked maximum power for each appliance....

Table 17.2 Statistical features.

Table 17.3 Hyperparameters.

Table 17.4 Threshold and maximum power of each appliance.

Table 17.5 Household appliances description of REDD dataset.

Table 17.6 REDD dataset.

Table 17.7 Performance comparison.

Table 17.8 PTECA comparison in ranking representation.

Table 17.9 F1 comparison in ranking representation.

Chapter 18

Table 18.1 Summary of existing work for optimal HVAC control with respect t...

Table 18.2 Summary of work on optimal HVAC control with respect to environm...

Table 18.3 Summary of existing work on RL for optimal water heater control....

Table 18.4 Summary of existing work for optimal water heater control.

Table 18.5 Summary of existing method settings on RL to control other devic...

Table 18.6 Summary of existing execution settings on RL to control other de...

Chapter 20

Table 20.1 Fault types simulated in the polish system.

Table 20.2 The frequency of the extreme weather events matched with outages...

Table 20.3 The cross‐tabulation of extreme weather events types and outage ...

Table 20.4 Prediction performance of different forecast models for Wind out...

Table 20.5 Prediction performance of different forecast model for Tree/Tree...

List of Illustrations

Chapter 1

Figure 1.1 The outline of the KDD process.

Figure 1.2 Hierarchy of data mining tasks.

Figure 1.3 Graphical representation of a clustering task.

Figure 1.4 Graphical representation of a regression task.

Figure 1.5 Graphical representation of the KNN algorithm for classification....

Figure 1.6 Graphical representation of the SVM.

Figure 1.7 In (a) a Perceptron compared to (b) an organic neuron.

Chapter 2

Figure 2.1 An example of load pattern shape clustering. Using k‐means algori...

Figure 2.2 An example of an ANN that receives as inputs market prices and en...

Figure 2.3 Taxonomy of data mining applications in PES.

Chapter 3

Figure 3.1 Number and percentage of publications per category (i.e. regressi...

Figure 3.2 Number of publications per year included in the review.

Chapter 4

Figure 4.1 Knowledge discovery in database process.

Figure 4.2 Frequently operations in data cleansing phase.

Figure 4.3 DM application: from data to knowledge.

Figure 4.4 Taxonomy of the clustering methods.

Figure 4.5 Example of two clustering validation indices results. (a) Normali...

Figure 4.6 Electrical customers' classification model methodology.

Figure 4.7 Electrical customers' characterization methodology.

Figure 4.8 Representative load diagram with (gray curve) and without (dashed...

Figure 4.9 Representative load diagrams (b) after and (a) before normalizati...

Figure 4.10 Division and reduction of database volume.

Figure 4.11 K‐Means algorithm results for

k

 = 4 (annual working days).

Figure 4.12 Cluster representative load diagrams using normalized shape indi...

Figure 4.13 Classification model using a decision tree.

Figure 4.14 Normalized shape indices of 4 MV costumers.

Figure 4.15 Example of a new electricity customer classification. (a) Typica...

Figure 4.16 Electrical tariffs design methodology based on data mining techn...

Figure 4.17 Electrical tariffs proposal for cluster 1 (Figure 4.12a). (a) Ta...

Figure 4.18 Electrical tariffs proposal for cluster 1 (Figure 4.12a). (a) Ta...

Figure 4.19 Data‐mining‐based methodology to characterize locational margina...

Figure 4.20 Clustering algorithms evaluation. (a) Clustering dispersion indi...

Figure 4.21 LMP energy value per hour and for each bus.

Figure 4.22 Typical annual LMP loss diagram for a certain set of transmissio...

Figure 4.23 Typical annual LMP congest.

Figure 4.24 Wind forecasting proposed methodology.

Figure 4.25 Wind speed forecasting – simulations 1 and 2 (Table 4.7).

Figure 4.26 Wind speed forecasting – simulation 5 (Table 4.7).

Figure 4.27 Wind speed forecasting – simulations 5, 6, and 7 (Table 4.7)....

Chapter 5

Figure 5.1 Synchrophasor Technology vs. SCADA data.

Figure 5.2 Synchrophasor data flow from phase conductor to phasor applicatio...

Figure 5.3 Synchrophasor data snapshot with anomalies.

Figure 5.4 Possible causes for synchrophasor data anomalies.

Figure 5.5 Data drift in PMU measurement.

Figure 5.6 PMUNET active leaning framework for ADCL.

Figure 5.7 Data flow architecture for anomaly detection.

Figure 5.8 Data flow architecture for event detection, classification, and l...

Figure 5.9 Cyber‐power test‐bed architecture.

Figure 5.10 IEEE‐14 bus system test cases using RTDS.

Chapter 6

Figure 6.1 Proposed methodology.

Figure 6.2 Discriminated consumption for the active community: DR flexibilit...

Figure 6.3 Optimal scheduling results: state of charge.

Figure 6.4 Average silhouette method results for the dataset selected – ESS ...

Figure 6.5 Elbow method results for the dataset selected – ESS status.

Figure 6.6 Gap statistic method results for the dataset selected – ESS statu...

Figure 6.7 Partitional clustering:

k

‐means results for the dataset selected ...

Figure 6.8 Partitional clustering: PAM results for the dataset selected for ...

Figure 6.9 Fuzzy clustering:

c

‐means results for the dataset selected for (a...

Figure 6.10 Hierarchical clustering: agglomerative results for the dataset s...

Chapter 7

Figure 7.1 (a) Diagram of the proposed feature selection technique for three...

Figure 7.2 (a) Outlier detection and removal using smoothing splines. (b) Ca...

Figure 7.3 MODWPT decomposition of a non‐fraudulent consumer.

Figure 7.4 Performance evaluation

Q = (AUC + MCC)/2

...

Chapter 8

Figure 8.1 Learning process overview.

Figure 8.2 Learning network architecture.

Figure 8.3 ANN topology.

Figure 8.4 Clustering results using buyer strategies as input.

Figure 8.5 Trajectories for all the four profiles that buyers have, resulted...

Chapter 9

Figure 9.1 Global EV stock, 2010–2019.

Figure 9.2 Passenger EV sales and market share in selected countries and reg...

Chapter 10

Figure 10.1 Change of basis in PCA.

Figure 10.2 The structure of GDFM.

Figure 10.3 Wind power forecast at #29 wind farm.

Figure 10.4 The RMSE performance of forecasting models.

Figure 10.5 The MAE performance of forecasting models.

Figure 10.6 Wind farms in Texas.

Figure 10.7 The PSD of actual wind power and synthesized wind power at Papal...

Figure 10.8 Actual wind power and its common component at Papalote Creek 1(t...

Figure 10.9 Actual wind power scenarios from the Sweetwater 4A and Sweetwate...

Figure 10.10 Forecasted wind power scenarios from the Sweetwater 4A and Swee...

Figure 10.11 Actual wind power scenarios from uncorrelated wind farms, Papal...

Figure 10.12 Forecasted wind power scenarios from uncorrelated wind farms, P...

Figure 10.13 Modified IEEE 118‐bus system.

Chapter 11

Figure 11.1 Creation of virtual weather stations (L1, L2, L3, and L4) from v...

Figure 11.2 Weather estimation process.

Figure 11.3 Weather data prediction framework.

Figure 11.4 (a) Locations of physical and virtual weather stations, (b) pare...

Figure 11.5 Data estimation weight matrix.

Figure 11.6 Connectivity of locations for weather predictions in the predict...

Figure 11.7 Input selection for prediction of the testing node in Cases 1, 2...

Figure 11.8 5‐minute, 10‐minute, and 15‐minute ahead solar irradiance predic...

Figure 11.9 5‐minute, 10‐minute, and 15‐minute ahead temperature predictions...

Chapter 12

Figure 12.1 Generic decomposition‐based forecasting system.

Figure 12.2 Five‐mode wind power decomposition (excessive).

Figure 12.3 Three‐mode wind power decomposition (according to physical signa...

Figure 12.4 Long short‐term memory block.

Figure 12.5 Long short‐term memory block.

Figure 12.6 Control hierarchy.

Figure 12.7 Electricity rates applied to the dump load depending on its oper...

Figure 12.8 Regulation reserve requirements (King Island power system).

Figure 12.9 Distribution of load and net‐load variability.

Figure 12.10 Wind variability as a function of hourly production (King Islan...

Figure 12.11 Variability and uncertainty of a wind power.

Figure 12.12 Wind power forecast error as a function of average hourly produ...

Figure 12.13 The King Island IPS.

Figure 12.14 One day of an IPS operation under no forecasting (ZDO mode is n...

Figure 12.15 One day of an IPS operation: (a) under PM forecasting; (b) unde...

Chapter 13

Figure 13.1 Installation layout for twin spacer damper.

Figure 13.2 Representation of twin spacer dampers' relative movement.

Figure 13.3 Coordinates representation of twin spacer dampers' movements.

Figure 13.4 Node location vicinity in the meshed model.

Figure 13.5 Model rendering of the twin‐spacer damper.

Figure 13.6 Characteristics of load modulation (frequency range: 0–100 Hz)....

Figure 13.7 Damping characteristics (phase lag: 0°).

Figure 13.8 Damping characteristics (phase lag: 60.48°).

Figure 13.9 Damping characteristics (phase lag: 131.04°).

Figure 13.10 Damping characteristics (phase lag: 211.68°).

Figure 13.11 Damping characteristics (phase lag: 283.68°).

Figure 13.12 Damping characteristics (phase lag: 321.84°).

Figure 13.A.1 Placement layout for span length: 150–240 m.

Figure 13.A.2 Placement layout for span length: 245–340 m.

Figure 13.A.3 Placement layout for span length: 345–395 m.

Figure 13.A.4 Placement layout for span length: 400–450 m.

Figure 13.A.5 Placement layout for span length: 455–500 m.

Figure 13.A.6 Placement layout for span length: 555–650 m.

Chapter 14

Figure 14.1 Flowchart for designing power distribution network.

Figure 14.2 Digital elevation map of the site (12 × 10.4 km).

Figure 14.3 Representation of load/generating centers in a two‐dimensional s...

Figure 14.4 Single‐line diagram of 9‐bus system including generators and loa...

Figure 14.5 Section representation between generating center and Sirani load...

Figure 14.6 Section representation between source and destination.

Figure 14.7 Section representation between generating center and Sirani load...

Figure 14.8 Optimized results of shortest distance between each section usin...

Figure 14.9 Optimized results of overall shortest distance using BBO and DA ...

Figure 14.10 Shortest route for laying electrical distribution network.

Figure 14.11 Conductors with voltage per unit values at different bus number...

Figure 14.12 Diesel generating system operation over a summer and winter day...

Figure 14.13 Sensitivity analysis by variation in demand (kW), diesel fuel p...

Figure 14.14 Energy index ratio vs. cost of energy.

Chapter 15

Figure 15.1 A sample model of microgrid with generation and storage componen...

Figure 15.2 A general algorithm for microgrids optimal planning with demand ...

Figure 15.3 Applicable objective functions for optimal planning of microgrid...

Figure 15.4 Data analysis for microgrid optimal planning with DR.

Figure 15.5 Available DER and ESS components in microgrids.

Figure 15.6 Applications of demand response in microgrids.

Figure 15.7 Demand response integration in a sample microgrid.

Figure 15.8 Types of demand response strategies in microgrids.

Figure 15.9 Classification of data mining techniques.

Figure 15.10 A three‐layer ANN‐based predictor for microgrids.

Figure 15.11 A deep learning artificial neural network structure.

Figure 15.12 Pattern identification using data clustering methods.

Figure 15.13 A general procedure of fuzzy logic design for DR application.

Chapter 16

Figure 16.1 Energy use by category in Uganda.

Figure 16.2 Fixed dome digester: CAMARTEC Model.

Figure 16.3 Solar cooker technologies. (a) Focusing type cooker (b) box sola...

Figure 16.4 Map for capital income and resources Uganda.

Figure 16.5 Statistics of charcoal and firewood in terms of the selected fun...

Figure 16.6 Statistics of energy security, time‐saving, health benefit, envi...

Figure 16.7 Statics of modern energy technology and development prospects. *...

Figure 16.8 Source media for awareness in modern energy technologies. * Sour...

Figure 16.9 Statistics for modern energy technologies and their promotion. *...

Chapter 17

Figure 17.1 Data preprocessing.

Figure 17.2 Window length for (a) refrigerator, (b) kitchen outlet.

Figure 17.3 (a) IOR1, (b) IOR2, (c) IOR3, and (d) IOR4.

Figure 17.4 Parallel bidirectional long short‐term memory model.

Figure 17.5 (a) Difference of power along aggregate power that consists of K...

Figure 17.6 CNN structure.

Figure 17.7 Structure diagram of (a) Long short‐term memory (LSTM), (b) bidi...

Figure 17.8 Ground truth and disaggregate power of refrigerator (FR) (a) DCN...

Figure 17.9 Ground truth and disaggregate power of kitchen outlet (KO) (a) D...

Figure 17.10 Ground truth and disaggregate power of dishwasher (DW) (a) DCNN...

Figure 17.11 Ground truth and disaggregate power of microwave (MW) (a) DCNN,...

Figure 17.12 Ground truth and disaggregate power of lighting (LT) (a) DCNN, ...

Figure 17.13 Estimated aggregate power (a) Ground truth, (b) DCNN, (c) GLU‐R...

Figure 17.14 Average of disaggregation performances (a) PTECA, (b) MAE, (c) ...

Chapter 18

Figure 18.1 Typical reinforcement learning feedback loop.

Figure 18.2 Taxonomy of reinforcement learning algorithms..

Figure 18.3 Taxonomy of model‐free reinforcement learning algorithms..

A

...

Figure 18.4 Multi‐agent reinforcement learning feedback loop.

Chapter 19

Figure 19.1 Conceptual architecture of FL.

Figure 19.2 Confusion matrix for classification approaches and common perfor...

Figure 19.3 Architecture of federated deep learning model consists of a serv...

Chapter 20

Figure 20.1 Wavelet‐based PMU anomaly detection and classification framework...

Figure 20.2 An MRA example using PMU1 frequency attribute. The detected even...

Figure 20.3 An example of abnormal event occurred across all units for (a) f...

Figure 20.4 PCA biplots of detected events using different PMU attributes. T...

Figure 20.5 Flowchart of using PMU data for fault localization and classific...

Figure 20.6 ACF for two randomly selected hours on the same day with differe...

Figure 20.7 Continuity – correlation ranges of PMU frequency in the left pan...

Figure 20.8 The wavelet spectra of PMU attributes at unit #5. (a) the decomp...

Figure 20.9 The diurnal rhythm of SNR obtained from frequency and voltage PM...

Figure 20.10 Block‐wise PCA. The subplots include (a) the rough spatial loca...

Figure 20.11 PMU angle difference time series during several adjacent days a...

Figure 20.12 Zonal model of Polish system.

Figure 20.13 The architecture of CNN model.

Figure 20.14 Zonal CNN model performance confusion matrix using time‐domain ...

Figure 20.15 Zonal CNN model prediction for each zone using the time‐domain‐...

Figure 20.16 Model accuracy comparison from different encoding approaches.

Figure 20.17 The transmission line in the BPA service area [23].

Figure 20.18 Frequency of major power outages (bar chart) and percentage amo...

Figure 20.19 Impacts of weather extreme factors for each transmission line t...

Figure 20.20 Boxplots of outage durations clustered by the extreme weather e...

Figure 20.21 Conceptual illustration of estimating components failure probab...

Guide

Cover

Table of Contents

Title Page

Copyright

Dedication

About the Editors

List of Contributors

Foreword

Begin Reading

Conclusions

Index

End User License Agreement

Pages

ii

iii

iv

v

xix

xx

xxi

xxii

xxiii

xxiv

xxv

xxvii

1

2

3

5

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

69

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

257

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

343

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

453

454

455

456

457

458

459

460

461

IEEE Press

445 Hoes Lane Piscataway, NJ 08854

IEEE Press Editorial Board

Sarah Spurgeon,

Editor in Chief

Jón Atli Benediktsson

Andreas Molisch

Diomidis Spinellis

Anjan Bose

Saeid Nahavandi

Ahmet Murat Tekalp

Adam Drobot

Jeffrey Reed

Peter (Yong) Lia

Thomas Robertazzi

Intelligent Data Mining and Analysis in Power and Energy Systems

Models and Applications for Smarter Efficient Power Systems

Edited by

Zita ValeGECAD Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development (GECAD) Polytechnic Institute of Porto (ISEP/IPP) Porto, Portugal

Tiago PintoGECAD Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development (GECAD) Polytechnic Institute of Porto (ISEP/IPP) Porto, Portugal and University of Trás‐os‐Montes e Alto Douro Vila Real, Portugal

Michael NegnevitskySchool of Engineering, University of Tasmania Hobart, Tasmania, Australia

Ganesh Kumar VenayagamoorthyHolcombe Department of Electrical and Computer Engineering, Real‐Time Power and Intelligent Systems Laboratory Clemson University Clemson, SC, USA and School of Engineering University of KwaZulu‐Natal Durban, South Africa

Copyright © 2023 by The Institute of Electrical and Electronics Engineers, Inc.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey. All rights reserved.Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per‐copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750‐8400, fax (978) 750‐4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748‐6011, fax (201) 748‐6008, or online at http://www.wiley.com/go/permission.

Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762‐2974, outside the United States at (317) 572‐3993 or fax (317) 572‐4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging‐in‐Publication Data

Names: Vale, Zita, editor. | Pinto, Tiago, PhD, editor. | Negnevitsky, Michael, editor. | Venayagamoorthy, Ganesh Kumar, editor.Title: Intelligent data mining and analysis in power and energy systems : models and applications for smarter efficient power systems / edited by Zita Vale, Tiago Pinto, Michael Negnevitsky, Ganesh Kumar Venayagamoorthy.Description: Hoboken, New Jersey : Wiley‐IEEE Press, [2023] | Series: IEEE press series on power and energy systemsIdentifiers: LCCN 2022043311 (print) | LCCN 2022043312 (ebook) | ISBN 9781119834021 (cloth) | ISBN 9781119834038 (adobe pdf) | ISBN 9781119834045 (epub)Subjects: LCSH: Electric power systems. | Data mining.Classification: LCC TK1001 .I577 2023 (print) | LCC TK1001 (ebook) | DDC 621.31–dc23/eng/20220909LC record available at https://lccn.loc.gov/2022043311LC ebook record available at https://lccn.loc.gov/2022043312

Cover Design: WileyCover Image: © metamorworks/Shutterstock

“To our dear Parents and Children”

About the Editors

Zita Vale, IEEE Senior Member, graduated in electrical engineering in 1986, received the PhD degree in electrical and computer engineering in 1993, and the Agregação title (Habilitation) in 2003 from the University of Porto, Portugal. She is a full professor in the School of Engineering, Polytechnic of Porto. She leads the research activities on Intelligent Power and Energy Systems at GECAD – Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development. She has been involved in more than 60 R&D projects and published more than 200 papers in international scientific journals. Her scientific research activities mainly focus on Power and Energy Systems Operation, Electricity Markets, Demand Response, Renewables, Electric Vehicles, and Distributed Generation and Storage. She has been developing artificial‐intelligence‐based models, methods, and applications for power and energy, using agents and multiagent systems, knowledge‐based systems, semantics, machine learning, data mining, and evolutionary computation.

She actively participates in several technical working groups and committees. She is the chair of the IEEE PES Intelligent Data Analysis and Mining (IDMA) Working Group and of the Open Data Sets (ODS) Task Force. She is the chair of the board of directors of ISAP – Intelligent Systems Application to Power Systems. She is involved in editing activities for different journals and books and is a regular reviewer and evaluator for papers and for project proposals and monitoring from different funding agencies around the world.

Tiago Pinto is an assistant professor at the Universidade de Trás os Montes e Alto Douro (UTAD), Portugal, and researcher at INESC‐TEC. He has concluded the BSc and MSc, both at the School of Engineering of the Polytechnic of Porto, where he has also developed his research work for more than 10 years, namely at GECAD – Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development. He got the PhD from UTAD in 2016, after which he engaged in a postdoc at the University of Salamanca, in collaboration with the University of Oxford and ENGIE. His main research interests focus on artificial intelligence and its application in power and energy systems, particularly machine learning, multi‐agent systems, decision support systems, and metaheuristic optimization. Through the involvement in more than 30 research projects in these fields, he has authored over 200 publications in international journals and conferences, and has co‐edited several books and special issues in journals related to power and energy systems and artificial intelligence.

Michael Negnevitsky received the BE (Hons) in Electrical Engineering degree and PhD degree from Byelorussian University of Technology, Minsk, Belarus, in 1978 and 1983, respectively. From 1978 to 1980, he worked at the Electrical Maintenance, Construction and Commissioning Company, and from 1984 to 1991, he was Senior Research Fellow at the Department of Electrical Engineering, Byelorussian University of Technology, Minsk. After his arrival to Australia, he worked at Monash University, Melbourne, and then the University of Tasmania. Currently he is Professor and Chair in Power Engineering and Computational Intelligence, and Director of the Centre for Renewable Energy and Power Systems.

His research interests include power system analysis and control, micro‐grids with distributed and renewable energy resources, smart grids, power quality and applications of artificial intelligence in power systems. He has published more than 450 papers in high‐quality journals and refereed conference proceedings, authored 14 chapters in several books, and received 4 patents for inventions. His book Artificial Intelligence (Addison Wesley 2002, 2005, 2011) has been translated into Mandarin, Cantonese, Korean, Greek, and Vietnamese and adopted in many universities around the world.

He is Fellow of Engineers Australia, Fellow of the Japan Society for the Promotion of Science, Member of CIGRE AP C4 (System Technical Performance) and AP C6 (Distribution Systems and Dispersed Generation), Australian Technical Committee. Dr. Negnevitsky has served as Secretary and Deputy Chair of IEEE PES Energy Development and Power Generation Committee, Chair of IEEE PES International Practices Subcommittee, and Chair of the IEEE PES Working Group on High Renewable Energy Penetration in Remote and Isolated Power Systems. In 2018, he received the Joint Australasian Committee for Power Engineering and CIGRE Australia Award for “outstanding career‐long contributions to research and teaching in electric power engineering as well as contribution to industry and CIGRE activities.”

Ganesh Kumar Venayagamoorthy is the Duke Energy Distinguished Professor of Power Engineering and Professor of Electrical and Computer Engineering at Clemson University since January 2012. Prior to that, he was a Professor of Electrical and Computer Engineering at the Missouri University of Science and Technology (Missouri S&T), Rolla, USA, where he was from 2002 to 2011. Dr. Venayagamoorthy was a Senior Lecturer in Department of Electronic Engineering, Durban University of Technology, Durban, South Africa, where he was from 1996 to 2002. Dr. Venayagamoorthy is the Founder and Director of the Real‐Time Power and Intelligent Systems Laboratory at Missouri S&T and Clemson University.

Dr. Venayagamoorthy received his PhD and MScEng degrees in Electrical Engineering from the University of Natal, Durban, South Africa, in April 2002 and April 1999, respectively. He received his BEng degree with a First‐Class Honors in Electrical and Electronics Engineering from Abubakar Tafawa Balewa University, Bauchi, Nigeria, in March 1994. He holds an MBA degree in Entrepreneurship and Innovation from Clemson University, SC (August 2016).

Dr. Venayagamoorthy's interests are in research, development and innovation of power systems, smart grid, and artificial intelligence technologies. Dr. Venayagamoorthy is a Fellow of the IEEE, IET (UK), the South African Institute of Electrical Engineers (SAIEE) and Asia‐Pacific Artificial Intelligence Association (AAIA), and a Senior Member of the International Neural Network Society.

List of Contributors

A. Ahmed

Smart Grid Demonstration and Research Investigation Lab

Washington State University

Pullman, WA

USA

Hirohisa Aki

Faculty of Engineering, Information and Systems

University of Tsukuba

Tsukuba, Ibaraki

Japan

Philipp Andelfinger

Institute for Visual and Analytic Computing

University of Rostock

Rostock

Germany

Victor Andrean

Department of Electrical Engineering

National Taiwan University of Science and Technology

Taipei

Taiwan

Ramón Aranda

Tepic Technology Transfer Unit, Center for Scientific Research and Higher Education of Ensenada

Tepic, Nayarit, Mexico

and

National Council of Science and Technology

Mexico City, Mexico City

Mexico

Nelson F. Avila

Independent Electricity System Operator

Toronto, ON

Canada

Wenlei Bai

Oracle Energy and Water

Austin, TX

USA

Rúben Barreto

Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development (GECAD) Institute of Engineering Polytechnic of Porto

Porto

Portugal

Miguel A. Carmona

Tepic Technology Transfer Unit, Center for Scientific Research and Higher Education of Ensenada

Tepic

Nayarit

Mexico

and

National Council of Science and Technology

Mexico City

Mexico City

Mexico

Rajeev K. Chauhan

Department of Electrical Engineering, Faculty of Engineering

Dayalbagh Educational Institute

Agra, Uttar Pradesh

India

Chia‐Chi Chu

Department of Electrical Engineering

National Tsing Hua University

HsinChu

Taiwan, R.O.C.

Juan M. Corchado

BISITE research group

University of Salamanca

Salamanca

Spain

Pavel Etingov

Pacific Northwest National Laboratory

Richland, WA

USA

Pedro Faria

GECAD – Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development

Porto

Portugal

Gerardo Figueroa

Sentiance NV

Antwerpen

Belgium

Zahra Forouzandeh

GECAD – Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development, Polytechnic of Porto, School of Engineering (ISEP)

Porto

Portugal

Reza Ghorbani

Renewable Energy Design Laboratory (REDLab), Department of Mechanical Engineering

University of Hawaii at Manoa

Honolulu, HI

USA

James Hamilton

School of Engineering, University of Tasmania

Hobart

Tasmania

Australia

Jens B. Holm‐Nielsen

Department of Energy Technology

Center for Bioenergy and Green Engineering Aalborg University

Esbjerg

Denmark

Zhangshuan Hou

Pacific Northwest National Laboratory

Richland, WA

USA

Qiuhua Huang

Pacific Northwest National Laboratory

Richland, WA

USA

Xinda Ke

Pacific Northwest National Laboratory

Richland, WA

USA

Irfan Khan

Supreme & Co. Pvt. Ltd.

Kolkata, West Bengal

India

Rahmat Khezri

College of Science and Engineering

Flinders University

Adelaide, SA

Australia

Olivera Kotevska

Computer Science and Mathematics

Oak Ridge National Laboratory

Tennessee

Oak Ridge

USA

Duehee Lee

Electrical Engineering Department

Konkuk University

Seoul

Korea

Kwang Y. Lee

Electrical and Computer Engineering Department

Baylor University

Waco, TX

USA

Fernando Lezama

Polytechnic of Porto (ISEP/IPP), Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development (GECAD)

Porto

Portugal

Kuo‐Lung Lian

Department of Electrical Engineering

National Taiwan University of Science and Technology

Taipei

Taiwan

Wen‐Kai Lu

Department of Information Management

National Taiwan University

Taipei

Taiwan, R.O.C.

Amin Mahmoudi

College of Science and Engineering, Flinders University

Adelaide, SA

Australia

Achora P.O. Mamur

Faculty of Sociology, Environmental and Business Economics

University of Southern Denmark

Esbjerg

Denmark

Samson Masebinu

Department of Energy Technology, Center for Bioenergy and Green Engineering

Aalborg University

Esbjerg

Denmark

Hamed Moayyed

GECAD – Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development, Polytechnic of Porto (P.PORTO)

Porto

Portugal

Behnam Mohammadi‐Ivatloo

Faculty of Electrical and Computer Engineering

University of Tabriz

Tabriz

Iran

Arash Moradzadeh

Faculty of Electrical and Computer Engineering

University of Tabriz

Tabriz

Iran

Bruno Mota

GECAD Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development (GECAD)

Polytechnic Institute of Porto (ISEP/IPP)

Porto

Portugal

Sushri Mukherjee

Indian Institute of Technology Delhi

Hauz Khas

New Delhi

India

Michael Negnevitsky

School of Engineering, University of Tasmania

Hobart

Tasmania

Australia

Angel D. Pacheco

Tepic Technology Transfer Unit, Center for Scientific Research and Higher Education of Ensenada

Tepic

Nayarit

Mexico

S. Pandey

Smart Grid and Technology

ComEd

Oakbrook Terrace, IL

USA

Chirath Pathiravasam

Holcombe Department of Electrical and Computer Engineering, Real‐Time Power and Intelligent Systems Laboratory

Clemson University

Clemson, SC

USA

and

Department of Electrical Engineering

University of Moratuwa

Katubedda

Sri Lanka

Tiago Pinto

GECAD Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development (GECAD)

Polytechnic Institute of Porto (ISEP/IPP)

Porto

Portugal

and

University of Trás‐os‐Montes e Alto Douro

Vila Real

Portugal

Dharmbir Prasad

Asansol Engineering College

Asansol, West Bengal

India

Carlos Ramos

GECAD Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development (GECAD)

Polytechnic Institute of Porto (ISEP/IPP)

Porto

Portugal

Sérgio Ramos

GECAD – Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development, Polytechnic of Porto, School of Engineering (ISEP)

Porto

Portugal

Huiying Ren

Pacific Northwest National Laboratory

Richland, WA

USA

Ansel Y. Rodríguez González

Tepic Technology Transfer Unit, Center for Scientific Research and Higher Education of Ensenada

Tepic

Nayarit

Mexico

and

National Council of Science and Technology

Mexico City

Mexico City

Mexico

Sajan K. Sadanandan

Smart Grid Integration

R&D Center, Dubai Electricity & Water Authority (DEWA)

Dubai

UAE

Evgenii Semshikov

School of Engineering, University of Tasmania

Hobart, Tasmania

Australia

Mahendra P. Sharma

Department of Hydro and Renewable Energy

Indian Institute of Technology

Roorkee, Uttarakhand

India

Cátia Silva

GECAD – Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development

Porto

Portugal

Rudra P. Singh

Asansol Engineering College

Asansol, West Bengal

India

João Soares

GECAD – Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development, Polytechnic of Porto, School of Engineering (ISEP)

Porto

Portugal

Anurag K. Srivastava

Smart Grid Resiliency and Analytics Lab

West Virginia University

Morgantown, WV

USA

Subho Upadhyay

Department of Electrical Engineering, Faculty of Engineering

Dayalbagh Educational Institute

Agra, Uttar Pradesh

India

Zita Vale

GECAD Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development (GECAD)

Polytechnic Institute of Porto (ISEP/IPP)

Porto

Portugal

Ganesh Kumar Venayagamoorthy

Holcombe Department of Electrical and Computer Engineering, Real‐Time Power and Intelligent Systems Laboratory

Clemson University

Clemson, SC

USA

and

School of Engineering

University of KwaZulu‐Natal

Durban

South Africa

Xiaolin Wang

School of Engineering, University of Tasmania

Hobart, Tasmania

Australia

Foreword

Recent machine learning and data analytics methods have proliferated into most areas of science, engineering, and commerce. There are excellent reasons for their increasing popularity and applications. Many real‐world problems are too complex to come up with closed‐form analytical solutions. However, such challenges did not make practitioners idle; instead, they have created working models, prototypes and even built systems with a careful understanding of critical components of the systems as a first step. The data generated from such systems are then analyzed by machine learning and data analytics methods to have a more comprehensive understanding of the systems.

This book titled Intelligent Data Mining and Analysis in Power and Energy Systems makes a huge leap in this direction in providing a better understanding of power and energy systems. Compiled by Zita Vale, Tiago Pinto, Michael Negnevitsky, and Ganesh Kumar Venayagamoorthy, the book begins with an introduction to machine learning and data analytics methods and then lays out state‐of‐the‐art methods in addressing various topics in power and energy system design, clustering, classification, forecasting, and analysis with latest machine learning and data analytics methods.

The book is self‐contained and written for both novice and experts on the topic. The topics are discussed in simple manner with adequate references and details, so that readers can understand the current state‐of‐the‐art and also find relevant past studies in a single volume.

If you are working in power and energy systems either as a researcher or a practitioner, this is a must‐have book to stay ahead in the game. Authors are experts in their own fields. The book will save your efforts in searching for materials on the topic, provide you with the latest methodologies, and direct you to other similar past studies.

Kudos to the editors for this compilation and authors for their contributions.

Kalyanmoy DebUniversity Distinguished ProfessorWithrow Senior Distinguished Research ScholarKoenig Endowed Chair ProfessorIEEE CIS Evolutionary Computation PioneerIEEE Fellow

Department of Electrical and Computer EngineeringMichigan State University, East Lansing, MI, USA

Introduction

In an era of ever‐increasing data, there is also an increasing need for the development of suitable intelligent data mining and analysis solutions that enable taking the value out of these data. Fostered by this increasing need and also boosted by the recent worldwide boom in artificial intelligence interest and development, we have been witnessing a significant development of a wide array of new advanced data mining models and methods. These models and methodologies have been instrumental in dealing with real problems, especially in highly complex domains such as power and energy systems [1].

The challenges in power and energy systems have changed completely during the past years, especially because of the increase in the distributed renewable energy sources and the consequently required transformations in power systems' operation, management, and planning, and also in electricity markets [2]. New players are emerging, such as prosumers, electric vehicles, new types of aggregators, energy communities, new local market operators, energy managers of different kinds, among many others [3, 4]. Consequently, new business models are also being proposed, experimented, and implemented as the way to involve such new players in the sector in an active way while creating a new value for these players and for the system, e.g. through the enhancement of the use of local generation and the fostering of consumption and generation flexibility trading [5].

Such significant changes in a traditionally conservative sector require an unprecedented adaptation and foresight capacity from the entire energy value chain, including from policy makers, regulators, operators, planners, and even from the smaller players. This is where the role of new and intelligent data mining and analysis models and methodologies become crucial, contributing to overcoming multiple problems with distinct characteristics. Some relevant examples are power system planning, state estimation, energy resource profiling, aggregation and forecasting, market negotiation, and energy management at multiple levels, including building, microgrid, smart grid, energy community, and distribution grid levels.

This book provides a comprehensive review of intelligent data mining and analysis applications in power and energy systems. This book is organized in six complementary parts, each focusing on a specific topic within the data mining and analysis domain, namely, data mining and analysis fundamentals, clustering, classification, forecasting, data analysis, and other machine learning applications. Each of the six parts is briefly described as follows:

Part I

:

Data mining and analysis fundamentals provide an overview on data mining and analysis foundations as a means to introduce the reader to the main concepts of the domain and facilitate the deeper understanding of the works described in the rest of the book. Besides the main concepts behind data mining and analysis, the first chapter is dedicated to highlighting the importance of data pre‐processing and feature engineering as a means to enable a suitable application of the state‐of‐the‐art models dedicated to the diverse traditional problems related to data mining. This introductory overview provides the means for a deeper understanding of data mining and analysis, bridging the reader into the power and energy systems application domain through the presentation of two systematic reviews, namely, on data mining and analysis applications in power and energy systems and on the contributions of deep learning in power system problems. These two chapters address different problems within power and energy systems, which benefit from the advances of data mining models related to clustering, classification, forecasting, and other common approaches.

Part II

:

Clustering presents a description of works that apply clustering models and methods to address power and energy system problems. These include standard clustering approaches, as well as the combination of clustering with other models, e.g. classification‐based, to solve different types of problems. Specifically, the power and energy system problems addressed by this part include consumer‐directed problems related to consumer clustering and demand profiling. Aggregation problems considering not only the aggregation of consumers but also of their consumption flexibility are explored as a means to solve demand response challenges in wider scales. Synchrophasor data analytics taking advantage on clustering models and their combination with other approaches are also addressed, aiming at anomaly detection, localization, and classification.

Part III

:

Classification includes the description of works that apply classification models such as artificial neural networks, support vector machines, K‐nearest neighbors, among others, to solve problems using labeled data. The application cases of these works are related to non‐technical loss detection in electric distribution systems, electrical vehicle integration in the power system under multiple worldwide perspectives and considering different types of technologies, and electricity market participation and decision support, namely, in the scope of bilateral contract negotiations using historic and overserved data from multiple negotiators' negotiation process.

Part IV

:

Forecasting is devoted to the description of works related to the forecasting of energy resources with distinct characteristics. These works are mainly focused on the forecasting of highly variable renewable energy sources; besides, the application of traditional regression‐based algorithms describes the advantages of specific approaches such as multivariate stochastic models, spatiotemporal models, and decomposition‐based models. The forecasting models are applied to solar irradiance and temperature estimation and to wind and solar power forecasting under distinct scenarios regarding historical data, power system characteristics, and overall renewable energy penetration.

Part V

:

Data analysis presents the application of data analysis models of distinct natures to address different types of power system‐related problems. These problems concern issues such as the vibration of transmission line conductors through the analysis of harmonic dynamic response and the design of power distribution network in hilly areas with the purpose of enabling off‐grid electrification. The application of intelligent demand response models as part of microgrid planning is another of the addressed problems. This part is finalized with a chapter focusing on socioeconomic analysis of renewable energy interventions toward affordable and sustainable household technologies.

Part VI

:

Other machine learning applications describe applications that use distinct types of machine learning approaches such as reinforcement learning, federated learning, and probabilistic modeling, addressing a varied set of challenges of natures. Such challenges include the state estimation of power electronic converters, using both white box and black box approaches. The problem of intelligent building energy management and control is addressed using reinforcement learning. Federated deep learning is applied to generate global supermodels for power system data analysis, and risk assessment of power system outages is performed through probabilistic modeling, considering weather and climate extremes.

Overall, this book comprises the description of a wide set of intelligent data mining and analysis models, methodologies, and applications, addressing problems of distinct natures within the field of power and energy systems, while highlighting the advantages of the already achieved breakthroughs in the domain and pointing out the main gaps that have not yet been solved, as pointers for future paths of continuous research and development.

    Zita Vale

    Polytechnic of Porto, Portugal

    Tiago Pinto

    Polytechnic of Porto, Portugal

    University of Trás‐os‐Montes e Alto Douro

    Portugal

    Michael Negnevitsky

    University of Tasmania, Australia

    Ganesh Kumar Venayagamoorthy

    Clemson University, USA

References

1

Ibrahim, M.S., Dong, W., and Yang, Q. (2020). Machine learning driven smart electric power systems: current trends and new perspectives.

Applied Energy

272: 115237.

2

Pinto, T., Vale, Z., Widergren, S., and editors. (2021).

Local Electricity Markets

, 1e. Academic Press 384 pp.

https://www.elsevier.com/books/local-electricity-markets/pinto/978-0-12-820074-2

.

3

Koirala, B.P., Koliou, E., Friege, J. et al. (2016). Energetic communities for community energy: a review of key issues and trends shaping integrated community energy systems.

Renewable and Sustainable Energy Reviews

56: 722–744.

4

de São, J.D., Faria, P., and Vale, Z. (2021). Smart energy community: a systematic review with metanalysis.

Energy Strategy Reviews

36: 100678.

5

Hall, S. and Roelich, K. (2016). Business model innovation in electricity supply markets: the role of complex value in the United Kingdom.

Energy Policy

92: 286–298.

Part IData Mining and Analysis Fundamentals

1Foundations

Ansel Y. Rodríguez‐González1, Angel Díaz‐Pacheco2, Ramón Aranda3, and Miguel Á. Álvarez‐Carmona4

1Unidad de Transferencia Tecnológica Tepic, Centro de Investigación Científica y de Educación Superior de Ensenada, Nayarit, México

2Departamento de Ingeniería en Electrónica, Campus Irapuato‐Salamanca, Universidad de Guanajuato, Guanajuato, México

3Unidad Mérida, Centro de Investigación en Matemáticas, Yucatán, México

4Unidad Monterrey, Centro de Investigación en Matemáticas, Nuevo León, México

Acronyms

ANN

artificial neural networks

IoT

Internet of Things

KDD

knowledge discovery in databases

KNN

k‐nearest neighbors

PCA

principal component analysis

SVM

support vector machine

SVR

support vector regression

tSNE

T‐distributed stochastic neighbor embedding

TWD

three‐way decision

1.1 Data Mining: Why and What?

Storing data and analyzing it to make better decisions are a process that humanity has performed at least since the creation of mathematics for commerce, particularly double‐entry book‐keeping, initially known in the Renaissance as book‐keeping “alla veneziana” [1]. This accounting system consisted of recording transactions using a general memorandum and a second, more detailed, and organized record. Additionally, transactions were recorded twice, on the one hand, the ledgers, on the other debtors. Since each income must be balanced with a counterpart, the system allows to find errors, understand where the costs and profits come from, and thus guide decision‐making.

With the development of larger and more complex business relationships stemming from the first and second industrial revolutions, the need for more powerful analysis tools arose. Thus, econometrics emerged, a branch of economics that uses mathematical and statistical models to analyze, interpret and make predictions about economic systems, predict variables, and find relationships between them and trends [2]. In general, econometrics is based on the construction of formal models with which it is possible to verify hypotheses, measure statistical variables, and carry out simulation tests.

The computing revolution made it inexpensive to carry out multiple (many) hypothesis tests, and, as a consequence, the search for the model that best fits the data was encouraged. To describe this process, terms such as data mining, data dredging, data snooping, and data fishing emerged [3, 4]. Additionally, the term data miner was coined to name the researchers that given a set of data, fit alternative equations since there are alternative subsets of possible explanatory variables and chose the best equation. Also, the term data miner was used to differentiate them from classical statistics researchers [5].

However, it is possible to find a model that fits a data set well, even if it is false (i.e. a model obtained from completely random data). This situation complicates the interpretation of the test results of hypotheses (significance levels) [6]