Surrogate Modeling and Optimization E-Book

Nam-Ho Kim

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 920

Veröffentlichungsjahr: 2026

Beschreibung

An expert reference on building surrogate models, using them for optimization, their associated prediction uncertainty, and potential failures, with practical implementation in MATLAB

Surrogate Modeling and Optimization explains the meaning of different surrogate models and provides an in-depth understanding of such surrogates, emphasizing how much uncertainty is associated with them, and when and how a surrogate model can fail in approximating complex functions, helping readers understand theory through practical implementation in MATLAB. This book enables readers to obtain an accurate approximate function using as few samples as possible, thereby allowing them to replace expensive computer simulations and experiments during design optimization, sensitivity analysis, and/or uncertainty quantification.

The book is organized into three parts. Part I introduces the basics of surrogate modeling. Part II reviews various theories and algorithms of design optimization. Part III presents advanced topics in surrogate modeling, including the Kriging surrogate, neural network models, multi-fidelity surrogates, and efficient global optimization using Kriging surrogates.

Each chapter contains a multitude of examples and exercise problems. Lecture slides and a solution manual for exercise problems are available for instructors on a companion website.

Topics discussed in Surrogate Modeling and Optimization include:

• Various designs of experiments, such as those developed for linear and quadratic polynomial response surfaces (PRS) in a boxlike design space
• Criteria for constrained and unconstrained optimization and the most important optimization theories
• Various numerical algorithms for gradient-based optimization
• Gradient-free optimization algorithms, often referred to as global search algorithms, which do not require gradient or Hessian information
• Detailed explanations and implementation on Kriging surrogates, often referred to as Gaussian Process, especially when samples include noise
• The combination of a small number of high-fidelity samples with many low-fidelity samples to improve prediction accuracy
• Neural network models, focusing on training uncertainty and its effect on prediction uncertainty
• Efficient global optimization using either polynomial response surfaces or Kriging surrogates

Surrogate Modeling and Optimization is an essential learning companion for senior-level undergraduate and graduate students in all engineering disciplines, including mechanical, aerospace, civil, biomedical, and electrical engineering. The book is also valuable for industrial practitioners who apply surrogate models to solve their optimization problems.

Leseprobe

Surrogate Modeling and Optimization

Theories, Applications, and Limitations

Nam‐Ho Kim

University of FloridaGainesville, Florida, USA

Copyright © 2026 by John Wiley & Sons, Inc. All rights reserved, including rights for text and data mining and training of artificial intelligence technologies or similar technologies.

Published by JohnWiley & Sons, Inc., Hoboken, New Jersey.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per‐copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750–8400, fax [phone], or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748–6011, fax [phone], or online at http://www.wiley.com/go/permission.

The manufacturer’s authorized representative according to the EU General Product Safety Regulation is Wiley‐VCH GmbH, Boschstr. 12, 69 469 Weinheim, Germany, e‐mail: [email protected].

Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.

Limit of Liability/Disclaimer of Warranty: While the publisher and the authors have used their best efforts in preparing this work, including a review of the content of the work, neither the publisher nor the authors make any representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762–2974, outside the United States at (317) 572–3993 or fax [phone].

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information aboutWiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging‐in‐Publication Data Applied for:

Hardback ISBN: 9781394245819

Cover Design: WileyCover Image: © agsandrew/Getty Images

Dedicated to Professor Raphael “Rafi” Tuvia Haftka, a great mentor, colleague, and friend

Preface

Surrogate modeling is a powerful engineering tool to approximate a quantity of interest (QoI) as a function of input variables. The QoI can be an outcome of computer simulation or experimental measurement, which may or may not include noise. Input variables can be any parameter that affects the QoI, but in this book, they represent design variables that can be changed during optimization. The functional relationship between the QoI and input variables is unknown in many engineering applications because it is often governed by integro‐differential equations. Computer simulation and experimental measurement can provide the value of QoI for selected input variables, but the behavior of the QoI over the entire input space is generally unknown. Surrogate modeling approximates this complex relationship in the form of simple analytical/statistical functions using a set of samples, which were obtained by computer simulations or experiments at the different values of input variables. The main goal of surrogate modeling is to obtain an accurate approximate function using a minimum number of samples. Accuracy in this context means the prediction capability of a surrogate model at unsampled locations, not the error between samples and surrogate predictions. The main use of a surrogate model is to replace expensive computer simulations and experiments during design optimization, sensitivity analysis, and/or uncertainty quantification.

There are many books on surrogate modeling and optimization in the market. Many of them focus on how to build surrogate models and how to use them for optimization. There are also many commercial and public‐domain programs for surrogate modeling. However, in reality, it requires significant technical expertise to use surrogate models properly. This is partly because building surrogate models is one thing and understanding them is another. An important goal of this book is to explain the meaning of built surrogates and to provide an in‐depth understanding of such surrogates. This book emphasizes when and how a surrogate model can fail in approximating complex functions. One of the best ways to understand the theory of surrogate models is to implement it using a computer program. Throughout this book, numerous Matlab codes are provided as practice tools.

This book is organized into three parts and ten chapters. Part 1 introduces the basics of surrogate modeling. This book considers the polynomial response surface (PRS) and Kriging surrogates as two pillars of surrogate modeling, as they approach the surrogate approximation in completely different directions. Chapter 2 introduces the traditional PRS surrogate. Even if PRS is a basic surrogate model, profound theories and experiences are available in the literature. The basic assumption of PRS is that the model form is correct, but that the samples have uncorrelated noise. Because of this assumption, the fitting process of PRS is regression, where the surrogate converges to the mean of the samples’ trend. Various error measures and ways of evaluating the goodness of fit are introduced in this chapter. The statistical view of regression can provide a good contrast with the Kriging surrogate in Chapter 7. Since the quality of surrogates is significantly affected by the number and location of samples, Chapter 3 presents various designs of experiments. Many standard designs of experiments are developed for linear and quadratic PRS in box‐like design space. For general surrogates, various optimal designs are introduced, as well as space‐filling designs of experiments. The main questions of design of experiments are (i) how many samples are good enough and (ii) where to locate the samples. This chapter focuses on answering these two questions for various methods of design of experiments.

Part 2 introduces various theories and algorithms of design optimization. Chapter 4 presents engineering optimization problems and their formulations. Optimality criteria are considered the most important optimization theories. Optimality criteria for both unconstrained and constrained optimization problems are discussed. In general, numerical algorithms for solving optimization problems can be categorized into gradient‐based and gradient‐free algorithms. Chapter 5 summarizes gradient‐based optimization algorithms. Most algorithms require determining a search direction and step size. In this process, the gradient and Hessian information play a critical role, which assumes that the objective and constraints are a continuous and smooth function of design variables. Chapter 6 summarizes gradient‐free optimization algorithms. They are often referred to as global optimization algorithms, but in this book, they are referred to as global search algorithms. This group of algorithms does not require gradient or Hessian information, and the objective and constraints do not need to be continuous. Some global search algorithms are deterministic, such as the Nelder‐Mead and DIRECT algorithms, while others are stochastic, such as genetic algorithms and particle swarm algorithms. But there is no algorithm that can guarantee finding the global optimum design.

Part 3 presents advanced topics in surrogate modeling, which include the Kriging surrogate, neural network models, multi‐fidelity surrogates, and efficient global optimization. Kriging surrogates as described in Chapter 7 are the second pillar of this textbook, where the basic assumption is that the model form is uncertain, but samples are accurate with distance‐dependent correlation. This yields an interpolating approximation, not regression as is the case in PRS. Although Kriging surrogate is considered one of the most advanced surrogates in many engineering fields, it is challenging to set up the surrogate model form properly and estimate its hyperparameters properly, which will be the focus of Chapter 7. Neural network models in Chapter 8 have gained a lot of popularity due to advances in computing power and deep learning algorithms. This book focuses on uncertainty associated with neural network training and predictions. In the surrogate perspective, neural network models work well in the interpolation region but perform poorly in the extrapolation region. Since obtaining many samples is expensive, multi‐fidelity surrogates in Chapter 9 have attracted a lot of attention recently. The basic idea is to combine a small number of high‐fidelity samples with many low‐fidelity samples (or low‐fidelity model itself) to improve prediction accuracy. In practice, however, multi‐fidelity surrogates tend to save computational time for obtaining high‐fidelity samples but are not able to improve prediction accuracy significantly. Chapter 10 presents an interesting approach to optimization using surrogates and adaptive sampling, which is referred to as efficient global optimization. The basic idea is to use prediction uncertainty information to find the location of an additional sample to search for the best design. The main mechanism is a smart mixture of exploring design space and improving a good design locally (exploitation).

The substances of this book come from portions of the Approximation and Optimization in Engineering course that I taught with Prof. Raphael “Rafi” Haftka for several years at the University of Florida. His unique perspectives toward surrogate modeling are refined into this book, including the viewpoint of error as uncertainty, issues relating to extrapolation in exploring design space, detailed understandings of assumptions in different surrogates, and situations in which a surrogate is more accurate than samples. A personal goal of this textbook is to leave such unique perspectives in an archived form.

Many examples and exercise problems were derived from students’ assignments when I taught this substance at the University of Florida. I am thankful to the students who took the course for their valuable suggestions, especially regarding the example and exercise problems.

March 2025

Nam‐Ho Kim

Acknowledgment

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No.RS‐2024‐00341872).

About the Companion Website

This book is accompanied by a companion website:

www.wiley.com/go/kim/surrogate_modeling 

Scan this QR code to visit the companion website

This website include:

Solution manual

PPT

Other

Part IBasics of Surrogate Modeling