Understanding and Managing Model Risk E-Book

Massimo Morini

• Herausgeber: John Wiley & Sons
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 892

Veröffentlichungsjahr: 2011

Beschreibung

A guide to the validation and risk management of quantitative models used for pricing and hedging Whereas the majority of quantitative finance books focus on mathematics and risk management books focus on regulatory aspects, this book addresses the elements missed by this literature--the risks of the models themselves. This book starts from regulatory issues, but translates them into practical suggestions to reduce the likelihood of model losses, basing model risk and validation on market experience and on a wide range of real-world examples, with a high level of detail and precise operative indications.

Leseprobe

Contents

Cover

Series

Title Page

Copyright

Preface

HOW WE PROCEED

WHAT ELSE YOU WILL FIND IN THIS BOOK

Acknowledgements

Part I: Theory and Practice of Model Risk Management

1: Understanding Model Risk

1.1 WHAT IS MODEL RISK?

1.2 FOUNDATIONS OF MODELLING AND THE REALITY OF MARKETS

1.3 ACCOUNTING FOR MODELLERS

1.4 WHAT REGULATORS SAID AFTER THE CRISIS

1.5 MODEL VALIDATION AND RISK MANAGEMENT: PRACTICAL STEPS

2: Model Validation and Model Comparison: Case Studies

2.1 THE PRACTICAL STEPS OF MODEL COMPARISON

2.2 FIRST EXAMPLE: THE MODELS

2.3 FIRST EXAMPLE: THE PAYOFF. GAP RISK IN A LEVERAGED NOTE

2.4 THE INITIAL ASSESSMENT

2.5 THE CORE RISK IN THE PRODUCT

2.6 A DEEPER ANALYSIS: MARKET CONSENSUS AND HISTORICAL EVIDENCE

2.7 BUILDING A PARAMETRIC FAMILY OF MODELS

2.8 MANAGING MODEL UNCERTAINTY: RESERVES, LIMITS, REVISIONS

2.9 MODEL COMPARISON: EXAMPLES FROM EQUITY AND RATES

3: Stress Testing and the Mistakes of the Crisis

3.1 LEARNING STRESS TEST FROM THE CRISIS

3.2 THE CREDIT MARKET AND THE ‘FORMULA THAT KILLED WALL STREET’

3.3 PORTFOLIO STRESS TESTING AND THE CORRELATION MISTAKE

3.4 PAYOFF STRESS AND THE LIQUIDITY MISTAKE

3.5 TESTING WITH HISTORICAL SCENARIOS AND THE CONCENTRATION MISTAKE

4: Preparing for Model Change. Rates and Funding in the New Era

4.1 EXPLAINING THE PUZZLE IN THE INTEREST RATES MARKET AND MODELS

4.2 RETHINKING THE VALUE OF MONEY: THE EFFECT OF LIQUIDITY IN PRICING

Part II: Snakes in the Grass: Where Model Risk Hides

5: Hedging

5.1 MODEL RISK AND HEDGING

5.2 HEDGING AND MODEL VALIDATION: WHAT IS EXPLAINED BY P&L EXPLAIN?

5.3 FROM THEORY TO PRACTICE: REAL HEDGING

6: Approximations

6.1 VALIDATE AND MONITOR THE RISK OF APPROXIMATIONS

6.2 THE SWAPTION APPROXIMATION IN THE LIBOR MARKET MODEL

6.3 APPROXIMATIONS FOR CMS AND THE SHAPE OF THE TERM STRUCTURE

6.4 TESTING APPROXIMATIONS AGAINST EXACT. DUPIRE’S IDEA

6.5 EXERCISES ON RISK IN COMPUTATIONAL METHODS

7: Extrapolations

7.1 USING THE MARKET TO COMPLETE INFORMATION: ASYMPTOTIC SMILE

7.2 USING MATHEMATICS TO COMPLETE INFORMATION: CORRELATION SKEW

8: Correlations

8.1 THE TECHNICAL DIFFICULTIES IN COMPUTING CORRELATIONS

8.2 FUNDAMENTAL ERRORS IN MODELLING CORRELATIONS

9: Calibration

9.1 CALIBRATING TO CAPS/SWAPTIONS AND PRICING BERMUDANS

9.2 THE EVOLUTION OF THE FORWARD SMILES

10: When the Payoff is Wrong

10.1 THE LINK BETWEEN MODEL ERRORS AND PAYOFF ERRORS

10.2 THE RIGHT PAYOFF AT DEFAULT: THE IMPACT OF THE CLOSEOUT CONVENTION

10.3 MATHEMATICAL ERRORS IN THE PAYOFF OF INDEX OPTIONS

11: Model Arbitrage

11.1 INTRODUCTION

11.2 CAPITAL STRUCTURE ARBITRAGE

11.3 THE CAP-SWAPTION ARBITRAGE

11.4 CONCLUSION: CAN WE USE NO-ARBITRAGE MODELS TO MAKE ARBITRAGE?

12: Appendix

12.1 RANDOM VARIABLES

12.2 STOCHASTIC PROCESSES

12.3 USEFUL RESULTS FROM QUANTITATIVE FINANCE

Bibliography

Index

For other titles in the Wiley Finance series please see www.wiley.com/finance

This edition first published 2011

© 2011 John Wiley & Sons, Ltd

Registered Office

John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom

For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com.

The rights of Massimo Morini to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher.

Wiley publishes in a variety of electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.

Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought.

Library of Congress Cataloging-in-Publication Data:

Understanding and managing model risk : a practical guide for quants, traders and validators / [edited by] Massimo Morini. – 1st ed. p. cm. – (Wiley finance series) Includes bibliographical references and index. ISBN 978-0-470-97761-3 (hardback) 1. Risk management. 2. Risk management–Mathematical models. I. Morini, Massimo. HD61.U53 2011 332.64′5–dc23 2011031397

ISBN 978-0-470-97761-3 (hbk), ISBN 978-1-119-96085-0 (ebk), ISBN 978-0-470-97774-3 (ebk), ISBN 978-0-470-97775-0 (ebk)

A catalogue record for this book is available from the British Library.

Preface

One fundamental reason for writing this book is that I do not think that models can ‘kill Wall Street’, as someone was heard to say during the credit crunch. Shortsighted policies and regulations, and bad incentives for market players, are much more likely killers (see Chapter 1 for some precise results regarding the role they can play in fuelling a crisis). And yet I am perplexed when I hear some fellow modellers deny any responsibility, saying ‘Models were not a problem. The problem was in the data and the parameters! The problem was in the application!’. As a researcher and bank quant, I find these disclaimers paradoxical. Models in finance are tools to quantify prices or risks. This includes mathematical relations, a way to use data or judgement to compute the parameters, and indications on how to apply them to practical issues. Only by taking all these things together can we talk of `a model’. Modellers should stay away from the temptation to reduce models to a set of mathematical functions that can be thought of separately from the way they are specified and from the way they are applied. If this were the case, models would really be only blank mathematical boxes and people would be right to consider them useless, when not outright dangerous.

This is not the definition of models considered in this book. I think that mathematical models are magnificent tools that can take our understanding of markets, and our capability to act in markets, to levels impossible to reach without quantitative aids. For this to be true, we must understand the interaction between mathematics and the reality of markets, data, regulations and human behaviour, and control for this in our management of model risk.

The fact that thousands of technical papers speak of very advanced models, and just a handful focus on model risk and how to manage it, is one of our problems. Too often models have been used to create a false sense of confidence rather than to improve our understanding. Increasing the complexity of the mathematical details to hide our ignorance of the underlying system is an abuse of the beauty and power of mathematics.

At the same time we have relegated model validation and risk management to become a formal and boring topic for bureaucrats. So I do not find it strange that this book has been written not by a risk manager or a validator, but by a front office quant who has spent the last ten years inventing new models, implementing them, and helping practitioners to use them for buying, selling and hedging derivatives. No one has seen how many unexpected consequences the practical use of models can have more often than a front office quant. This forces us to think of model robustness and of the effect of different calibrations or estimations of parameters. While risk managers and validators can at times afford to take a formal approach to model risk, front office quants must go deeper into the mathematical aspects of models for their implementation, and are also those who then have to deal with the most practical side of model risk.

I have also been helped by the fact that I am a researcher and trainer in the field of quantitative finance, am up-to-date with the variety of models developed by quants and enjoy the benefit of many discussions with my fellow researchers and students about the use and misuse of models. Another important element is the role I have been allowed to play in the study of the foundations of modelling at my bank, and the close collaboration with a wise and far-sighted risk management and validation group team during my last years at Intesa Sanpaolo.

In this book I have tried to avoid the two opposite extremes that I have seen too often. On one hand, training material on risk management often gives a lot of details on formal compliance or simple techniques to produce numbers that are acceptable to put in reports, but lacks the quantitative approach that would be needed to understand models deeply, and the practical examples on how real risks can arise from the use of models and hit the business of your bank or institution. Now a consensus is growing, even among regulators, that we need something different. On the other hand, many papers on financial models are weighed down with mathematics and numerics, but just a few focus on the consequences that different modelling choices can have on enterprise-wide risk and on the analysis of financial conditions and practical misuses that can lead to model losses. It is also rare to find papers that show how many alternative models are possible giving you the same good fit and efficient calibration but leading to completely different pricing and risk assessment for complex products. Before the crisis models did not play the role of allowing as transparent as possible a translation of assumptions into numbers. They have often hidden poor and oversimplified assumptions under a lot of numerical and mathematical details.

In this book you will find the rigorous mathematical foundations and the most recent developments in financial modelling, but they are analyzed taking into account the regulatory and accountancy framework, and they are explained through a wide range of practical market cases on different models and different financial products, to display where model risk hides and how it can be managed. The consequences of model assumptions when applied in the business, including explanation of model errors and misunderstandings, the comparison of different models and the analysis of model uncertainty are a focus of this book, to build up a practical guide for reducing the likelihood of model losses.

Those who like mathematics, will find as much of it as they can desire, especially in the second part of the book. But in the first part of the book there are also hundreds of pages of explanations in plain words, without formulas, that I strongly advise should not be ignored. They are sometimes the only way to think about the real purposes for which formulas are developed, and they are often the only way to explain models to many who will use them. Those who do not really like mathematics will be happy to see that in these pages all concepts are also explained without formulas. But please, do make an effort to engage with the mathematics. Here it is explained, often from the foundations, and always put in relation to practice, you may be surprised to find just how useful it can be. This also makes the book suitable for students that want to see financial models within the context of their application, and for users that have to choose different models and want to explore their hidden consequences.

Some of the mathematical complexities we have seen in models in the past decade are probably useless or even disturbing. But financial problems are seriously complex, and very often a high level of mathematical ability is really needed. I do think, however, that the high level of theoretical complexity reached by models must be balanced by a practical and not-too-complex approach to model risk management. In what follows you will find all the mathematics needed to understand models, but you will not find complex theoretical and mathematical frameworks for how to perform model risk management or validation. We want to reduce model risk, not to compound the risk of complex models with the risk of complex model validation. We keep our distance from fascinating but overcomplex frameworks that are often inapplicable and inhibit fresh thinking.

My aim is to help regulators, senior management, traders, students, and also quants themselves to a deeper understanding and awareness of the financial implications of quantitative models. Even more importantly, I want to provide quants, risk managers and validators with tools for investigating and displaying effectively the reasons for choosing one model and rejecting another, and for understanding and explaining why in many cases model uncertainty is unavoidable and models must not be used to create a false sense of confidence or as a shield for dangerous business decisions. Before the recent crisis, this analysis and this explanation failed too often and the consequences have been harsh.

In any case: if the book fails to fulfil this role, at least it has reached such a size that it can be used by quants and technical traders to stop physically any dangerous model misuse or misunderstanding. The sheer weight of its pages will force the errants to stop and think about what they are doing, without, one hopes, leaving any permanent physical consequences.

A final remark is in order. No book should even try to be a definitive work on model risk. If this were the case, we might feel entitled to stop thinking about and doubting our model tools, and a crisis worse than the one we have just seen would be forthcoming. In spite of the range of models and markets considered, this search for risks, errors and misunderstanding in the development and use of models is necessarily very partial and incomplete. But I am confident that coming with me on this quest will make you a better hunter.

One of the exercises for the reader is to spot the model risks that managed to escape the nets of this book, or survive defiantly among its pages, and propose solutions. I have even set up a website: www.managingmodelrisk.com.

HOW WE PROCEED

The book is divided into two parts. In the first, I want to build solid knowledge of the theory and the empirical evidence underlying the best practice of model risk management, constructing a practical scheme for model choice and model validation. I want the reader to not just accept each step passively, but to gain a thorough understanding of why it is useful and of how it must be applied in different situations. Since the different possible situations in financial markets are a continuous infinity of a high order, the only way to gain confidence is to explore each step deeply through market examples. I have tried in the examples to keep a practical and ‘teaching’ approach, as confirmed by the number of ‘handwritten’ figures that come from my courses for practitioners.

This book covers a wide range of asset classes. The lion's share is probably played by interest rates and credit, which is not surprising because in almost all banks model risk management has a special focus on these asset classes. The most natural examples in the first part of the book, that deals with errors in model assumptions and model application, come from credit, where these issues have emerged most often, particularly in the recent credit crunch. The second part of this book deals with more technical errors, particularly in computational methods, hedging, and mathematical techniques. Here, most of the examples come from interest rates, because it is here that the most advanced techniques were developed and applied. These two asset classes are also those that are experiencing the most changes in modelling approach now. However, equity modelling is mentioned very often throughout the book, and actually the majority of the issues dealt with in the book can have an application within complex equity models, as I often point out. We also speak of cross-currency, and liquidity and hybrid modelling have sections devoted to them.

Below is an extended summary of the contents

In Chapter 1 we want to understand what Model Risk really means in practice. To achieve this goal:

We study the foundations of quantitative pricing and their relationship with the actual workings of the markets.We see the most relevant analyses of model risk given in the literature, and we test them on the reality of the past crises, from the stock market crash of 1987 to the LTCM collapse, and the Russian default, up to the credit crunch, to see which model errors really led to large losses and how this risk could be managed.We investigate the links between the way we use models and the accounting standards, in particular the concepts of fair value, mark-to-market and levels 1, 2 and 3 for pricing.We describe the prescriptions of regulators to see which constraints they set on modelling and which indications they give on model risk management.

In Chapter 2 we consider three market examples, so as to apply the scheme for Model Validation and Model Risk Management developed at the end of Chapter 1.

We consider three asset classes: credit, equity and interest rates.For each asset class we consider a few payoffs, and apply to them a range of different models, including the most popular modelling alternatives in the market. One goal of this chapter is to understand how to perform model comparison and model choice.We show how to deal with model uncertainty with provisions such as Reserves and Model Lines or Limits to Exposure. We perform market intelligence and show how to interpret the results of it with reverse engineering.The first example is introduced here for the first time, for the other two we analyze the existing literature and then go beyond it.

In Chapter 3 we look at stress-testing to understand the core risk of a payoff by using models, an issue already tackled in the previous chapter, and we look at the stress-testing of models to understand their weaknesses, an issue resumed later in Chapter 6.

We devote particular attention to avoiding the pitfalls that are most likely to occur when performing stress-testing.We investigate what cases of stress one should consider (market conditions, payoff features, characteristics of the counterparties…) and we see a few examples of how to use historical and cross-section market information to design stress scenarios.As a playground we display here, via stress-testing, the errors in the practice of credit derivatives that were at the center of the crisis, including the still widespread copula and mapping methods, and present alternatives to these.

In Chapter 4 we consider the most painful event in terms of model losses: when a model consensus in the market suddenly breaks down and is replaced by a radically different standard.

We carry the study on with the purpose of understanding the mechanisms of consensus change, already considered in the first chapter, so as to be not fully unprepared for the changes that will happen in the future.The first example of the death of a model, and the birth of a new one, regards the changes that happened recently to the pricing of even the simplest interest rate derivatives: the separation of discounting and forwarding, the multiplication of term-structures and the explosion of basis spreads. In this analysis we investigate the hidden assumptions of a modelling framework, by seeing how the traditional mathematical representation of interest rates taught in books must be replaced by a different approach.The second example, related to the first one, deals with the inclusion of liquidity and funding in pricing. Since we are still in the middle of this transformation of pricing foundations, we can now study the risks to which we would be exposed depending on the direction the market takes.

The second part of this book is devoted to those aspects of the practice in the financial markets where model risk management is most crucial.

In Chapter 5 we focus on hedging, an activity based on models but dangerously overlooked by the research in quantitative finance, or addressed in a theoretical way unrelated to practice. We take a different approach.

We study how models are used in real hedging, and how this differs from their use in pricing. These differences must be studied and the intrinsic risks understood and managed. The principal example is on local and stochastic volatility models for equity options.We look at how to perform a P&L-Explain test, where one tests the hedging performance of a model. We want to understand the limitations of this technique but also what it can actually tell us about the appropriateness of a model.

In Chapter 6 we focus on computational methods, in order to understand how they must be assessed, stress-tested, and their efficiency monitored.

We focus on approximations since these can hide the sneakiest model risk. In fact when market conditions change approximations often break down, but the market may take some time to react.The examples we see regard mostly the approximations used in the interest rate market, for example convexity adjustment, BGM-model approximations or the SABR formula. In testing them we also show the problems they are having in the current market conditions.We see how an approximation can be tested against an exact method or against a more precise numerical procedure. We also show examples and exercises of the risks in simulation and numerical integration.

In Chapter 7 we analyze the risks associated with two common operations: interpolation and extrapolation. We show two approaches:

How to use non-trivial market information in order to minimize the need for extrapolation. We see this in particular for the volatility smile.How to use the mathematical properties of some quantities in order to make interpolation more consistent and avoid the use of extrapolation. Here we focus on the correlation skew.

In Chapter 8 we tackle the risk involved in correlation modelling from two different perspectives:

We present useful technical solutions for modelling and parameterizing correlations, with examples from different asset classes where correlations need to have different properties.We explore the most common errors made when devising assumptions about correlation, such as assuming rigid relations for factors that have a degree of independence (the 1-correlation risk) and conversely the risk of taking as unrelated those things that have structural links (the 0-correlation risk). Two market cases are observed.

In Chapter 9 we complete the treatment of a topic that is covered in almost all other chapters: calibration. We look at exposing the residual mode uncertainty that remains after a calibration, and minimizing this uncertainty by enrichment of the calibration set.

Introducing some model risk management tools needed to perform diagnostics of a calibration procedure, such as assessing the stability of the resulting model.

Chapter 10 is devoted to an issue that at times is not included in a narrow definition of model risk, but has high relevance: the risk of errors in the description of the payoff.

We consider the case when the errors arise from a superficial interpretation of the termsheet or of the legal prescriptions. We see an example that has a strong impact on the pricing of counterparty risk.We consider the errors that arise from simplifications introduced to ease the mathematical representation of a payoff. The example is on Index options.

Chapter 11 considers an application of models which is typical of hedge funds or proprietary trading desks: using models for statistical or model arbitrage, exploiting temporary inconsis- tencies among related products. We see in practice two classic examples:

Capital-structure arbitrage, based on equity and bonds/CDS, and here addressed with a recent structural model. Cap-swaption arbitrage in a Libor market model.We show by looking at empirical results how “arbitrage trades" can be easier to risk manage as directional trades on market uncertainty.

WHAT ELSE YOU WILL FIND IN THIS BOOK

In explaining model risk and model validation, we describe in detail practical examples where we cover a number of relevant topics for today's finance, not mentioned, or only hinted at, in the above summary:

Correlation modelling for equity with stochastic volatility, interest rates, FX rates, default events.The comparison of local vs stochastic volatility models both in terms of hedging and in terms of pricing path-dependent/forward-start derivatives.The most dangerous correlation errors in the computation of wrong-way counterparty risk.The modern pricing of interest rate derivatives with multiple curves for basis swaps and alternative discounting curves.The up-to-date treatment of the impact of funding liquidity in pricing.The impact of market illiquidity on the way we compute prices, and its relation to model uncertainty.How to set quantitative triggers to detect when a market formula is going to break down.Bubbles, arbitrage and market completeness in practice.A detailed account of the development of the credit crunch and its relationship with model choices and model errors.Diagnostic tools used on the behaviour of a model, such as the way to compute the model-implied evolution of volatilities and smiles.What is really explained by P&L-Explain tests.Different examples of reverse-engineering to understand which models can have generated observable prices.The analysis of the most relevant problems when using copulas for default events, the impossibility to control the timing of related events, and a solution to this.The analysis of gap risk using different models that treat information differently.The meaning, advantages and risks of taking into account the default of our institution in pricing (DVA).Detailed examples from asset classes including credit, interest rates, equity, cross-currency and funding.The analysis of the behaviour of the SABR model and the limits of its pricing formulas.The large number of changes to modelling standards which are required by the post-crisis market patterns.The risks hidden within the pricing procedures for plain vanilla derivatives.An alternative way to model correlations that can explain the correlation skew.Counterparty risk adjustment and the indetermination associated with an unclear legal definition of default payments.The reality of the lack of fundamental information in markets and the role this plays in derivatives marketing and trading.Dealing with funding liquidity and credit simultaneously and the risks of double-counting, loss of competitiveness or excessively aggressive behaviour.New analysis on the pricing of Bermudan swaptions and CMS derivatives.We explore the popular issue of calibrating a model to European options and then applying it to early exercise American/Bermudan derivatives.The explanation via liquidity and counterparty risk of the presence of basis swaps, and the hedging consequences of multiple curves.The explanation and a non-standard analysis of a range of models that include local and stochastic volatility models, jump models, the Libor market model for interest rate deriva-tives, structural models, copulas, mapping methods, reduced form credit models.Two analyses of correlation risk in hedging, for equity and for rates.And much more… but not inflation, nor the variance-gamma model!

Acknowledgements

The author acknowledges fruitful conversations with Bruno Dupire, Riccardo Rebonato, Marco Avellaneda, Emanuel Derman, Igor Smirnov, Umberto Cherubini, Jon Gregory, Vladimir Piterbarg, Paul Wilmott, Emilio Barucci, Josh Danzinger, Antonio Castagna, Claudio Albanese, Ziggy Johnsson, Christian Fries, Marc Henrard, Rama Cont, Alberto Elizalde, Pierpaolo Montana, Andrej Lyashenko, Vladimir Chorny, Lorenzo Bergomi, Alex Lipton, John Crosby, Gianluca Fusai, Pat Hagan, Francesco Corielli, Lane Houghston, Stewart Hodges, Francesca Minucci, Wim Schoutens, Nicola Pede. All of the participants who attended my workshops and courses are deeply thanked (some of them are actually mentioned in the body of the book).

I am also grateful to my colleagues in the Financial Engineering of Banca IMI, for the discussions on the foundations and the details of modelling. Some of them must be named individually: Nicola Moreni for his rigour, Daniele Perini for his precision, Giulio Sartorelli for the extraordinary depth, Mario Pucci for always mixing wit with fun, Federico Targetti for his eclecticism, Gianvittorio Mauri and Ferdinando Ametrano for their experience, Paola Mosconi and Alessio Calvelli for bringing clever fresh thinking (and adding Roman wisdom to our Nordic strictness). A thank you also goes to Andrea Bugin, our boss, for always favouring deep reasoning and discussion, and to Alberto Mina, for his endless hard work while we were reasoning and discussing (waiting for your own book, Alberto). Last but not least I thank Giorgio Facchinetti, whose intellectual honesty and technical solidity has proven the best test for any new original idea. Among the other colleagues I need to mention (and I am surely forgetting many) Luigi Cefis, Pietro Virgili, Cristina Duminuco, Francesco Fede, Sebastiano Chirigoni, Salvatore Crescenzi, Giuseppe Fortunati, Fabio Perdichizzi, Cristiana Corno, Francesco Natale, Federico Veronesi, Luca Dominici, Stefano Santoro, Pierluigi D'Orazio, Raffaele Giura, Michele Lanza, Roberto Paolelli, Luca Brusadelli, Biagio Giacalone, Marcello Terraneo, Massimo Baldi, Francesco Lago, Stefano Martina, Alessandro Ravogli, Cristiano Maffi, Valeria Anzoino, Emiliano Carchen, Raffaele Lovero. Marco Bianchetti and Andrea Prampolini are thanked for the insight they gave me on many occasions, but even more for being so close to what a trader and a validator should be in the dreams of a quant. A special thank you in the quant community goes to my masters of old Damiano Brigo, Fabio Mercurio, my brother Maurizio, Nick Webber, Pietro Muliere and my late professors Umberto Magnani and Carlo Giannini.

Among those that made this book physically possible I thank Pete Baker, Aimee Dibbens, Tessa Allen, Lori Boulton, Mariangela Palazzi-Williams and all the Wiley team. The advice of Raul Montanari is gratefully acknowledged.

I thank my son Vittorio, for showing me, even too often, that the desire to understand how things really work comes before all the theory and the books, and my daughter Giulia, for teaching me regularly, at the age of three, how things should be properly explained. A thank you to Enzo and Mirella, for their principles have proven as good in the global financial markets as they were in a village in the Italian countryside. No thanks will ever be sufficient for Elena.

“The wide world is all about you; you can fence yourselves in, but you cannot forever fence it out.”

Gildor in ‘The Lord of the Rings', by J.R.R Tolkien

“There was no way, without full understanding, that one could have confidence that conditions the next time might not produce erosion three times more severe than the time before. Nevertheless, officials fooled themselves into thinking they had such understanding and confidence, in spite of the peculiar variations from case to case. A mathematical model was made to calculate erosion. This was a model based not on physical understanding but on empirical curve fitting … Similar uncertainties surrounded the other constants in the formula. When using a mathematical model careful attention must be given to uncertainties in the model.”

Richard Feynman, from ‘Rogers' Commission Report into the Challenger Crash Appendix F -- Personal Observations on Reliability of Shuttle'

“It does not do to leave a live dragon out of your calculations, if you live near him.”

‘The Hobbit', by J.R.R Tolkien

“Official management, on the other hand, claims to believe the probability of failure is a thousand times less. One reason for this may be an attempt to assure the government of NASA perfection and success in order to ensure the supply of funds. The other may be that they sincerely believed it to be true, demonstrating an almost incredible lack of communication between themselves and their working engineers.”

Richard Feynman, from ‘Rogers' Commission Report into the Challenger Crash Appendix F -- Personal Observations on Reliability of Shuttle'

“Now, therefore, things shall be openly spoken that have been hidden from all but a few until this day … And I will begin that tale, though others shall end it … You may tarry, or come back, or turn aside into other paths, as chance allows. The further you go, the less easy will it be to withdraw.”

Elrond in ‘The Lord of the Rings', by J.R.R Tolkien

Part I

Theory and Practice of Model Risk Management

1

Understanding Model Risk

1.1 WHAT IS MODEL RISK?

In the last years, during and after the credit crunch, we have often read in the financial press that errors on ‘models’ and lack of management of ‘model risk’ were among the main causes of the crisis. A fair amount of attacks regarded mathematical or quantitative models, like the notorious Gaussian copula, that were accused to be wrong and give wrong prices for complex derivative, in particular credit and mortgage-related derivatives. These criticisms to valuation models have been shared also by bank executives and people that are not unexperienced on the reality of financial markets. In spite of this it is not very clear when a model must be considered wrong, and as a consequence it is not clear what model risk is.

We can probably all agree that model risk is the possibility that a financial institution suffers losses due to mistakes in the development and application of valuation models, but we need to understand which mistakes we are talking about.

In the past, model validation and risk management focused mainly on detecting and avoiding errors in the mathematical passages, the computational techniques and the software implementation that we have to perform to move from model assumptions to the quantification of prices. These sources of errors are an important part of model risk, and it is natural that model risk management devotes a large amount of effort to avoid them. We will devote a share of the second part of this book to related issues. However, they regard that part of model risk that partially overlaps with a narrow definition of operational risk: the risk associated to lack of due diligence in tasks for which it is not very difficult to define what should be the right execution. Is this what model validation is all about? In natural science, the attempt to eliminate this kind of error is not even part of model validation. It is called model verification, since it corresponds to verifying that model assumptions are turned correctly into numbers. The name model validation is instead reserved to the activity of assessing if the assumptions of the model are valid. Model assumptions, not computational errors, were the focus of the most common criticisms against quantitative models in the crisis, such as ‘default correlations were too low’.

The errors that we can make in the assumptions underlying our models are the other crucial part of model risk, probably underestimated in the past practice of model risk management. They are the most relevant errors in terms of impact on the reputation of a financial institution that works with models. A clear example is what happened with rating agencies when the subprime crisis burst. When they were under the harshest attacks, rating agencies tried to shield themselves from the worst criticisms by claiming that the now evident underestimation of the risk of credit derivatives was not due to wrong models, but to mistakes made in the software implementation of the models. Many market operators, that knew the models used by rating agencies, did not believe this justification, and it had no other effect than increasing the perception that wrong models were the real problem. What is interesting to notice is that admitting wrong software appeared to them less devastating for their reputation than admitting wrong models.

Unfortunately, errors in mathematics, software or computational methods are easy to define and relatively easy to detect, although this requires experience and skills, as we will see in the second part of the book. Errors in model assumptions, instead, are very difficult to detect. It is even difficult to define them. How can we, as the result of some analysis, conclude that a model, intended as a set of assumptions, has to be considered wrong? We need to understand when a valuation model must be called wrong in order to answer to our first crucial question,