Operational Weather Forecasting E-Book

Peter Michael Inness

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Serie: Advancing Weather and Climate Science
• Sprache: Englisch

Seitenzahl: 421

Veröffentlichungsjahr: 2012

Beschreibung

This book offers a complete primer, covering the end-to-end process of forecast production, and bringing together a description of all the relevant aspects together in a single volume; with plenty of explanation of some of the more complex issues and examples of current, state-of-the-art practices.

Operational Weather Forecasting covers the whole process of forecast production, from understanding the nature of the forecasting problem, gathering the observational data with which to initialise and verify forecasts, designing and building a model (or models) to advance those initial conditions forwards in time and then interpreting the model output and putting it into a form which is relevant to customers of weather forecasts. Included is the generation of forecasts on the monthly-to-seasonal timescales, often excluded in text-books despite this type of forecasting having been undertaken for several years.

This is a rapidly developing field, with a lot of variations in practices between different forecasting centres.  Thus the authors have tried to be as generic as possible when describing aspects of numerical model design and formulation.  Despite the reliance on NWP, the human forecaster still has a big part to play in producing weather forecasts and this is described, along with the issue of forecast verification – how forecast centres measure their own performance and improve upon it.

Advanced undergraduates and postgraduate students will use this book to understand how the theory comes together in the day-to-day applications of weather forecast production.  In addition, professional weather forecasting practitioners, professional users of weather forecasts and trainers will all find this new member of the RMetS Advancing Weather and Climate series a valuable tool. 

• Provides an end-to-end description of the weather forecasting process
• Clearly structured and pitched at an accessible level, the book discusses the practical choices that operational forecasting centres have to make in terms of what numerical models they use and when they are run.
• Takes a very practical approach, using real life case-studies to contextualize information
• Discusses the latest advances in the area, including ensemble methods, monthly to seasonal  range prediction and use of ‘nowcasting’ tools such as radar and satellite imagery
• Full colour throughout
• Written by a highly respected team of authors with experience in both academia and practice.
• Part of the RMetS book series ‘Advancing Weather and Climate’

Leseprobe

Table of Contents

Series Page

Title Page

Copyright

Series Foreword

Advancing Weather and Climate Science

Preface

Acknowledgements

Chapter 1: Introduction

1.1 A brief history of operational weather forecasting

Chapter 2: The Nature of the Weather Forecasting Problem

2.1 Atmospheric predictability

2.2 The importance of observations in weather forecasting

2.3 An overview of the operational forecast process

Summary

Chapter 3: Meteorological Observations

3.1 What do we need from a meteorological observing system?

3.2 Data transmission and processing

3.3 Observing platforms

Summary

Chapter 4: NWP Models – the Basic Principles

4.1 The basic ingredients of an NWP model

4.2 Building the physical principles into a model

4.3 Setting the initial conditions for the forecast

Summary

Chapter 5: Designing Operational NWP Systems

5.1 Practical considerations for an NWP suite

5.2 Ensemble prediction systems

5.3 Model output—what can NWP models produce?

5.4 Using NWP output to drive other forecast models

Summary

Chapter 6: The Role of the Human Forecaster

6.1 The role of the senior forecasting team

6.2 Production of forecasts for customers

Summary

Chapter 7: Forecasting at Longer Time Ranges

7.1 Where does the predictability come from in longer range forecasts?

7.2 Observations of ocean and land surface variables

7.3 Monthly to seasonal forecasting systems

7.4 Presentation of longer range forecasts

Summary

Chapter 8: Forecast Verification

8.1 Deterministic forecast verification

8.2 Verification of probability forecasts

8.3 Subjective verification

Summary

References

Index

Advancing Weather and Climate Science Series

Series Editors:

Peter Inness, University of Reading, UK

William Beasley, University of Oklahoma, USA

Other titles in the series:

Mesoscale Meteorology in Midlatitudes

Paul Markowski and Yvette Richardson, Pennsylvania State University, USA

Published: February 2010

ISBN: 978-0-470-74213-6

Thermal Physics of the Atmosphere

Maarten H.P. Ambaum, University of Reading, UK

Published: April 2010

ISBN: 978-0-470-74515-1

The Atmosphere and Ocean: A Physical Introduction, 3rd Edition

Neil C. Wells, Southampton University, UK

Published: November 2011

ISBN: 978-0-470-69469-5

Time-Series Analysis in Meteorology and Climatology: An Introduction

Claude Duchon, University of Oklahoma, USA and

Robert Hale, Colorado State University, USA

This edition first published 2013 © 2013 by John Wiley & Sons, Ltd

Wiley-Blackwell is an imprint of John Wiley & Sons, formed by the merger of Wiley's global Scientific,

Technical and Medical business with Blackwell Publishing.

Registered office: John Wiley & Sons, Ltd, The Atrium, Southern Gate, Chichester, West Sussex,

PO19 8SQ, UK

Editorial offices: 9600 Garsington Road, Oxford, OX4 2DQ, UK

The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

111 River Street, Hoboken, NJ 07030-5774, USA

For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com/wiley-blackwell.

The right of the author to be identified as the author of this work has been asserted in accordance with the

UK Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher.

Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought.

Library of Congress Cataloging-in-Publication Data

Inness, Peter (Peter M.)

Operational weather forecasting / Peter Inness and Steve Dorling.

p. cm.

Includes bibliographical references and index.

Summary: “This book will cover the end-to-end process of operational weather forecasting”–Provided by publisher.

ISBN 978-0-470-71159-0 (cloth) – ISBN 978-0-470-71158-3 (pbk.) 1. Weather forecasting. I. Dorling, Steve (Stephen R.) II. Title.

QC995.I44 2013

551.63–dc23

Series Foreword

Advancing Weather and Climate Science

Meteorology is a rapidly moving science. New developments in weather forecasting, climate science and observing techniques are happening all the time, as shown by the wealth of papers published in the various meteorological journals. Often these developments take many years to make it into academic textbooks, by which time the science itself has moved on. At the same time, the underpinning principles of atmospheric science are well understood but could be brought up to date in the light of the ever increasing volume of new and exciting observations and the underlying patterns of climate change that may affect so many aspects of weather and the climate system.

In this series, the Royal Meteorological Society, in conjunction with Wiley-Blackwell, is aiming to bring together both the underpinning principles and new developments in the science into a unified set of books suitable for undergraduate and postgraduate study as well as being a useful resource for the professional meteorologist or Earth system scientist. New developments in weather and climate sciences will be described together with a comprehensive survey of the underpinning principles, thoroughly updated for the 21st century. The series will build into a comprehensive teaching resource for the growing number of courses in weather and climate science at undergraduate and postgraduate level.

Series EditorsPeter InnessUniversity of Reading, UKWilliam BeasleyUniversity of Oklahoma, USA

Preface

If you are interested in learning how weather forecasts are produced in today's forecasting centres then this book aims to be a complete primer, covering the end-to-end process of forecast production. Other textbooks cover specific aspects of the process and, in particular, the formulation of numerical models, but here we aim to bring a description of all the relevant aspects together in a single volume, with plenty of explanation of some of the more complex issues and examples of current, state-of-the-art practices.

This book grew out of a module on ‘Operational Forecasting Systems and Applications’, which is part of the University of Reading MSc in Applied Meteorology. Because the University of Reading also runs an MSc course in Numerical Modelling of the Atmosphere and Oceans, and another in Data Assimilation, the module deliberately avoids too much detail on the mathematical formulation of numerical models and the statistical and numerical formulation of data assimilation schemes. This book follows the same approach and is intended to be an overview of the end-to-end process of weather forecast production at a major National Weather Service. The physics and numerics of models and the formulation of data assimilation schemes are touched upon in the module, but other options exist for students at Reading if they want to learn about these aspects in more detail. Because the students at Reading come from all around the world, the module also attempts to be as generic as possible when discussing operational weather forecasting. However, because of our location in the United Kingdom (with UK Met Office staff based in the University's Meteorology Department) and with the European Centre for Medium-range Weather Forecasts (ECMWF) two miles down the road, many of the examples included in the module (and hence this book) are based on practices at these two major forecasting centres.

When I started to put together material for the MSc module in 2005, it soon became clear that there was no good single textbook available on the process of operational weather forecast production. There were several excellent texts on the numerical formulation of models and the design of data assimilation schemes, and several more have appeared since. The lack of books on operational weather forecasting is probably because this is a rapidly developing field, with a lot of variations in practices between different forecasting centres, making its operational application a difficult field to describe in a textbook. Another issue is that the people involved in operational forecasting on a day-to-day basis are also far too busy doing their jobs to take time out and write a book about it all! I discovered this when, as series editor for this Wiley-Blackwell series on Advancing Weather and Climate Science, I started to approach potential authors for a textbook on this subject. Many people said that a textbook would be useful but they were unable to commit themselves to writing one.

I decided, therefore, that it would be worth trying to commit the material from the Reading MSc module, together with further detail and examples, to a book. Steve Dorling at the University of East Anglia, who teaches similar modules while also being Innovations Director at the UEA-based Weatherquest Ltd, a private-sector company, agreed to join me in this venture. We realised at the outset that we would have to accept that some aspects of the book would become out-of-date fairly rapidly, and that using specific examples of practices at one forecasting centre might also alienate potential readers with an interest in practices at a different centre. Despite this we agreed that it was worth trying to commit the current state of the art to a textbook and have tried to be as generic as possible when describing aspects of numerical model design and formulation. We hope the final product and any subsequent editions will be adopted by professional meteorological training colleges, by universities and indeed by individuals fascinated by meteorology.

This book then is aimed primarily at advanced undergraduate and masters level students of meteorology and so some knowledge of basic meteorology is assumed. Many universities around the world teach courses in theoretical meteorology and numerical modelling, using the many excellent books that are available covering these topics. Our book is aimed at helping the students on those courses to understand how the theory comes together in the day-to-day applications of weather forecast production. To some extent it could be regarded as an aid to converting from a student of meteorology into an operational practitioner. Discussions with staff in operational forecasting centres who are involved in the recruitment and training of new staff have commented that this would be a very useful purpose for a textbook. Many students leave university courses with an excellent grasp of the theory of meteorology but only a rather slim knowledge of the actual practice of forecast production.

This book is neither a manual on how to build a numerical model or how to be an operational weather forecaster but instead aims to cover the whole process of forecast production, from understanding the nature of the forecasting problem, gathering the observational data with which to initialise and verify forecasts, designing and building a model (or models) to advance those initial conditions forwards in time and then interpreting the model output and putting it into a form which is relevant to customers of weather forecasts. One subject which we wanted to include was the generation of forecasts on the monthly-to-seasonal timescales. This has been an area of research for many years and some operational centres have been doing this type of forecasting for some time but again it is not well covered in textbooks.

As far as references are concerned we have taken a fairly sparse approach. When describing fundamentals of numerical weather prediction (NWP) model design we have referenced a very few seminal papers but other textbooks that cover this topic in more detail (such as Eugenia Kalnay's excellent ‘Atmospheric Modeling, Data Assimilation and Predictability’) already provide extremely comprehensive reference lists in this area. We have included references when describing specific schemes, methods and techniques used at different operational forecasting centres, or particular studies conducted at these centres. However, because operational forecasting systems change rapidly and undergo frequent upgrades, many new developments never make it into the wider literature before they become out of date. The priority of operational centres is to maintain and develop their forecasting systems and update their internal documentation rather than publish their methods in the reviewed literature. Much of the detail of operational forecasting products in this book also comes from web sites which change regularly. We have provided web addresses where appropriate but the reader must be prepared for the information on those sites to change or for the addresses themselves to change or disappear.

Peter InnessUniversity of Reading

Acknowledgements

A number of forecasting centres have given us access to their documentation, training material and staff to help in the production of this book. In particular we would like to thank the European Centre for Medium-range Weather Forecasts (ECMWF) for a large amount of material from its excellent training courses which appears throughout the book. The UK Met Office has also provided a considerable amount of material and we would particularly like to thank Tim Hewson, a senior forecaster at the UK Met Office, for his guided tour around the National Meteorological Centre at Exeter and discussions on the role of a senior forecaster within a large forecasting organisation. The NOAA Climate Prediction Center and the online plotting and analysis facility of the NOAA Earth Systems Research Laboratory have also proved to be excellent sources of data and figures.

I would like to thank many colleagues at the University of Reading for interesting discussions on many aspects of theoretical and applied meteorology which have helped build the framework for this book. Thanks also to former colleagues at the Met Office College (1996–1999) with whom I worked training forecasters and meteorological research staff. The things I learnt whilst working there have been invaluable in many aspects of my current work. Thanks are also due to all the University of Reading students who have provided feedback on the MSc module ‘Operational Forecasting Systems and Applications’. Much of this feedback has been incorporated into this book—not least the suggestion that it ought to be written in the first place. Finally, thanks to my family who keep me from getting too absorbed in the work and help me relax and enjoy life.

SD would like to thank the University of East Anglia (UEA) for granting a sabbatical period which was very valuable in preparing material for the text. Thanks also to ECMWF for the stimulating course on ‘Use and Interpretation of ECMWF Products’, to Weatherquest Ltd colleagues and clients for the chance to discuss, design and deliver operational services, and to all the students of meteorology at UEA for their enthusiasm over the last twenty years. To Tim, Val, Rachel, Heather, Lewis and Lyndsey Dorling, thank you for the greatest support of all.

Chapter 1

Introduction

The production of weather forecasts for use by the general public, governments, the military, news media and a wide range of industrial and commercial activities is a major international activity. It involves tens of thousands of people and many billions of pounds worth of high-tech equipment, such as computing and telecommunications networks, satellites and land-based observing systems. Most nations across the world have a national meteorological agency which generates forecasts for both government and commercial customers, although the size and scope of these agencies varies widely from country to country. There are also many private companies producing weather forecasts. These companies often specialise in forecasts for particular niche markets—news media, commercial shipping, offshore oil and gas exploration and production, the agricultural sector and so on. In some cases, weather forecasts can have very large financial benefits to particular customers. Power generation companies can make many thousands of pounds on the basis of a single weather forecast of a cold spell, as this allows them to buy gas at a low price prior to the increased demand during cold weather pushing the price up. Offshore gas and oil exploration companies can avoid multimillion dollar damage to drilling platforms by shutting down operations prior to the onset of a severe storm. Major supermarket chains use weather forecasts to plan their stock control in the knowledge that a spell of cold weather will result in increased sales of foods like soup whereas a spell of warm weather will increase demand for ice cream and barbecues.

The production of weather forecasts is based on a sound scientific understanding of how the atmosphere works, coupled with a vast amount of technical investment in observing, computing and communication facilities. The current state-of-the-art forecasting facilities at the world's major meteorological centres have evolved over a period of many years and are the result of scientific research and technical development. At the heart of any major weather forecasting centre is a numerical weather prediction (NWP) model—a computerised model of the atmosphere which, given an initial state for the atmosphere, derived from observations, can generate forecasts of how the weather will evolve into the future. Prior to the advent of NWP, forecasts were produced manually, usually by forecasters drawing maps of the current state of the atmosphere and then, using their knowledge of typical weather patterns together with a set of empirical rules, attempting to predict how that state would evolve. These days the computer model has taken over the task of predicting the evolution of the atmospheric state but there is still a big role for the human forecaster in the system.

This book is not a guide to the detailed formulation and coding of an NWP model. Nor is this book a manual on how to be a weather forecaster. Individual meteorological services produce training manuals for their forecasters which go into the specifics of forecast production within their organisations. Rather, this book will describe the end-to-end process of weather forecast production, with a focus on the NWP tools available to forecasters in major meteorological centres. The nature of the weather forecasting problem is discussed in Chapter 2. Chapter 3 focuses on the observations of the weather that allow forecasting centres to set the initial conditions for their forecasts. These same observations also form an important part of the forecast production process in the sense that human forecasters will continually check the observations against the NWP forecasts and modify the forecasts as and when necessary. Chapters 4 and 5 look at the basic ingredients of NWP models and how these ingredients are applied in order to produce operational forecasting systems for specific tasks. In Chapter 6 the role of the human forecaster within the NWP forecast production process is examined. Chapter 7 concentrates on the specific challenge of forecasting for periods of several weeks to several months ahead and, finally, Chapter 8 looks at how NWP forecasts are verified and measured, and how this process then feeds into the process of continuing development and improvement.

1.1 A brief history of operational weather forecasting

Many different human activities have always been sensitive to prevailing weather conditions and so people have been trying to predict the weather, both on a day-to-day basis and for the coming few months, since ancient times. As far back as the earliest civilizations, the weather during the growing season has affected crop production and so farmers have always taken note of the weather and climate. The ancient Egyptians, for instance, kept detailed records of the flooding of the Nile—an annual event that had a big impact on soil fertility and which was strongly affected by the intensity of the rainy season around the headwaters of the Nile.

Any kind of weather prediction prior to the scientific advances of the nineteenth century was largely based on folklore and perceived relationships between events in nature and the weather. Certainly there was no scientific basis to most of these forecasting methods but farmers and people who spent most of their days in the open air were certainly well attuned to the prevailing conditions and by observing changes in clouds and wind could make reasonably good predictions of the forthcoming weather for the next few hours or even a day or two. However, there was no systematic attempt to predict the weather in any kind of organised way.

The first organised meteorological agency was set up by the British government in the mid-nineteenth century as a response to loss of shipping on the trade routes that sustained the British Empire. In 1854 the British Board of Trade appointed Admiral Robert FitzRoy as its ‘meteorological statist’. FitzRoy was an oceanographic surveyor with a reputation for producing detailed and accurate hydrological charts of coastal waters around the world. The Board of Trade hoped that he would also be able to produce an equivalent meteorological atlas charting weather conditions around the world which would serve to inform shipping lines and crews of the risks of storms, allowing them to make choices about when to sail and what routes to take.

FitzRoy began this task but he was also interested in the possibility of being able to issue more specific predictions of the weather conditions in coastal waters around Britain on a day-to-day basis. Since the invention of the barometer by Toricelli in the seventeenth century, people had started to realise that atmospheric pressure was correlated with weather conditions, with falling pressure often presaging unsettled or even stormy conditions, and rising or steady pressure indicating settled conditions. FitzRoy made use of this concept by designing a barometer specifically for use in ports, with information on what weather conditions to expect given an observed trend in the pressure. A version of this barometer was set up in all the major harbours around the United Kingdom, so that captains could consult the barometer prior to setting sail. Interestingly, as well as advice on the expected weather conditions associated with rising and falling pressure, there was also advice on how to interpret the colour of the sky at dawn and dusk in terms of forthcoming weather conditions. The recent advent of the electric telegraph allowed FitzRoy to take the use of these barometers one stage further. Regular readings from the barometers around the country, together with information about other weather variables, such as wind and cloud, could be telegraphed back to FitzRoy's office in London and the readings could be plotted onto a chart that summarized the atmospheric pressure distribution around the country. A sequence of these charts could then be used to predict the winds and prevailing weather around the country in the coming hours and even a day or so ahead. Furthermore, by building up an archive of these charts—today we'd probably call it a database—it might be possible to compare the current pressure distribution with similar pressure patterns from the past and use the knowledge of what happened to the weather in the earlier cases to predict what might happen this time. Predictions could then be telegraphed back to the port authorities. In this way FitzRoy invented the concept of operational weather forecasting, and probably also coined the term ‘weather forecast’.

Of course, the infinitely variable nature of the weather meant that many of FitzRoy's forecasts went wrong and, particularly after making his forecasts more widely available to the public through the daily newspapers, FitzRoy received a lot of criticism from the public and the scientific community. This criticism was one of the several factors that led to FitzRoy's suicide in 1865. The work of the Meteorological Office of the Board of Trade continued though, with the forecasting methods being refined and new ones being developed. Napier Shaw took over the directorship of the Meteorological Office in 1905 and was instrumental in introducing more scientifically based forecasting methods.

The French government set up its first national weather service in 1855, once again as a response to the loss of shipping—this time due to a storm in the Black Sea during the Crimean War. In 1870 the US government, under President Ulysses S. Grant, also set up a meteorological service, this time under the auspices of the US War Office. Grant's recent experience as a general in the American Civil War had made him well aware of the impact of the weather on military operations, and the military forts around the United States provided the ideal locations for a weather observing network. The US National Weather Service became a civilian agency in 1890 when it was moved into the Department of Agriculture. The Australian Bureau of Meteorology was set up in 1906, although prior to this date each state had its own meteorological service.

Throughout the twentieth century, further scientific advances in weather prediction continued to be made. The British meteorologist Lewis Fry Richardson developed, single-handed, a method of forecasting the evolution of the state of the atmosphere using the set of physical equations that govern atmospheric motion. During World War I, in which he served as an ambulance driver, Richardson developed a way of solving these equations numerically and even performed a six-hour forecast of the pressure in central Europe, using data from 20 May 1910. This was a Herculean task involving many thousands of calculations that needed to be performed and double checked without the aid of any kind of electronic calculation device. The result was completely wrong but in his 1922 book, Weather Prediction by Numerical Process, Richardson set down his methods; these have formed the basis of numerical weather prediction ever since.

During World War II, the massive use of military aviation and shipping to conduct the fighting over wide areas demanded advances in weather forecasting. Aircraft on long range bombing missions relied on favourable weather conditions in order to be able to see the ground on arrival over their targets and naval vessels, particularly those involved in the Pacific campaign in the part of the world most prone to tropical cyclones, needed to be able to avoid damaging storm conditions. Weather observing and forecasting practices improved rapidly through this period, with forecasters for the first time starting to pay attention to the development of the middle and upper troposphere in order to determine what might happen to the weather. The use of weather balloons carrying instrumented packages to observe the upper troposphere became more widespread as a result. Probably the most famous single weather forecast of all time was made during WWII, with forecasters led by Captain James Stagg advising the Allied Command to delay the D-Day Normandy landings in June 1944 by 24 hours.

Following the war, research into forecasting methods followed two main strands. In the United Kingdom meteorologists developed techniques based on looking at maps of the state of the middle and upper troposphere in order to predict where significant weather system developments would occur. In the United States researchers at a number of different institutions were working on developing and refining the numerical methods proposed by Richardson to produce forecasts. The availability of numerical computing devices, such as the ENIAC machine, greatly aided this strand of development. The results of the first computerised atmospheric forecast were published in 1950 by a research group at Princeton University, which included the mathematician John von Neumann and the meteorologist Jule Charney. The numerical model used had been developed over the preceding few years and was actually somewhat simpler than that used by Richardson. However, it produced a reasonably good 24-hour prediction of the evolution of the mid-tropospheric flow over the continental United States and this encouraging result led to further refinements of the numerical methods. At this stage, the method was in no way ready for operational use—the computing time alone was about 24 hours for a 24-hour forecast and this didn't include the time spent preparing the initial conditions for the forecast and inputting them into the computer.

The first operational numerical weather forecasts were made by the Swedish Military Weather Service in 1954. The development of the methods used in these forecasts was led by Carl-Gustav Rossby, a native Swedish meteorologist who had worked in the USA, principally at Chicago University, during the 1930s and 1940s. On returning to Sweden in 1947 he founded the Swedish Institute of Meteorology and continued to develop numerical forecasting methods.

The UK Met Office came to numerical weather prediction (NWP) rather late. During the 1950s and early 1960s the UK Met Office had to do its research into NWP on borrowed computers and it wasn't until the mid-1960s that it actually had its own computing facility. In 1965 the UK Met Office started to routinely produce numerical weather forecasts and in 1967 it had the distinction of producing the first numerical prediction of precipitation. Prior to this, numerical forecasts had only predicted the evolution of pressure, geopotential height and vorticity patterns and it was left to experienced forecasters to interpret these patterns in terms of the actual weather that would occur in association with them.

By the 1970s most of the major meteorological agencies around the world were well established and starting to use NWP methods as the basis of their operational forecasts. One major development in the 1970s was the setting up of the European Centre for Medium-range Weather Forecasts (ECMWF), in Reading, UK, in 1975. The aim of the centre was to develop operational NWP forecasts for the medium range (out to about 15 days) using funding from all the main European meteorological services, which would not have been able to develop such facilities with their own resources. The forecasts would then be made available to the National Weather Services of all the member states. ECMWF pioneered the operational use of ensemble forecasting techniques (more details are given in Chapter 5) and has since developed numerical methods and models for prediction on the monthly to seasonal timescale. Such techniques require vast amounts of computing power and it is only through the collaborative funding of all the member states that such facilities can be maintained and upgraded. The supercomputing facilities at ECMWF regularly top the league table of computing power in the United Kingdom and the NWP forecasts produced using these facilities are widely regarded as being the best in the world. The US National Center for Environmental Prediction (NCEP) started to produce ensemble forecasts in the early 1990s. More recently many forecasting centres have introduced ensemble methods as computing facilities have developed. The UK Met Office and the Australian, Chinese, Japanese, Korean, French, Brazilian and Canadian National Meteorological Services all now routinely run ensemble forecasts in some form.

Since the 1970's NWP methods have become more accurate and the models used have become faster. Communication networks have improved massively over this period too. As recently as the early 1980's an outstation weather forecaster working at a military airfield for instance, would have had very little access to the output from numerical models, and what few products were disseminated often arrived too late to be of any use in making forecasts for the aircrews. This lack of up-to-date model output made many forecasters somewhat wary of using NWP products to guide their forecasts. Similarly outstation forecasters saw very few satellite images. It wasn't until the advent of high bandwidth communication networks in the 1990's that forecasters working at locations remote from the main meteorological service headquarters got to see a wide range of guidance from numerical models together with regular, detailed satellite imagery. So going right back to FitzRoy who was able to make use of the newly invented electric telegraph system, it is clear that communication networks play a vital part in operational meteorology.

Improvements in forecast accuracy since the 1980s are illustrated in Figure 1.1. It shows, for the ECMWF model, the point in the forecast at which the correlation between the forecast and observed 500 hPa geopotential height anomaly in the Northern hemisphere extra-tropics falls below 60%. This value is considered to be a measure of the usefulness of the forecast, with values above 60% representing skilful forecasts. The blue dashed line shows the value for each month since January 1980, and the red solid line shows the 12-month running mean of the monthly values. In 1980 the correlation fell below 60% at about 5.5 days into the forecast. By 2010 this had risen to about 8.5 days into the forecast. Effectively this means that in 2010 ECWMF was producing forecasts which were skilful for an average of three days longer than in 1980. Other forecast centres show similar improvements in skill. Of course, to most users of weather forecasts the anomaly correlation of the Northern hemisphere 500 hPa geopotential height is a pretty meaningless measure of the quality of a weather forecast and there are many ways of measuring forecast skill that are more focused on the interests of specific customers. These are discussed in detail in Chapter 8.

Figure 1.1 A time series of the anomaly correlation between the forecast and observed 500 hPa geopotential height anomaly in the northern hemisphere extra-tropics from the ECMWF forecast model. The vertical scale shows the point of the forecast in days at which this correlation falls below 60%. The blue dashed line shows this value for every month and the red solid line shows the 12-month running mean. (Reproduced by permission of ECMWF.)

Recent developments in operational forecasting have been many and varied, all helped along by regular increases in the availability of computing power to meteorological agencies. More forecasting centres are now running ensemble forecasting systems and a wider range of numerical models, some with global coverage and others with very fine scale resolution over limited areas. So-called ‘storm resolving models’, which can explicitly represent organised convective storms, have started to come into operational use over the past few years and new ways of incorporating observations, such as rainfall rates from meteorological radar, into models are improving the way that these highly detailed forecasts are initialised. At the other end of the time and space scales, more forecasting centres are now running monthly and seasonal forecast models. Some of these developments are described in more detail in the relevant chapters of this book.

Despite the advances in technology which are driving improvements in forecast accuracy, there is still a fundamental place in the weather forecasting process for human experts. Even the most sophisticated numerical models of the atmosphere may produce forecasts which diverge significantly from reality even at quite short time ranges. In these circumstances a team of expert forecasters can spot the problems at an early stage and consider how best to amend the forecast. Models also have known systematic errors and biases, particularly when producing forecasts of local detail, and an experienced forecaster will be able to take account of these issues when producing forecasts for specific customers. Many customers of weather forecasts also require a human forecaster to act as an interface between them and the numerical weather forecast, such as military aircrew receiving a face-to-face briefing from a forecaster prior to flying weather sensitive missions or local government agencies needing briefing and hour-by-hour advice on potential disruption due to snow and ice or flooding. Often the most visible weather forecasters to the general public are those presenting weather forecasts on television but it must be remembered that these people are just the visible face of a large team of experts running, monitoring and analysing the numerical forecasts and deciding what the key weather issues for each day's forecast will be.

Chapter 2

The Nature of the Weather Forecasting Problem

2.1 Atmospheric predictability

To solve any scientific problem, it is essential to fully understand the nature of the problem itself. The critical factors that affect the outcome need to be identified and fully integrated into the methods used to solve the problem. Weather forecasting is no exception to this. In essence, forecasting the weather is a problem of atmospheric physics, with many different physical processes contributing to the final outcome of the forecast. These processes need to be included in our forecasting methods as realistically as possible if we are to achieve skilful forecasts on a regular basis.

Weather forecasting can be thought of in terms of a mathematical problem too, and this is where real insight can be gained into how best to go about solving it. Weather forecasting, at least on the timescale of a few hours to a week or so, can be described as an ‘Initial Value Problem’ (IVP). Mathematically speaking, this is a problem in which the outcome is significantly determined by the conditions fed in at the start. In the case of weather forecasting, these conditions would be the state of the atmosphere at the starting point of the forecast. If we wish to produce a forecast for tomorrow, we need to start by determining, as precisely as possible, the state of the atmosphere today. We can then apply the various physical laws to those conditions to advance the state of the atmosphere forward in time towards tomorrow's forecast state. If instead we start the forecast with yesterday's weather as our initial condition, and have a perfect forecasting system which takes account of every possible relevant physical process, then using this system to advance the state of the atmosphere forwards by 24 hours will end up giving us a forecast of today's weather, not tomorrow's.

It seems obvious, therefore, that accurately specifying the initial conditions for a weather forecast is as crucial to the outcome of the forecast as understanding all the processes that lead to the atmosphere changing its state. This has been known for many years and the art of meteorological analysis, whereby a skilled forecaster draws up a map or set of maps of the current state of the atmosphere (or at least its most recently observed state), has long formed a central part of the forecast process. However, it has only been since the advent of numerical weather prediction that it has become clear just how critical the accurate specification of the initial conditions of a weather forecast actually is to the outcome of that forecast. The pioneering work in this area of Ed Lorenz and others during the 1950s and 1960s has passed into meteorological legend. Meteorologists and mathematicians at this time were interested in the non-linear nature of the equations which govern the evolution of the state of the atmosphere and many other physical systems (Box 2.1).

Box 2.1 What is a non-linear system?

The easiest way to describe the characteristics of a non-linear system is to first describe a linear one. A linear system is one in which, if a change to the initial conditions produces a change to the state of the system of size x at some time t in the future, then if we multiply the change to the initial conditions by a factor a then the change to the state of the system at time t will be proportional to a. So a car driving at a constant speed and taking a certain length of time to complete a journey is a linear system. If the car drives at twice the speed, the journey will take half the time. If the car travels at half the speed the journey will take twice as long. A non-linear system is one in which this proportionality does not apply. Non-linear systems contain feedbacks which can amplify or dampen initial changes to the state of the system. Very few real physical systems are actually linear.

Lorenz demonstrated the fact that small changes to the state of the atmosphere in a numerical model can result in large changes in the evolution of that state by careful mathematical analysis of the equations that govern atmospheric motion, and also by analysing other simpler but still non-linear systems. However, the example that really brings home the effect of the non-linearity of the atmosphere happened almost by mistake. Lorenz wanted to re-run a segment of a numerical forecast rather than start from the very beginning of the forecast in order to save computing time. He re-started his new forecast using numbers taken from a paper printout that were given to three decimal places. However, the computer hardware was using numbers to six decimal places. He found that his new forecast soon started to diverge significantly from the original despite the fact that the difference in the fourth and subsequent decimal places between the new initial conditions and the original numbers was actually far smaller than the accuracy to which the meteorological variables could be measured.

One consequence of this realisation is that a tiny perturbation to the atmosphere could result in a very different evolution of the weather systems around it. The most often quoted example of this is that a butterfly flapping its wings in Brazil could cause a tornado in Texas (an example taken from the title of a talk given by Lorenz himself in 1972). Of course, it is impossible to perform an experiment with the real atmosphere in which a perturbed and unperturbed atmosphere can be compared to see how differently they evolve, but it is accepted as fact that small perturbations in a complex fluid system such as the atmosphere can indeed significantly affect the subsequent evolution of that system.

This tells us something quite profound about the nature of weather forecasting. However comprehensively we observe the state of the atmosphere at the initial time of a forecast, the instrument errors inherent in our measuring systems alone will mean that the forecast will start to diverge from reality as the forecast evolves in time. And, of course, the meteorological observations that we do have by no means give a comprehensive coverage of the entire atmosphere, as we shall see in Chapter 3. This means that every numerical weather forecast that will ever be made is doomed to become wrong at some point. Add to this the facts that the equations which we are using to predict the evolution of the atmosphere and the methods we use to advance the forecast forwards in time are also less than perfect and one starts to marvel, as Lorenz himself did, that any weather forecasting at all is even possible. However, on a day-to-day basis, it is clear that weather forecasts for the next few hours to days ahead can indeed demonstrate useful skill although there are times when they do go quite wrong, even at rather short lead times.

So every weather forecast will start to diverge from reality. Put another way, the atmosphere is inherently unpredictable. It would be nice to know at what point the divergence of the forecast from reality will start to become significant but this, too, is unpredictable and is itself dependent on the state of the atmosphere. In some conditions weather forecasts a week or more ahead turn out to be rather good, but in other conditions the forecast just one day ahead can be quite inaccurate. Forecasts for a location in the Middle East during July for instance can be extremely accurate perhaps 10 days ahead due to the very small variations in the weather in that part of the world during the summer. However, forecasts for somewhere like Iceland in November can be very wrong just one day ahead due to the strong variability in weather conditions over the North Atlantic on very short timescales.

Many studies have examined the limits of atmospheric predictability and the general consensus is that something around 14 days represents a limit for deterministic forecasts. What people usually mean by a ‘deterministic’ forecast is one that gives specific values for meteorological variables such as temperature, windspeed, rainfall amount and duration and so on. Strictly speaking, however, a deterministic forecast is one in which the future state of the system is predicted by extrapolating the current state forwards in time using a fixed law or set of laws. The 14-day limit will generally be lowered by the fact that the numerical models of the atmosphere we use for prediction are themselves imperfect and so 7–10 days is a more realistic estimate of the limits of predictability. This may itself be an overestimate in situations where the atmosphere is undergoing rapid changes and large fluctuations.

Forecasters would like to have a feel for how predictable the atmosphere is every time they issue a new forecast to their customers. They could express this knowledge in terms of confidence in their forecast or indeed limit the lead time of their forecast to be shorter than the predictability limit for that particular initial state of the atmosphere. However, the predictability of the atmosphere is itself unpredictable—it is flow dependent as well as depending on the quality of the observations used to set the initial conditions.

One approach to this problem, which is described in much more detail in Chapter 5, is the use of ensemble forecasts. Instead of running a numerical weather forecast once, from a single set of initial atmospheric conditions, an ensemble forecast involves running the forecast ‘many’ times, each with a set of initial conditions which are slightly perturbed from the original, but all equally likely given the available observations and their known errors. If all of the many forecasts produce a very similar evolution of the atmosphere through the entire forecast period then it can be assumed that the atmosphere is in a state which is not sensitive to small perturbations in the initial conditions and we can be relatively confident in the forecast. If, however, all the forecasts start to diverge rapidly from each other only a short time into the forecast period then we can assume that the atmosphere is in a rather unpredictable state and so express low confidence in the forecast or simply not issue a forecast at all beyond the point at which the forecasts start to diverge significantly. There is no way of knowing in advance which if any of the many forecasts will be the most realistic and it is quite possible for the atmosphere to evolve in a way which none of the individual forecasts (usually referred to as ensemble members) predicted. Hence, ensemble forecasting does not necessarily lead to more accurate forecasts but it is a method which allows forecasters to make an assessment of the predictability of the atmosphere each time they issue a new forecast. Another benefit of ensemble forecasting is that it allows the generation of probability forecasts, whereby instead of making a deterministic forecast of a particular event (e.g. ‘It will rain in London tomorrow morning’) a forecaster can issue a forecast of the probability of that event happening (e.g. ‘There is an 85% chance of rain in London tomorrow morning’). This might sound like hedging one's bets as a forecaster but probability forecasts have a real economic benefit to certain types of customer, as shown in Chapter 8.

2.2 The importance of observations in weather forecasting

It hardly needs to be stated that the availability of good quality observations of the state of the atmosphere, with appropriate temporal and spatial coverage, is crucial to the generation of skilful weather forecasts. The discussion above has focused upon the critical sensitivity of numerical weather forecasts to the specification of the initial conditions. It has also become clear, though, that no matter how good our observations are and however comprehensive their coverage is, it will still never be possible to produce perfect deterministic weather forecasts out to lead times of more than a couple of weeks.

The importance of good quality observations to the weather forecasting process is often most apparent in situations where weather conditions are changing rapidly and the atmosphere is in a ‘mobile’ state. The way that the initial conditions for a numerical weather forecast are set is discussed in detail in Chapters 4 and 5. But, put very simply, the initial conditions are set using a blend of the short-range forecast from the previous run of the computer model (known as the ‘background field’) and a combination of many different types of measurement. In situations where the state of the atmosphere isn't changing rapidly therefore, most of the important information will already be contained in the background field and the observations will only be making small adjustments to this state. A study in 2004 by Cardinali et al., using the European Centre for Medium-range Weather Forecasts (ECMWF) operational forecast model, looked at the relative amounts of information coming from the observations and the background field in the initial conditions of NWP forecasts. They showed that, on average, over the period of boreal Spring 2003, the amount of information contained within the initial conditions which was due to the observations was only 15%, with the other 85% coming from the background field. Of course, the background field is itself influenced by observations inserted during the previous runs of the model, so this 15% estimate is probably a lower bound estimate. Even so, this perhaps implies that the observations are of rather secondary importance. This is not true of course, as the 15% of the information which comes from the observations could be regarded as the most crucial 15%; the information we didn't know about prior to the new model run. If we never used any observations whatsoever and just started each numerical weather forecast from the background field then our forecasts would rapidly become useless. In fact, this would be exactly the same as just running one continuous forecast that never took account of any observations of the atmosphere, so after 7–10 days of forecast time (possibly sooner) we would run into the predictability limit.

The real importance of good quality observations becomes very clear in rapidly evolving situations when it is possible that the information coming from the observations might considerably exceed 15%. An excellent example of this comes from another study made using the ECMWF model by Leutbecher et al. in 2002. This study looked at the damaging depression that swept across Europe on the 26 December 1999—a storm named ‘Lothar’ by the Free University of Berlin. This storm caused extensive damage to buildings, electricity and telephone networks and forests across a broad swathe of France and Southern Germany; it led to 137 deaths. The costs of this storm were estimated to exceed US$ 10 billion. The low pressure centre developed initially on the western side of the Atlantic and made a rapid crossing of the ocean. It underwent rapid intensification off the west coast of France prior to making landfall and then tracked across Northern France and central Germany. Several operational forecast models failed to forecast its track, intensity and development at quite short lead times (1–3 days). Figure 2.1 shows the forecast position of the Lothar storm from the ECMWF model at a lead time of 48 hours, together with the actual position of the storm at this time. The forecast predicted a depression centred slightly to the southeast of its actual position, but this was not deep enough by 15 hPa. The result of this was that the predicted winds across Northern France and Germany were too weak in the forecast. The forecast also predicted a depression to the south of Ireland which was not there in reality.

Figure 2.1 Mean sea level pressure (MSLP) fields at 12:00 UTC on 26 December 1999 from (a) a T+48 hour forecast from the ECMWF operational model and (b) the verifying analysis, that is ‘truth’. (Reproduced by permission of ECMWF.)

The Leutbecher et al