Dice World E-Book

Brian Clegg

• Herausgeber: Icon Books
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 362

Veröffentlichungsjahr: 2013

Beschreibung

LONGLISTED FOR THE 2014 WINTON ROYAL SOCIETY PRIZE FOR SCIENCE BOOKS As troubling as we pattern-seeking humans may find it, modern science has repeatedly shown us that randomness is the underlying heartbeat of nature. In Dice World, acclaimed science writer Brian Clegg takes readers on an incredible trip around our random universe, uncovering the truths and lies behind probability and statistics, explaining how chaotic intervention is behind every great success in business, and demonstrating the possibilities quantum mechanics has given us for creating unbreakable ciphers and undergoing teleportation. He explores how the 'clockwork universe' imagined by Newton, in which everything could be predicted given enough data, was disproved bit by bit, to be supplanted by chaos theory and quantum physics. Clegg reveals a world in which not only is accurate forecasting often impossible but probability is the only way for us to understand the fundamental nature of things. Forget the clockwork universe. Welcome to Dice World, a unique portrait of a startlingly complex cosmos, from the bizarre microscopic world of the quantum to the unfathomable mechanics of planetary movements, where very little is as it seems...

Leseprobe

Printed edition published in the UK in 2013 by

Icon Books Ltd, Omnibus Business Centre,

39–41 North Road, London N7 9DP

email: [email protected]

www.iconbooks.net

This electronic edition published in 2013 by Icon Books Ltd

ISBN: 978-184831-564-8 (ePub format)

ISBN: 978-184831-565-5 (Adobe ebook format)

Text copyright © 2013 Brian Clegg

The author has asserted his moral rights.

No part of this book may be reproduced in any form, or by any means, without prior permission in writing from the publisher.

Typeset by Marie Doherty

Contents

Title page

Copyright information

Acknowledgements

Chapter 0: Alea jacta est

Chapter 1: Improbable world

Finding patterns

The patterns of science

The randomness confusion

Weighing up risk

From classical to chaos

Chapter 2: More random than random

The success factor

Random success

Superstition conjures causation

A natural cycle

Chapter 3: A measure for luck

The inhuman economist

Gambling with chance

Chapter 4: It’s all in the stats

What’s it worth?

The law of large numbers

Distributing the outcomes

Chapter 5: The clockwork universe

The universe according to Newton

No need for that hypothesis

Chapter 6: Just three bodies

Relativity becomes special

The eternal triangle

Chapter 7: Chaos!

The unpredictables

The next bestseller

Chapter 8: Statistical substance

Sample selection

Blinded by science

All in the context

Statistics aren’t fair

The rare event

Chapter 9: What does random mean?

Generating randomness

Cherry picking

The unbalanced target

How significant is significant?

Probability on trial

The sources of randomness

Chapter 10: Really random

Into the quantum atom

The light revolution

A case of uncertainty

Chapter 11: No quantum cats

In the quantum tunnel

Dead or alive?

Interpreting the quantum

Chapter 12: Improbable world redux

The quantum mechanical window

Einstein’s hidden truths

Quantum secrecy

Computing with quanta

Beam me up

Chapter 13: Follow the heat

Laying down the law

The law of change

Quantifying disorder

Entropy plays dice

Chapter 14: Maxwell’s demon

Unmixing the mixture

Uniqueness is disorder

The mystery of time

The clockwork runs down

Chapter 15: Crystal balls and winning goats

It’s in the stars

Future vision

A simulated world

Cars and goats

Born on a Tuesday

Chapter 16: The Reverend Bayes and the golden retriever

The case of the informative mug

Good guesses are better than nothing

Mr Bayes’s oracle

Chapter 17: Free will?

My lucky numbers

I had no choice

A spanner in the clockwork

The random deity

Index

Acknowledgements

For Gillian, Rebecca and Chelsea.

With particular thanks to Keith Rapley and the late John Carney, my mentors when working in operational research – arguably the ultimate discipline for applying probability and statistics to real-world problems.

Many thanks to all those who have helped make this book possible, especially Duncan Heath, my editor at Icon Books, and all the contributions from Andrew Furlow, Dr Peet Morris, Denni Saunders, Dr M.G. Harris, Cathy Murphy, Perry Rees, Katherine Kelly, Paul Tuck, Amanda Lever, Edward Cope, Helen Witney, Dr Henry Gee, Mats Anderson, Liz Warrick, Stacey Croft, Desney Harrington, Kathy Peacock, Sarah Mussi, Matt Brown, Euan Adie, Ethan Friedman, Lynn Price, Amy Cope, Chris Reeves, Diane Rendell, Sue Broughton, Mark Lloyd, David Howkins, Stephen Godden, Dr Harriet Dunbar-Morris, Oriana Morrison-Clarke, Stewart Desson and Henry Lord.

CHAPTER 0

Alea jacta est

The first stage of writing a non-fiction book is usually to sketch out a summary of each chapter that will go in the finished work. At that time, the possibility of writing this chapter had not occurred to me – hence it being chapter zero. The very existence of this chapter is a great example of random influences at work.

Shortly after I began my background research for Dice World, I read a book called The Black Swan by Nassim Nicholas Taleb. This was not part of my research – it was intended as bedtime reading, light relief from work. I was taking a look at a cult classic that I’d never got around to – I didn’t even realise at the time that the book featured chance and probability. It was something of a shock, given that I had already chosen the title Dice World for my own book, to find Taleb railing against the image of dice as a cultural reference for randomness. Taleb refers to this as the ‘ludic fallacy’, his idea being that dice represent the fake, controlled, predictable randomness of games, not the real, wild randomness of life.

To rub this in, Taleb tells his readers a story of two fictional characters faced with a classic game-based challenge that is often used to demonstrate the difficulty many people have with understanding probability. Let’s imagine we have a fair coin, which when flipped has a 50:50 chance of coming up heads or tails. On average, we expect the coin to come up heads half of the time and tails half of the time. We flip that coin 99 times in a row and get a head every single time. What’s the chance that we will also get a head on the next throw?

In Taleb’s story, the first character, an accountant, comes up with the standard ‘correct’ response you would get from anyone who has a good understanding of the basics of probability. On the hundredth throw, there is still a 50 per cent chance of getting a head. The coin has no memory. To think that somehow you are more likely to get tails this time because heads have come up so many times before is what’s known as the gambler’s fallacy. There is no connection between the 100th toss and the ones that came before it. You are starting from scratch with a single toss of the coin. The outcome is still 50:50 heads or tails.

The other character in The Black Swan, a city trader, thinks that this statistical view on life is rubbish. It’s not that he agrees with the gambler’s fallacy that the coin is ‘due’ to come up tails. He has no intention of falling into that trap. In fact, the trader will tell you, there’s a high chance instead of getting another head. And he is probably right. Why? Because in the real world, it is much more likely that the person who told us it was a fair coin was lying than it is that you will get 99 heads in a row from tosses of an unbiased coin. The real world is not like a game with nice, easily calculated probabilities and no outside influences. The real world cheats.

That is fair enough as a realistic observation of what the world is like, but I think that Taleb misses the point when it comes to using the image of dice as a device to suggest randomness. The dice are symbolic, they are not a computer model of real-world experience. The way that dice provide an illustration of nature’s indifference to our human affairs and desires is an ancient conception that Albert Einstein used as a parable for 20th-century science. Einstein’s various statements along the lines of ‘God does not play dice’ were used to illustrate his frustration with the way that quantum theory suggested the apparently predictable real world was in fact based on unpredictable chance. It was never intended to provide a detailed analysis of the different kinds of probability and randomness out there – just a nice picture to illustrate something out of our control. And it works.

The quote that provides the title of this mini-chapter, ‘alea jacta est’, attributed to Julius Caesar as he crossed the Rubicon and meaning ‘the die is cast’, illustrates exactly how the throw of the dice provides powerful imagery, in this case identifying a point of no return. To deny dice their figurative role seems unreasonable.

Symbols aren’t intended to be reality, and the resonant role of dice and other gaming mechanisms helps us grasp some aspects of probability, even if we have to then extend far beyond this to get a true picture of the real world. Denying their symbolism is a bit like complaining that the road sign showing a landslip only features a handful of rocks where a real landslip may well have millions of bits of debris in it. Symbols are not supposed to be the real thing. So this book proudly remains Dice World.

If you are a certain age (which I am), Dice World may also bring to mind another book, which was very popular when I was at university – The Dice Man by Luke Rhinehart. That novel featured a disaffected psychologist who decides to let dice rule his life, handing every decision he makes over to the throw of the dice to make the choice. For me it was a very sad and depressing novel: it is one thing to be aware of the significance of randomness in our lives – an important thing indeed. But it is something quite different to simply give away all rational choice to a random number generator.

Dice World is not the natural home of the Dice Man. Most of us don’t make our decisions by throwing dice. But like it or not, randomness can be effectively symbolised by a throw of the dice. And as we will discover, randomness is the underlying heartbeat of our universe.

Welcome to Dice World.

CHAPTER 1

Improbable world

The world is a complicated and messy place, especially when you consider the complexities we add to it with our carefully constructed environment. Take a really simple act that most of us perform every day without giving it a thought – switching on an electric light. This is clearly not something we are genetically programmed to deal with from birth. Human beings are pretty well identical to the creatures that evolved to survive on the savannah after their ancestors stopped living in trees over 100,000 years ago. Once you get beyond basic bodily functions and activities, the vast majority of our time in the modern world is spent doing things that the human body did not evolve to do. All the rest of our activities and experiences are relatively newly learned. We live unnatural lives.

It’s certainly true that there weren’t many light switches 100,000 years ago. So we all have to learn how to turn the light on – and for most of us (until we venture across the Atlantic and find that they incomprehensibly mount their switches the wrong way up on the wall) it is a natural-seeming, easy act. We flick the switch and the light comes on. No real thought involved. It’s trivial.

But imagine that you had to program a robot from scratch to switch on the light in your living room. You would need to specify exactly where the switch was located. This would involve providing detail of where each wall was, which wall the switch was on, at what height it was located and at what distance it was from the wall’s edge. Alternatively you would need to show your robot exactly what the switch looked like from every possible angle, so the robot could search for it visually. You would also need to specify where and in which direction to apply pressure to the switch, how much pressure to use (it would be embarrassing if the robot snapped the thing off) and when to stop pressing.

What seemed trivial turns out to be anything but a simple task. But more to the point, if you now moved that robot into the hallway and asked it to carry out the same job there, you would have to start all over again. There might be a totally different design of switch with dissimilar physical characteristics. It’s highly unlikely this new switch would be in the same place on the wall in the hall as the switch is in the lounge. Set the robot in action without reprogramming it and you would probably end up with a hole punched in the plaster.

As human beings, we simply can’t afford the time and effort to do the equivalent of re-programming our brains each time we encounter a different light switch. And so we deal with patterns. We don’t learn exactly what each light switch that we encounter is like. Instead we have a broad pattern in mind which specifies ‘This is how you switch on a light using a wall switch’. It enables us to recognise the switch in a broad range of styles and then just to do it – press the switch, get the light. Until some clever designer comes up with a switch that works when you speak to it or touch the lamp itself – and then you have to start the discovery process all over again.

Finding patterns

Of course, we didn’t evolve an ability to recognise patterns to cope with light switches. But exactly the same flexibility of pattern-matching enables us to spot a predator – or a familiar friendly face – even if we have never been in a particular exact circumstance before, and so to take appropriate action. We work with patterns that give us the ability to reduce the almost infinite set of possible deductions from our sensory inputs to a manageable set we can work with, using the mental shorthand that enables us to just ‘flick the switch’, ‘run from the tiger’ or ‘see and say “Hi” to Nic.’

We are so good at this pattern-matching that we can achieve it even when we have a surprisingly low amount of information on which to make a judgement – in this we are usually a lot better at filling in the gaps than computers are. This is why the ‘CAPTCHA’ system, used by websites to ensure that people are taking part rather than software programs, makes use of distorted text with characters that are twisted or run into each other. This is a visual input that a human can usually interpret, but a piece of software struggles with turning into useful data.

Take the three partial pieces of text below:

No one would be challenged to see that the top word reads ‘BANK’, even though a sizable percentage of the text is missing. We find it trivial to fill in the gaps. In the second example, a whole 50 per cent of the text has been chopped off, but there is still enough there to be sure what the word is. It is only the positioning of the final chop, introducing ambiguity with two possible interpretations of ‘BANK’ or ‘RANK’, that finally beats our superb ability to take a partial pattern and reconstruct the whole.

Much of the time this human ability to detect patterns is a real plus. It means that we can work with limited data – and in the real world the set of data that we have available is almost always incomplete. But the danger we face is that the pattern-constructing and -matching systems of our brains are so powerful that we imagine patterns when there is nothing there.

This is a good survival principle. It’s better to be sufficiently sensitive that you occasionally see a predator where there isn’t one, rather than risk missing a killer that is lurking in the bushes. So we create bogeymen out of shadows and misinterpret all kinds of evidence. We see faces in the shadows, in the clouds, or even in the burn marks on a slice of toast. Pure randomness with no pattern is something we find difficult to relate to – our brains expect to see patterns and they do.

The patterns of science

This pattern-matching isn’t just about our low-level, immediate, day-to-day interaction with the environment around us (important though that is). It is also the basis of science. It’s strange, in a way, that many of us struggle with science because all the scientific method does is to take the basic mechanism we all use to understand the world without even thinking about it, and formalise that mechanism into a process.

In science we are looking for patterns and rules to explain what the universe and its components do and how they do it. It’s as simple as that. The mechanisms modern scientists use may get heavy-duty and scarily mathematical, but the basic principle is still one of looking for patterns. What scientists do is arguably just a simple and rather beautiful formalisation of our natural approach to exploring the unknown.

We start off in a state of ignorance. We gather enough data to be able to formulate a hypothesis about what’s going on. Then we test that hypothesis – a kind of predictive pattern – against subsequent observations; if it continues­ to work, we can build on it. If it fails us, we have to start all over again. That’s the scientific method. It should be how we naturally interact with the world too, but all too often, once we get a hypothesis, we get fond of it. We can’t let it go despite plenty of evidence to the contrary. And that’s when science slips into superstition.

To have any hope of making a scientific approach work, we have to expect some degree of consistency of behaviour from the universe. Take something we think of as a constant, a fixed point of certainty – the speed of light. If this varied from day to day or second to second with no logical reason for that variation, and no way of ever anticipating what the speed will be today, then we could never make use of the speed of light, as astronomers do all the time, to help us understand the universe. Given how much of our exploration of the universe is dependent on light and its speed, this would be totally devastating for cosmology. In fact, without a degree of consistency, the whole concept of science would collapse. We would live in a universe that might as well be magical. It is impossible to draw any hypotheses if every time you do an experiment you get totally different results.

This doesn’t mean that there won’t be circumstances when the speed of light does vary. We know that it is different in a vacuum from when it is passing through a substance – it is slower in air, still slower in water and so on. There are even substances called Bose Einstein condensates that can effectively bring light to a standstill. This is because photons of light don’t pass through matter unaffected but interact with electrons, being absorbed and re-emitted, slowing down their progress. But this isn’t a problem for science, because these variations are predictable. I know that the speed of light is different when it’s going through space than from when it’s going through glass. But for the same medium under the same conditions, I expect to get the same result.

I chose the speed of light intentionally because there is even a theory (a perfectly reasonable theory, though not one with a lot of support at the moment) that the speed of light has not stayed the same over time. According to this theory, over the billions of years of existence of the universe, the speed of light has changed very slightly. If this is true, while it would modify some of our conclusions about exactly what was happening long ago in galaxies far, far away (as they say), it too wouldn’t be a huge problem for science, because it is something we could predict and consider the influence of over time.

The randomness confusion

There is, however, one aspect of dealing with reality where our superb pattern-forming skills totally let us down. This is where there genuinely is no pattern; where there is no logic that lets us work out what will happen next; where randomness rules and chaos ensues. The good news is that for many of our basic interactions with the world, repeatability is the name of the game and randomness is under control. But whenever we are dealing with the odds in a game of chance, or the discovery that every aspect of the universe at a fundamental level relies on randomness, we have a serious problem of understanding because our pattern-forming brains start floundering.

Just listen to the victims after a disaster has occurred. They will almost inevitably ask ‘Why?’ – Why us? Why here? Why now? We all want to find a pattern. We want a reason. But usually with this kind of event there isn’t one. The event itself will have a cause, but there is no reason for the ‘Why us?’ type questions. Just imagine, as sometimes happens, that a child has been struck by lightning, or swept away by a flash flood. I have no doubt that his or her family would be asking ‘Why us, when there are so many families who don’t have to suffer this?’ We struggle so much to accept that any event can be the result of true randomness. Many in the past have invoked wrathful deities to explain an outbreak of sickness in their village, blaming it on the bad behaviour of the inhabitants. Such reasoning doesn’t make any sense, but it establishes a pattern.

Even now, in the 21st century, this can happen. In 2010 an Iranian cleric announced to the world that women who wear unsuitable clothing or behave promiscuously are to blame for the incidence of earthquakes. ‘Many women who do not dress modestly … lead young men astray, corrupt their chastity and spread adultery in society, which increases earthquakes,’ said Hojatoleslam Kazem Sedighi, according to the Iranian media. Sedighi, who was responding to an announcement from Iran’s president Ahmadinejad that Tehran was at risk of being hit by an earthquake, told his followers, ‘What can we do to avoid being buried under the rubble? There is no other solution but to take refuge in religion and to adapt our lives to Islam’s moral codes.’ The pattern is coming to the fore again: the earthquake and the suffering it brings has to have a cause, and obviously it is the behaviour of women that is causing it.

We want a pattern, but so often, everything from dramatic real world events to the weird world of atoms and subatomic particles is governed by a randomness that makes our brains hurt. There may be causes, as there certainly are with earthquakes, but the patterns they form may be impossible to detect with any accuracy because the system involved is too complex and chaotic. Or there may be no cause to an event at all, as is the case with the point in time that a specific radioactive atom decays. Either way, if we decide there is a pattern, we are deluding ourselves.

Weighing up risk

Our inability to deal with chance is illustrated beautifully by the way that, time and again, all of us fail when trying to weigh up risk. Take the following example. Let’s say that you hear on the news that a taxi driver has been arrested for attacking a young female passenger. (This happened in my town a couple of years ago.) For the next few weeks, if you have a daughter or know someone who seems a potential victim, you will be reluctant to let them take a taxi. It’s human nature, you might say. And it is. But in terms of risk, human nature is getting the response profoundly wrong.

Generally speaking, the risk of being attacked by a taxi driver is very low in the UK. Many thousands of journeys are taken every day without a problem. It is only awareness of the news that has made the risk suddenly seem higher. In fact, the risk has actually just gone down substantially. Because the only taxi driver in the area known to attack female passengers is now in custody. It is safer to take a cab than it has been for months – and yet human nature, a commonsense reaction to a scary potential pattern, is to feel that it is more dangerous and to offer to drive your daughter everywhere. This is a good example that makes it clear just how much this problem influences all of us. A statistician will understand perfectly well that there is no extra danger and that the risk is very low. But they are still likely to warn their daughters to be careful in such circumstances. It’s not rational, but it is human.

For a second example, let’s think of the more large-scale risks that human beings face. When we attempt to estimate risk, we tend to give excessive weight to possibilities with a clear and familiar cause rather than an open, generic risk. Children are far more likely, for instance, to be killed by traffic than by predatory human beings, but we can often focus more on the dangers of paedophiles than on those of traffic because these individuals present a threat that features more in the stories we are told by the media than the less dramatic but much more deadly traffic statistics. Unfortunately for our children’s safety, paedophiles make better news stories than traffic accidents.

Randomness confuses the hell out of human beings. Given our dependence on patterns as outlined above, and given that randomness is, in effect, the absence of pattern, we are led inexorably to a difficulty of dealing with randomness. That’s a problem, because, as we will discover in this book, different aspects of randomness, chance and probability lie behind most of the world we experience. Patterns are, if anything, an oddity in the universe. Randomness is the norm.

From classical to chaos

There are broadly two types of randomness – the classically random and what I will describe as chaotic randomness. ‘Chaos’ is a term that is bandied around a lot. I am employing this term – or more specifically the phrase ‘chaotically random’ – in a more particular sense than the English language term that simply means disordered, but in a broader sense than the kind of chaos theory that was encountered in the movie Jurassic Park and that lies behind our inability to predict the weather beyond a few days out.

These two categories, classical and chaotic randomness, are rich in paradox. Individual events in classical randomness are impossible to predict, but the overall behaviour of a collection of classically random objects (like a gas of atoms), obeys rules – a so-called distribution – that makes it relatively easy to decide what will happen to the collection in the future. An obvious example of classical randomness is a typical gambling game (assuming that the game isn’t rigged). The outcome should be truly random, but the odds of winning are clear and will predict statistically, over time, what is going to happen. The rules of classical randomness can’t tell you the result of a specific game, but they can tell you how the different outcomes should be distributed.

So, for example, taking the totally skill-free game of roulette, there are eighteen black and eighteen red slots on the roulette wheel into which the ball can drop, so if we ignore for the time being the green 0 or 00 slots where the house wins, there should be an 18⁄36 (or 1 in 2) chance of winning if you bet on either black or red.

I’ll just take a moment to look at the different ways of representing probability, as there are surprisingly many different ways of saying the same thing. If something has a 1 in 4 chance of occurring, then on average, one time out of four tries will produce this result. Think of drawing a playing card from a well-shuffled pack with no jokers. There is a 1 in 4 chance of getting any particular suit. We can also represent this as a fraction. So we could say that there was a 1⁄4 or 0.25 probability of getting, say, a heart. The probability varies between 0: no chance at all (the chance, say of drawing a card from a nonexistent fifth suit) to 1: will definitely happen (the chance of drawing a card with anything printed on it from the standard 52 cards).

We can also use a percentage, which is just the same as the fraction, but multiplied by 100. So our 0.25 chance becomes a 25 per cent probability. In 25 per cent of cases, for instance, we expect to draw a heart. Gamblers use odds, which are, if anything, more confusing than probabilities. So we might say the odds of drawing a heart are 3 to 1 (often written as 3⁄1) because you are three times as likely to get one of the other suits as you are to get a heart. A final way of representing a chance is to use a ratio of percentages. You might say the chances of drawing a heart are 25:75 in that you would get it right 25 times to 75 times you got it wrong, though this representation is almost always used when there are equal chances of getting one thing or another, making it a 50:50 chance. This is the same as a 1 in 2 chance, or a probability of 1⁄2, 0.5 or 50 per cent. In betting terms it is an evens chance.

Back with our roulette wheel and its 1 in 2, or 50:50 chance of winning or losing. This isn’t a safe enough bet for casinos, which as businesses want to make sure that they will get a profit. So they add a zero slot (often there are two of them on wheels in the US). If the ball ends up in this slot, no one wins but the casino. It should be crystal clear for players what this means – long term, the casino will win. But of course this doesn’t mean that lucky players can’t clean up – as long as they stop while they are ahead.

A roulette wheel is a physical device, and as such is not a perfect mechanism for producing a random number between 1 and 37 (or 38 in the more money-grabbing US casinos). Although wheels are routinely tested, it is entirely possible for one to have a slight bias – and just occasionally this can result in a chance for players to make a bundle­. It certainly did so for 19th-century British engineer Joseph Jagger, who has, probably incorrectly, been associated with the song ‘The Man Who Broke the Bank at Monte Carlo’, which came out around the same time as Jagger had a remarkable win in Monaco. The song probably referred instead to the conman Charles Wells, who won over a million francs at Monte Carlo and did indeed ‘break the bank’. (This doesn’t mean that he cleaned out the casino, simply that he used up all the chips available on a particular table.) There were many attempts to find how Wells was cheating, but he later admitted that it was purely a run of luck, combined with a large amount of cash enabling him to take the often effective but dangerous strategy of doubling the stake on every play until he won.

However, Jagger probably deserves the accolade more than Wells, as his win was down to the application of wits rather than luck – and was even more dramatic.* Jagger finally amassed over 2 million francs – the equivalent of over £3 million (or US$5 million) today. He hired a number of men to frequent the casino and record the winning numbers on the six wheels. After studying the results he discovered that one wheel favoured nine of the numbers significantly over the rest. By sticking to these numbers he managed to beat the system until the casino realised it was just this wheel that was suffering large losses and rearranged the wheels overnight. Although Jagger soon tracked down the wheel, which had a distinctive scratch, the casino struck back by rearranging the numbers on the wheel each night, making his knowledge worthless.

In a casino, with the odd exception like Jagger, chance is under tight control, but when the second type of randomness, chaotic randomness, is in action, control rarely lasts long. Where classical randomness involves truly unpredictable individual events, chaotic randomness isn’t actually random at all. Think, for example, of attempts to predict when an earthquake will occur or what the weather will be weeks ahead. The individual events in such chaotic systems are predictable, but in practice the interactions of the elements in the system are so complex that it only takes very small changes when chaotic randomness is in play to make massive differences in outcomes.

Where the random events of classical randomness come together to form a predictable distribution, chaos refuses to be so easily predicted. In chaos, individual items don’t fit into neat distributions, meaning that chaos can bring with it huge surprises – what Nassim Nicholas Taleb refers to as ‘Black Swans’. In a sense, chaotic randomness is far more random than true, classical randomness.

* I have always found it bizarre that casinos consider it to be cheating if players use skill to win. Unless there is a fault with the wheel, there is no skill in roulette, but there certainly can be in games like blackjack, where there are a limited number of cards available to play, and so by counting the cards that have already been dealt, a player with a superb memory (or a concealed computer) can increase their chances of winning. Apparently you get thrown out of casinos if you are caught doing this and barred from re-entering. Imagine if athletes were banned from their sport if they showed any sign of skill. It just demonstrates what should be obvious: casinos aren’t a way of playing fair games, they are businesses designed to take money off people.

CHAPTER 2

More random than random

Both types of randomness – classical and chaotic – can catch us out, but there is nothing more dangerous than when we try to apply the rules of one kind of randomness to the other. The distributions of classical randomness work well in physics, or to describe how the heights of human beings vary. They describe perfectly the outcome of a fair casino game. But if we try to use these distributions to predict the behaviour of the kind of chaos that lies behind an earthquake, we will eventually come a cropper – and because of our pattern-seeking natures, we can easily rationalise a chaos moment away, pretending that it is just a blip we can ignore in an otherwise classical distribution, rather than the definitive action of a chaotic system.

Take an example derived from Bertrand Russell, used by Taleb in The Black Swan. Consider the life of a turkey. This is a particularly thoughtful turkey, which makes predictions about its future happiness and well-being. It looks back over its life so far and sees a normal distribution (of which more later) of good and bad days. On the whole, by assuming it is dealing with classical randomness, it can predict the range of its positive and negative experiences, and how the good and bad days will be distributed. And then Christmas comes. Chaos intervenes. There comes a point that is way off the scale as far as all its experience to date goes.

We are good at being like this turkey. Every business puts a huge amount of effort into putting together a budget and forecasting what its performance will be like over the next year. There are even painful post-mortems, examining why reality differed from the prediction. These businesses (and you can apply the same picture to economists and politicians) are turkeys, merrily predicting the future from the past and getting upset when chaos steps in. It’s not your forecast that’s wrong, guys – no need to have a post-mortem – it’s the assumption that you can make effective predictions in most real-life circumstances.

To take on a more life-and-death example, think of a plane crash. This is another case of a chaotic inter­vention in the generally classical distribution of experiences­ of flight. Here we can see a particular danger­ that emerges from our inability to handle this kind of randomness. We think plane crashes are much more likely to happen than they really are. This is because we are presented with them much more often in the media than their impact deserves. As a result we are much more scared of travelling by plane than we are of going on a road.

Typically there are one or two thousand deaths in plane crashes worldwide each year (many of them in smaller airliners that most of us don’t use). By comparison, around 1.25 million people are killed on the roads each year around the world. Yet a combination of media exposure and our difficulty handling randomness means we get particularly scared of flying. I put my hands up. I hate flying. It just is scary if you think about what could happen in a plane crash. Yet experiencing this is a very unlikely occurrence.

What’s more, if a plane crash is reported on the news, your awareness of the dangers of flying goes up, just like the dangerous taxi driver effect in the previous chapter. There is no reason whatsoever for the risk to go up – but because we are more aware of the possibility, taking your next trip by plane feels more scary. A similar thing happened on a much larger scale after the 2001 attack on the World Trade Center in New York. In reality, because of the security clampdown, terrorist incidents were much less likely immediately after this event, yet everyone felt that they were at greater risk, thanks to the human inability to deal with randomness and chaos.

The success factor

One major implication of our difficulty with chaotic randomness is that we ascribe much to talent that, in activities where chaos has the upper hand, really should be allocated to luck. Success as an author, or investing in stocks and shares, or in running a large company is primarily down to how well the books, investments or companies fare in terms of what chaos can throw at them. The big spikes (the equivalent of the turkey’s Christmas) far outweigh the subtle influence that having extra-special talent on board can bring to the party.

This doesn’t mean the inverse – that it’s easy to succeed with no talent – is true (despite the evidence of some TV celebrities). There is usually a competence threshold below which bad performance makes failure inevitable, but as long as you have a reasonable level of competence it is chaos that determines who will be labelled the super-talented and the big successes in businesses where chaotic randomness reigns. This isn’t true of all business activities at all times of course. It’s interesting to look at chain restaurants as an example that is, for the moment, relatively free from chaos.

If you run a McDonalds franchise you will probably exist in a comfortable existence of relatively small variations from prediction. Your budgets will be meaningful. While the exact numbers of customers will vary randomly, it will be within easily maintained limits, predicted by a handy distribution. In ordinary circumstances, demand for hamburgers has no good reason to be chaotic. Hamburger restaurants can fail, but it can usually be put down to clear causes, like having the wrong location. With an appropriate footfall and the right product at the right price, people will buy hamburgers. However, you shouldn’t feel too safe. The life of turkeys is inevitably chaotic, but for the hamburger restaurant a major change in the environment can introduce chaos. If you have a major outbreak of salmonella, your business could disappear. Or the council could close the road you are based on, taking away your trade.

Longer term, even with such a solid business, the world could change, producing a dramatic move away from your products. Could anyone have imagined in the 1980s that Kodak would go into bankruptcy? Yet changes outside of the company’s ability to predict took away its trade. Kodak was guilty of thinking that the chaotic intervention of digital photography was a blip that could be ignored. It thought it knew the photography business like no one else. Instead, the introduction of digital produced a transformation of the industry. Kodak moved too slowly and too late and suffered as a result.

The way that chaos creates apparent experts that in fact have no expertise can be demonstrated in the form of a classic scam. Let’s say you were an unscrupulous person and wanted to make a lot of money from punters who like to gamble on the horses. You can offer them a prediction technique that seems to guarantee sure-fire success. You can sell people a scheme that genuinely enables you to predict the winners of a whole string of races correctly.

First, select a series of races in each of which there is only a small number of runners. For simplicity of doing the sums, I’m going to imagine each race has just four horses in it. Now advertise your amazing betting system. And wait for the punters to pay you a large amount of money for the output of randomness. It works like this. You will provide people with a prediction of winners – they pay you for the advance knowledge. To prove your system works, you will give them the first four winners for free. After that, they have to pay you £1,000 a time.

Let’s say 4,096 people sign up for the first free prediction (you may well get many more to do this – it’s free, after all – but it’s a convenient number for the example). You simply split your punters into four groups, giving 1,028 customers the prediction that the first horse will win, the next 1,028 that the second horse is the winner and so on. At the end of this stage, you will have 1,028 punters who received a successful prediction. Discard the rest. Now repeat with your successful punters in the next race, dividing the 1,028 into four groups of 256 and predicting a different winning horse to each group.

Of the punters in the second race, 256 will be winners. Repeat this process again, tipping each of the four horses in the next race to 64 of your remaining punters. Then repeat again in the fourth race, dividing your remaining 64 punters into four groups of sixteen. Sixteen people will once again be correctly told the winner. These sixteen people will now have been given the winners to four races in a row. A fair number of them will have enough faith in your system to pay up for your next random prediction.

There are a number of ways to run the scam. You could, for instance, charge an increasing amount for each prediction, refunding the failures. But however the finances are organised, with a large enough set of punters at the end of the process, you will have a group of people who are convinced that you are an absolute genius. Because you will genuinely have perfectly predicted the winner in race after race. In reality, you had no skill, no talent whatsoever (except a talent for deception). The only reason­ you appeared to predict the answer is that the sixteen remaining punters were lucky enough (in this case, just a 1 in 256 chance) to be in the winning group each time.