For Your Information E-Book

Philip Tetlow

• Herausgeber: Van Haren Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 765

Veröffentlichungsjahr: 2024

Beschreibung

In recent times, physicists have come to appreciate information’s central role in the universe’s grand plan. That and the fact that an explicit understanding of the informational relationships involved may well be key to unlocking many of the universe’s deepest secrets. That makes the birth of both Computer and Information Science not only essential to the explosion of modern technological success, but also to our understanding of reality itself. In recognizing that, what unfolds is a story not only about Alan Turing and his pioneering colleagues, but also great thinkers like Albert Einstein, Michael Faraday, Ludwig Wittgenstein and others. It therefore pulls in much of modern history and touches on seminal events like the birth of the atomic bomb. It also hints at the reasons behind the various social and political divides we see in the world today. So, in many ways, the story of how we became more informed about information is also the story of the modern age. What you will read of here is the role that information plays in that ongoing saga and many of the twists and turns that have brought us to where we are with information today. In it you will learn that, unbeknown to Turing and others, their work would not only help overthrow the Nazis and thaw the chilling atmosphere of the Cold War to come, but also echo down the ages to remain relevant in a conflict still raging today. That sees the Computer and Information Scientists at loggerheads as they fight to find a right and justifiable place for meaning in information’s definition. About The Open Group Press The Open Group Press is an imprint of The Open Group for advancing knowledge of information technology by publishing works from individual authors within The Open Group membership that are relevant to advancing The Open Group mission of Boundaryless Information Flow™. The key focus of The Open Group Press is to publish high-quality monographs, as well as introductory technology books intended for the general public, and act as a complement to The Open Group standards, guides, and white papers. The views and opinions expressed in this book are those of the authors, and do not necessarily reflect the consensus position of The Open Group members or staff.

Leseprobe

For Your Information

The Open Group Publications available from Van Haren Publishing

The TOGAF® Standard, 10th Edition:

Introduction and Core Concepts

Architecture Development Method

Content, Capability, and Governance

Leader’s Guide

ADM Practitioners’ Guide

Business Architecture

Enterprise Agility and Digital Transformation

A Pocket Guide

TOGAF® Business Architecture Foundation Study Guide

TOGAF® Enterprise Architecture Foundation Study Guide

The TOGAF Series:

The TOGAF® Standard, Version 9.2

The TOGAF® Standard, Version 9.2 – A Pocket Guide

TOGAF® 9 Foundation Study Guide, 4th Edition

TOGAF® 9 Certified Study Guide, 4th Edition

The Open Group Series:

The IT4IT™ Standard, Version 3.0

IT4IT™ for Managing the Business of IT – A Management Guide

IT4IT™ Foundation Study Guide, 2nd Edition

The IT4IT™ Reference Architecture, Version 2.1 – A Pocket Guide

Cloud Computing for Business – The Open Group Guide

ArchiMate® 3.1 Specification – A Pocket Guide

ArchiMate® 3.2 Specification

The Digital Practitioner Pocket Guide

The Digital Practitioner Foundation Study Guide

Open Agile Architecture™ – A Standard of The Open Group

Hospital Reference Architecture Guide: The Complete and Expanded English Translation of the Dutch ZiRA

The Open Group Press:

The Turning Point: A Novel about Agile Architects Building a Digital Foundation Managing Digital

Ecosystems Architecture

For Your Information - About Information, the Universe, and the Modern Age

The Open Group Security Series:

O-TTPS™ – A Management Guide

Open Information Security Management Maturity Model (O-ISM3)

Open Enterprise Security Architecture (O-ESA)

Risk Management – The Open Group Guide

The Open FAIR™ Body of Knowledge – A Pocket Guide

All titles are available to purchase from:

www.opengroup.org

www.vanharen.net

and also many international and online distributors.

Title:

For Your Information

Subtitle:

About Information, the Universe, and the Modern Age

Author:

Philip Tetlow

Series:

The Open Group Press

A Publication of:

The Open Group

Publisher:

Van Haren Publishing, ’s-Hertogenbosch, www.vanharen.net

ISBN Hard copy:

978 94 018 1224 5

ISBN eBook (pdf):

978 94 018 1225 2

ISBN ePub:

978 94 018 1226 9

Edition:

First edition, first impression, August 2024

Layout:

The Open Group

Copyright:

© 2024, The Open Group. All rights reserved.

Document Number:

G245

Published by:

The Open Group, August 2024.

Comments relating to the material contained in this document may be submitted to:

The Open GroupApex PlazaForbury RoadReadingBerkshire, RG1 1AXUnited Kingdom

or by electronic mail to: [email protected]

Cover image by Clarisse Croset, souce https://unsplash.com/photos/two-girl-illustrationstikpxRBcsA used with permission under the Unsplash License.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior permission of the copyright owner. Specifically, without such written permission, the use or incorporation of this publication, in whole or in part, is NOT PERMITTED for the purposes of training or developing large language models (LLMs) or any other generative artificial intelligence systems, or otherwise for the purposes of using, or in connection with the use of, such technologies, tools, or models to generate any data or content and/or to synthesize or combine with any other data or content.

Table of Contents

Preface

Trademarks

Acknowledgements

Foreword

Prologue

Way Points

Part I: History and Science

1. Bring on the Information Cake

2. Back to the Beginning

3. There, Within the Name

4. Princeton and Bell Labs

5. Meaning

6. Logic and Common Foundations

7. The World War Web

8. The Quantum Revolution

9. Loop Quantum Gravity through Quantum Information

10. The Post-Internet World

11. Modern Thinking on Networks and Information

12. Intelligence Paranoia and the Arrival of the Digital Computer

Bridge

Part II: What Does This Teach Us?

13. Physics is a Kind of Language

14. Conservation

15. Information’s Needle

16. Information’s Thread

17. Semantics in Physical Reality

18. Bringing It All Together

19. Epilogue

Appendices

Appendix A: The Mathematics Involved

Appendix B: Referenced Documents

Index

Preface

The Open Group Press

The Open Group Press is an imprint of The Open Group for advancing knowledge of information technology by publishing works from individual authors within The Open Group membership that are relevant to advancing The Open Group mission of Boundaryless Information Flow™. The key focus of The Open Group Press is to publish high-quality monographs, as well as introductory technology books intended for the general public, and act as a complement to The Open Group standards, guides, and white papers. The views and opinions expressed in this book are those of the authors, and do not necessarily reflect the consensus position of The Open Group members or staff.

The Open Group

The Open Group is a global consortium that enables the achievement of business objectives through technology standards and open source initiatives by fostering a culture of collaboration, inclusivity, and mutual respect among our diverse membership of more than 900 organizations. Our membership includes customers, systems and solutions suppliers, tool vendors, integrators, academics, and consultants across multiple industries.

The mission of The Open Group is to drive the creation of Boundaryless Information Flow™ achieved by:

•   Working with customers to capture, understand, and address current and emerging requirements, establish policies, and share best practices

•   Working with suppliers, consortia, and standards bodies to develop consensus and facilitate interoperability, to evolve and integrate specifications and open source technologies

•   Offering a comprehensive set of services to enhance the operational efficiency of consortia

•   Developing and operating the industry’s premier certification service and encouraging procurement of certified products

Further information on The Open Group is available at www.opengroup.org. The Open Group publishes a wide range of technical documentation, most of which is focused on development of standards and guides, but which also includes white papers, technical studies, certification and testing documentation, and business titles. Full details are available at www.opengroup.org/library.

About the Author

Philip Tetlow, PhD, C.Eng, FIET, is an Executive IT Architect, Web Scientist, and onetime W3C member. He has been a Vice President of IBM’s Academy of Technology, an IBM Quantum Ambassador, and is currently CTO of Data Ecosystems at IBM (UK).

He has more than 30 years’ experience in the IT industry, specializing in the delivery of large complex IT systems centered on data and analytics, and has been a key contributor to many important discussions on data and information.

He is a Visiting Professor of Practice at Newcastle University and an Adjunct Professor of Web Science at Southampton University. In 2017, he gave the TED talk on 8 Steps to Understanding Information — and Maybe the Universe, and he has twice won the IBM Academy of Technology’s President’s Award.

In 2008, his first book The Web’s Awake won an honorable mention in the Association of American Publishers Awards. It then went on to score 9 out of 10 in a British Computer Society review and currently holds a 5-star rating on Amazon UK.

Dedications

For Barrie, John, Yorick, Peter, Sal, and all that keeps me grounded. Here’s looking forward.

Trademarks

ArchiMate, FACE, FACE logo, Future Airborne Capability Environment, Making Standards Work, Open Footprint, Open O logo, Open O and Check certification logo, OSDU, Platform 3.0, The Open Group, TOGAF, UNIX, UNIXWARE, and X logo are registered trademarks and Boundaryless Information Flow, Build with Integrity Buy with Confidence, Commercial Aviation Reference Architecture, Dependability Through Assuredness, Digital Practitioner Body of Knowledge, DPBoK, EMMM, FHIM Profile Builder, FHIM logo, FPB, IT4IT, IT4IT logo, O-AA, O-DA, O-DEF, O-HERA, O-PAS, O-TTPS, Open Agile Architecture, Open FAIR, Open Process Automation, Open Subsurface Data Universe, Open Trusted Technology Provider, Sensor Integration Simplified, Sensor Open Systems Architecture, SOSA, and SOSA logo are trademarks of The Open Group.

Altair is a trademark of Altair Engineering, Inc.

AltaVista is a trademark of Yahoo Holdings, Inc.

Amazon is a trademark of Amazon.com, Inc.

Apple, HyperCard, iPhone, and Macintosh are trademarks of Apple, Inc.

Commodore is a trademark of C= Holdings (formerly Commodore International B.V.).

D-Wave is a trademark of D-Wave Systems, Inc. eBay is a trademark of eBay, Inc.

Facebook and Instagram are trademarks of Meta.

Friends Reunited is a trademark of Friends Reunited Limited.

Google and YouTube are trademarks of Google, Inc.

HTML, W3C, and XML are registered trademarks of the World Wide Web Consortium.

IBM is a registered trademark of International Business machines.

Intel is a registered trademark of Intel Corporation.

Microsoft is a registered trademark and Bing is a trademark of Microsoft Corporation.

Mosaic is a trademark of The Mosaic Company.

Netscape is a trademark of Netscape Communications Corporation.

Olivetti is a registered trademark of Telecom Italia.

Motorola is a registered trademark of Motorola Trademark Holdings, LLC.

MTV is a trademark of Viacom International.

Snapchat is a trademark of SNAP Inc.

Tandy is a trademark of Tandy Corporation Ltd.

Twitter is a trademark of Twitter, Inc.

Wikipedia is a trademark of Wikimedia Foundation, Inc.

Yahoo! is a trademark of Yahoo, Inc.

All other brands, company, and product names are used for identification purposes only and may be trademarks that are the sole property of their respective owners.

Acknowledgements

First, I must thank the amazing Charles H. Bennett of IBM Research and the equally amazing Bruno Michel from TED. Although they probably don’t realize it, in their own way they both brought me to this book. Without their wise words and encouragement, none of this would have come to pass.

Next, I must credit the simply glorious words of James Gleick and Carlo Rovelli. Their books have taken me far beyond the world I expected and their unparalleled ability to explain the complex in accessible ways has helped me greatly. They are closely followed by a list of many other amazing authors, too long to mention. As I write, their various works litter my browser’s bookmarks and office shelves — a constant reminder to tidy up!

On to those who’ve been kind enough to share their insight. That’s quite a roll call in itself, containing many individuals who are exceptional in their own right. It includes the likes of David Kenyon and John Palser at Bletchley Park Museum, Professors Brian Randell and Cliff Jones at Newcastle University, Jamie Martin at IBM’s archives, William Hery, now retired from Bell Labs Innovations, and my old friends John Tait, Wendy Hall, and the late Yorick Wilks. They’ve selflessly given their time to help with investigations and been the error checkers, the panel of experts who helped point out the weaknesses of those intent on sensationalizing many well-known historic events and scientific facts. They represent what I hope is the best available means to set the record straight. It is their compass that I have followed.

Now the friends, colleagues, and confidants who’ve chivvied me on and provided the space to think properly. They know who they are, but I’ll call them out anyway. A round of applause please for Mark Moloney, Toby Dupont, Tadhg Murphy, John Handy-Bosma, Alastair McCullough, Peter Ford, Ken McCreash, Peter Appleby, Peter Smith, Richard Hopkins, Susan Schreitmueller, Andrea Martin, Rashik Parmar, John Cohn, Mike Hudgell, Dean Newton, Junaid Butt, Fausto Martelli, Nathan Sykes, Sergio Rodriguez, Mark Mattingley-Scott, Kugendran Naidoo, Dinesh Garg, Leigh Chase, Ed Pyzer Knapp, Nick Brönn, Kate Marshall, Barrie Thompson, Grace Brown, Chris Burton, Neal Fishman, and Charlie Drees. Where would the world be without the kinship of others.

Last, but of course not least, I simply must mention my family. Writing can be a lonely, indulgent task and I know it pulls me away from them. Yet still they stay close, always there, always supportive. I’m blessed and I know it. I love them dearly.

Foreword

Bletchley Park was the wartime home of the Government Code (GC) and Cypher School (CS) in the UK — now relocated to Cheltenham and renamed as Government Communications Headquarters the (GCHQ). For the last 103 years, this organization has been the UK’s principal signals intelligence organization. When, in 2015, I was appointed as Research Historian for the Trust, which now runs Bletchley as a visitor site, I expected to have to learn a great deal about codes and cryptanalysis. Eight years later, I do indeed know a lot about codes and, more correctly, ciphers. I can talk at length about Enigma, JN-25, FISH, and explain the intricacies of a Vernam Cipher or a One-Time-Pad.

However, the more I’ve studied the workings and wartime successes of GC&CS, the more I’ve come to believe that cryptanalysis is not the whole story. The purpose of Bletchley Park was to provide intelligence to Allied leaders and commanders; information derived from enemy communications. Much of this communication was indeed encrypted, but in many ways that was incidental. It wasn’t important because it was encrypted, it was encrypted because it was important. At its heart, the wartime GC&CS dealt with information — data, and although the volumes of that data were tiny in comparison to the scales dealt with in the modern world, they were, for the 1940s, both enormous and groundbreaking. Bletchley Park’s Hollerith machine section alone consumed over two million 80-column punch-cards per week — six tons of cards — all individually punched and processed by female operators. Intelligence data was also painstakingly recorded on millions of index cards, all ordered and cataloged to allow meaningful intelligence information to be stored and rapidly retrieved when required. Bletchley Park is famous for its machines, the Bombes or Colossi, but those machines were actually only cogs in what was an enormous human-mechanical information system.

Bletchley Park was also the home of a number of famous men and women, the great minds of their era, all of whom applied themselves to the various problems of reading enemy messages. Nevertheless, the great Codebreakers were not all the same. Some were lifers; men like John Tiltman, who while not a household name, was a giant in the world of cryptanalysis from the 1920s until the 1980s. Some, like Margaret Rock, came late to codebreaking, but stayed and saw out their careers with GCHQ. Others, meanwhile, were only accidental codebreakers; their wartime work being a short episode in careers which focused before and after the war on other things, albeit related things. In this category are Alan Turing, Bill Tutte, Max Newman, and others who form subjects of this book, along with their American counterparts, including Claude Shannon and John von Neumann. Each of these individuals was to an extent forced to interrupt their pre-war intellectual life to work on projects — either bombs or code — not of their own choosing. Each returned to their original lines of thought after 1945, but did so profoundly influenced by their wartime experience. My own reading about these characters led me to wonder about their wider world, the connections between them, and the influence of their wartime activities on their longer term grander ideas.

We had just produced a temporary exhibition about Bill Tutte, telling the story not only of his startling achievements as a cryptanalyst, but also his later influence on our modern world, when Phil Tetlow, the author of this book, arrived at Bletchley Park. Inspired in part by our exhibition, Phil took the bold step of walking up to our information desk and asking to speak to a historian. Several hours and multiple cups of coffee later, I discovered that I was not alone in my fascination with the information story at the heart of Bletchley, and with the extraordinary network of minds who worked in it and were influenced by it. Between us, we traced the threads of this story and it led us to a place 3,400 miles away: The Institute for Advanced Study (IAS) at Princeton University in New Jersey. This was housed in the 1930s in Fine Hall — then the mathematics department, now the home of the department of East Asian Studies — a beautiful Tudor-style building which is best known as one of the locations for A Beautiful Mind — the biopic of one of its later alumni, John Nash. The IAS in the 1930s was a vital nexus linking many of the characters in this book, including Turing, who worked there at the behest of Max Newman, as well as figures such as J. Robert Oppenheimer and John von Neumann, who would work on the other great intellectual endeavor of World War II, the Manhattan Project. The rolls of the IAS also include luminaries like Kurt Gödel, and of course Albert Einstein. Arguably, the meeting of these minds in the years before World War II and the interaction of these human particles, created a powerful intellectual reaction which brought about, and continues to underpin the modern post-industrial Information Age.

I’m a historian. I stopped studying mathematics at age sixteen — something I now live to regret — and while I can use a computer, I make no pretense at understanding how it works. Thus, while I’m able to recognize the intellectual phenomenon that these individuals represented, it appears in my mind rather like a misty forest viewed in the early morning sun: a vision of almost blinding light against which misty shadows are visible, but exact forms are difficult to make out. The author of this book, on the other hand, is powerfully equipped with the skills and tools of information science, mathematics, and physics to see past the light and fog and truly understand the forest. So, what this book does is to explore the thinking of each of these individually in order to draw out their powerful and fundamental ideas and call out the connections between them. In so doing, Tetlow builds a coherent history of the idea of information as it developed in the twentieth century, explaining at the same time both its power and its elusiveness as an idea which in many ways continues to resist precise definition. He therefore gives us a clear view of the role of these ideas in shaping the world we live in today.

However, he doesn’t stop there. The second part of the book goes on to look deeper into the role of information in our understanding of the universe and the often contradictory theoretical principles we use to understand it. Tetlow, therefore, argues that the universe itself is fundamentally informational.

This is not a book for the faint-hearted. The scope and complexity of the ideas it contains is extraordinary, ranging from subatomic particles, via micro-chips, to the World Wide Web and indeed the nature of the universe itself. We meet some of the greatest minds of the last 150 years along the way and seek to understand how they too saw the world and discussed and argued over its true nature. Many of the individuals in this story are now dead, but the story itself is only just beginning. This book presents an argument but does not necessarily provide a conclusion. Rather, what it provides is an introduction, an invitation to a new way of thinking about things which I hope will absolutely not be the last word on the subject. It asks, perhaps demands, all of us to go away and ponder where we go next. It opens with a reference to the great Douglas Adams. This is entirely appropriate as the second part of the book can fairly be described as a meditation on “life, the universe, and everything”. Adams, however, went on to point out that the answer “42” was really only the beginning of a broader search, not just for answers but also questions. I was deeply flattered to be asked to write the foreword for this book, as I feel that my contribution in the process was relatively minor. Nevertheless, my journey with Phil to this point has been a fascinating and inspiring one, and yet I am sure it is not over. Like this book, it feels like just the beginning of a great quest.

Read on and get on board; you don’t want to miss it.

Dr David Kenyon, Research Historian, Bletchley Park

July 2022

Prologue

In many ways, this is a personal story as it reflects on my career-long battle with the slippery subject of information. In other ways, not so much as it tells of something undeniably relevant to us all.

For my day job, I work at IBM® and advise big companies on how to get the most from their data. It’s interesting work. I get to think hard about some fascinating problems, while meeting lots of amazing people along the way. Many of them have changed the world with their data, and I know I’m lucky to be part of that. On occasion, I also get to contribute toward some rather influential discussions. For instance, I once helped persuade those who run the World Wide Web to think differently about how data changes the way it works.

Out of that came the field we now recognize as Sociotechnical Science [1][2][3][4][5][6], although some might know it better as Web Science [7][8][9][10]. Whatever the name, and however it came about, I’m just glad that I was able to assist in my own small way. Back then, in 2005-2006, it was all very new and, to me at least, in the excitement of fresh debate, some of the ideas being discussed felt almost trivial.

How wrong I was.

The point I’m trying to make has nothing to do with self-importance. Far from it. We were just a bunch of interested practitioners shooting the breeze about electronic data. No, all I’m trying to say is that something so apparently innocuous and innocent close-up can have a massive impact at scale. It’s about how data, or more importantly the idea of information, has grown to profoundly interest and influence us all. Or at least, that was the point I wanted to make when I started to write this book.

Back in those early days of Web Science, I was certainly naïve. I’d worked in and around information technology for some time and had studied the standard texts on data processing before that. So, I suppose I considered myself an expert. I thought I knew a thing or two about data and information.

Again, how wrong I was.

In truth, the warning signs were there much earlier. As an eager undergraduate, I can remember a conversation with my most senior tutor. Back then, I asked about the difference between data and information, and my question was met with a painful pause. That exchange still haunts me to this day, as I’ve eventually come to understand that, at the most fundamental level at least, we still don’t really understand what information is.

Hence the reason for this book. I’ve always felt it important to question the nature of matters central to our world, regardless of how abstract or surreal they might be. That’s what got me started, as I decided to just sit down and write to the point where I’d worked things out to my own satisfaction at least.

At first, I didn’t have anything specific or significant to say, but writing is something that comes naturally to me. Besides, I’d written books before and they’d gone down reasonably well. So, this time around I thought I’d start by trawling the archives and associated texts. Then perhaps I’d seek out the stories of those linked with Information Theory, and, where direct contact was no longer available, I’d reach out to the next best alternatives.

In the end, that’s exactly what I did, and to my astonishment, the results greatly exceeded my expectations. Sure, I’d mixed with people associated with historic figures like Claude Shannon, Alan Turing, and Ludgig Wittgenstein — all names you will meet as you read on. But, and it’s a big but, I’d never questioned them specifically about their relationships, or tried to fit their backstories into any broader narrative.

In my quest, I set off almost aimlessly and headed for the safety of the Information Retrieval community I knew well. There, I looked up old friends and reacquainted myself with their brilliance. They include second and third-generation Wittgenstein students who’d worked out the Web’s powerhouse search algorithms, and I badly wanted to hear their views on information. What they highlighted was a patchwork of half-stories rising from strongly philosophical roots.

Then to the computer scientists and the same questions, but asked differently. Again, a patchwork. But the more the questions were asked, the more the various stories began to connect. Ultimately it came down to personalities, perspectives, and interpretations, while all the time, history came alive: the common interests, the conversations, the coincidences in and around familiar ground. From there on, it wasn’t so much about research, but more about the reminiscences of welcoming insiders, in that most were flattered just to be questioned by anyone other than their immediate peers, and the word soon went out. What resulted was both illuminating and delightful. But in all of it, it wasn’t the direct answers that became important. Rather, it was the side comments and anecdotes, the inside-jokes and little-shared familiarities, the facial expressions and faint nods of agreement. It was as if the delicate threads of a frayed silk scarf were magically coming back together — like the experts had let out a huge sigh of relief. This was their chance to tell their story.

Within the mix were a few surprises, as most of the discussion was about past relationships and work long since overtaken, in that it was all very retrospective, but in a deeply authoritative way. Nevertheless, when memories and authority combine, magic can indeed happen. By asking experts to reconsider historical events from a modern perspective, in many ways what resulted was different and new. Insights were dusted down and recast in ways that proved interesting to all. What none of us expected though, was the value that would come from connecting all these stories together in a fresh light.

For decades, there’s been an unwritten tension between information technology experts. As most agree, not only is the idea of information still poorly understood, but there are several disparities between what the information specialists and computer scientists hold to be true. These cause regular arguments, especially when it’s difficult to keep the two camps apart. And not least on that list of differences is the idea of meaning — or, more formally, the subject of semantics. The reason for this is predominantly historical when all’s said and done, in that both Claude Shannon and Alan Turing — the venerable fathers of Information Theory and Computer Science respectively — managed to get away without using the idea of semantics explicitly in any of their mainstream work. Shannon in particular appears to have hated it, regarding it as nothing more than a nuisance, in that he deliberately expunged it from all his work, and, with hindsight, that stance served him well. Turing, on the other hand, might well have been more tolerant, but again he still failed to make obvious reference. In both cases that’s important, as modern trends in information technology, like Artificial Intelligence, often rely heavily on semantics to add real value. That makes them critically dependent on work produced after Shannon and Turing’s prime and which focuses on informational semantics. So, in simple terms, without the formal idea of meaning in today’s information technology, Facebook™ would have no friends and we’d still only have pocket calculators to keep us company. Shannon and Turing simply had to have left something out. Meaning has to fit in somewhere in information’s story.

With all of that as the backdrop, I kept writing and continued to ask questions of my betters, while reading as much as I could — all in the vain hope that somewhere along the line it might all make sense. Truth be told, I never really expected it would. That was until an innocent afterthought nearly passed me by.

I won’t let the secret out just yet, but one winter’s evening I found myself chatting with my old friend and mentor John Tait, a well-known information retrieval expert and uncannily a nearby neighbor. As is common when we infrequently meet, I was unfairly pressing him on the history of all things information and goading him on with the promise of free jam and honey.[1] By that point, I knew a lot about Turing and especially Shannon’s Information Theory, but you could hardly call my thought processes coherent. That was until John decided to change the direction of that evening’s conversation.

“Of course, there’s another way to read all of this”, he said, and I remember a sinking feeling, as if some unnecessary detour was about to follow. But no. As he went on, I had to stop him several times for a repeat. He was explaining something eminently plausible, yet unfamiliar, about the idea of meaning, and in a way that felt blissfully disrespectful of established convention. It was like hearing an echo from a familiar, yet distant, conversation, or seeing a famous painting from a fresh angle. Here was a definition of semantics I was unaware of, yet one that fitted both my understanding of modern thinking and of work around the time of World War II. It involved the translation of text from one language to another, then back again, and remarkably, it wasn’t at all at odds with Shannon’s ideas. To me at least, it felt more like an extension. It was at once both simple and profound. I didn’t know whether to kick myself for missing it or to jump for joy instead.

But could I use this new twist to help bring together the story I instinctively wanted to tell? Surely not. To John, the point he’d made was unimportant, but not to me and certainly not to those I’d previously spoken to or read about regards information. An eclectic mix for sure, but was this new insight the key? It felt like a tall order. Even so, as I wrote, the barriers began to fall and I felt compelled to visit Bletchley Park, the home of the allied wartime code breakers in the UK. There, unexpected discoveries were made again; so compelling, in fact, that I had to call in the help of Bletchley’s historian to confirm that I wasn’t wildly wrong.

In the end, it almost became a ground rush. With a little lateral thinking, a way through became clear and I began to realize that there was indeed a story to be told. The result is here for you to read. What you have is hopefully objective while still personal, qualified, and compelling. It’s as much about the future as it is the past and, to me at least, now not so much a work about information or technology anymore. Rather, I think, you’re about to read an unexpected thriller, charting the recent highs and lows of human achievement. For that reason, it’s as much about you as me, or any of the amazing people I’ve worked with to bring you this tale. I hope you enjoy it. It has quite literally changed my perspective on information and its place in the universe.

In my search to find meaning’s role in it all, I was rewarded with several epiphanies. For instance, it became abundantly clear that post-war pressures had a significant bearing on the birth of the information technology industry and the way that many of its seminal pioneers directed their work. That became especially clear with regard to Claude Shannon’s research and his insistence on reducing his Information Theory down to an absolute bare minimum. What resulted was the age of bit-wise information and a fixation with minimal information representation that ultimately spilt over into high-end physics. Regardless of the brush fire of technological advance that it would produce, it would also limit the imagination of many generations of engineers to come. With hindsight, it’s likely that too much emphasis was placed on Shannon’s insistence on minimalism. A broader tolerance of information’s makeup at the start would have not only kick-started many new branches of science much earlier, but also helped speed up the arrival of the broad-based communication protocols we now see as essential to the high-bandwidth world of today. Thankfully, though, all of that is now nicely in hand as fields like Complexity Theory and deep theoretical physics openly get to grips with the challenges of information. Likewise, and for all his creative genius, Alan Turing’s early work on logic also led to much restrictive engineering practice. In working through his ideas, he also sought out the clarity of minimal representation and, like Shannon, he embraced the advantages of working with a restricted palette. That’s what happens with new discoveries. First comes the purification of ideas, then their effect on tooling and practice follows naturally. For instance, first comes the realization that wood can be joined using metal, then later appears a slew of different screws, nails, hammers, and so on. As a result, if Turing were alive today and allowed to revisit his original ideas, I’m pretty sure that he would openly embrace all that we now know about the quantum world and especially Quantum Computing. Now that we can precisely rotate atoms and use the complex outputs that result, we can think about relegating the two-way switching relays of old and are afforded a much wider range of computational expression. I’m sure Turing would have loved that, and unlike Shannon, who continually sought out clarity through simplicity, the subdued flamboyance of his character would surely have reveled in the realization that Quantum Computing provides a superset playground for his original ideas.

In coming to this realization, I also came to understand the need to break the promise I’d made to myself when I started to write. This story needed to contain more than just established truth.

As the various threads and historic sub-plots swirled in my mind, much of the theory I’d covered began to open up, in that I not only found accord across mainstream ideas, but also several unexpected gaps — spaces that appeared open to the idea of meaning in information’s story, and which pointed to a way beyond the known road’s end. So, taking as many colleagues and confidants with me as I could, I decided to push out into the unknown. That’s why this book is now split into two parts, whereas my original intention was genuinely only to write one. The first, therefore, tries to be entirely factual and explains how we’ve come to understand information as we do today. It, therefore, more or less provides a historical perspective on our relationship with information from the mid-19th century onwards and hopefully contains one or two unexpected twists along the way. Think of it like the wide end of a funnel. Its specific purpose is to show just how many different perspectives on information have sprung up over the years and then to hint at how these might be narrowed down.

Then on to Part II, which is all about driving out pinpoint perspectives. It therefore first tries to explain our very latest ideas on information, before deliberately pushing the envelope. In doing that, it brushes past expert opinions on the vastness of the World Wide Web before delving down below the minute scale of Quantum Information’s essential ingredients. Both perspectives help stitch together elements from well-established ideas and align them to support the most important storylines in this book. Altogether, they provide a new and powerful viewpoint on how to include meaning in information’s definition — an extension to Shannon’s Information Theory, if you will.

In working that viewpoint through, I was helped by some of the very best minds in IBM and together we worked hard to get things right. In the end, it took nearly three years and resulted in a white paper [11] which nicely accompanies this book — which itself took over six years to write. Before the paper’s publication in early 2022, it went through many more rounds of rigorous review than is normal and secured approval at the very highest level.[2] But best not get too excited about such things. As the great physicist David Bohm once reminded: The word “theory” derives from the Greek “theoria”, which has the same root as “theatre”, in the word meaning “to view” or “to make a spectacle”. With that in mind, it might be said that any theory is just a way to form insight, or to see the world from a particular point of view, and not some knowledge of how the world actually works [12]. That is certainly the idea here. However, if I’m right, then phenomena like quantum entanglement and quantum teleportation are just ways in which the universe seeks to conserve its beautiful symmetry at fundamental levels. They speak of its need to network and find structure through the value of information and take us to the very forefront of work on quantum computation and Information Theory. In saying that, I would also point out that trying to explain information properly isn’t easy, not easy at all. Its multifaceted nature means that, by definition, different interpretations work well within different domains and contexts. As a result, conflicting views among experts are common; ongoing debate can be vigorous and the thoughts and ideas explained here represent just one viewpoint among many. I’ve, therefore, tried my hardest to be as accurate as I can and to reference and attribute the work of others in this book properly. Furthermore, where the ideas of others have needed more than just reference, I’ve hopefully clearly called them out by name. For those experts reading this book who feel I haven’t credited them appropriately, I apologize. It was never my intention to underplay your contribution or significance to the field.

Beyond Part II you’ll find an epilogue and an appendix, both of which are there to complement the book’s main storylines and perhaps make them easier to digest. The epilogue, therefore, provides a modern update on one or two of the more interesting historical points covered, while also giving a nod to the physicist’s current worry list. The appendix sets out to help with some of the mathematics underpinning many of the ideas discussed. In addition, you’ll also find summary notes at the start of each chapter to help you chart your course. These are repeated in Way Points. As you read, you may, of course, choose to miss out individual sections, but, whichever way you choose, here’s hoping it’s all For Your Information.

Endnotes

[1] I’m a beekeeper and often make jam as a hobby.

[2] Dario Gill, PhD, Head of IBM Research.

Way Points

Part 1: History and Science

Chapter 1:Bring on the Information Cake

First explains why information feels so familiar, then goes on to explain why our commonly held views might be somewhat superficial.

Chapter 2:Back to the Beginning

Examines the early history of electronic communications.

Chapter 3:There, Within the Name

Looks at the origins of Information Theory and Computer Science and hints at the deeply philosophical crossover of their roots. It also introduces key players like Claude Shannon, Max Newman, and Alan Turing.

Chapter 4:Princeton and Bell Labs

Explains the pivotal role played by Princeton University and Bell Labs in the birth of information technology and how Shannon’s early ideas on information developed.

Chapter 5:Meaning

Introduces the difficult subject of meaning and explains its relevance to the broader story of information.

Chapter 6:Logic and Common Foundations

Looks at how the roots of modern mathematics, semantics, and logic are closely linked, and how a young Alan Turing found himself at the crossroads of all three.

Chapter 7:The World War Web

Highlights the work of the codebreakers at Bletchley Park during World War II and explains how many of the ideas central to Information Theory could well have been developed there first. In particular, it calls out the contributions made by Bill Tutte, who worked with Alan Turing on cracking the German Lorenz cypher.

Chapter 8:The Quantum Revolution

Introduces the field of quantum mechanics as a precursor to explaining why the idea of information is now so important to our modern-day understanding of physical reality.

Chapter 9:Loop Quantum Gravity through Quantum Information

Delves down into modern thinking on the fundamental makeup of the universe and information’s role in it all.

Chapter 10:The Post-Internet World

Examines the arrival of the internet and the World Wide Web.

Part 2: What Does This Teach Us?

Chapter 11:Modern Thinking on Networks and Information

Provides a broad assessment of modern-day ideas linked with information, innovation, and technological success.

Chapter 12:Intelligence Paranoia and the Arrival of the Digital Computer

Brings the modern story of information technology up to date and explains several non-obvious drivers behind the rapid progress involved.

Chapter 13:Physics is a Kind of Language

Sets the context to help understand how and why information and physics are intimately linked. Fields like thermodynamics and Complexity Theory are introduced and their relationship with the ideas of difference and similarity.

Chapter 14:Conservation

Dives deeply into thermodynamics to understand the lesser-known roles of conservation and symmetry. Then explains why our best understanding of information sees it as being a physical thing.

Chapter 15:Information’s Needle

Covers electromagnetism and quantum mechanics to look at the ideas of coercion, choice, and difference in physical systems.

Chapter 16:Information’s Thread

Explains quantum entanglement and its relationship with conservation and information.

Chapter 17:Semantics in Physical Reality

Looks in detail at an ontological view of reality.

Chapter 18:Bringing It All Together

Final thoughts on information.

Part I: History and Science

Chapter 1. Bring on the Information Cake

Summary: First explains why information feels so familiar, then goes on to explain why our commonly held views might be somewhat superficial.

“All you really need to know for the moment is that the universe is a lot more complicated than you might think, even if you start from a position of thinking it’s pretty damn complicated in the first place.” — Douglas Adams

The universe has a wonderful symmetry. A harmony. A geometry.

If you want to know why, I mean really, really want to know why… try this:

The upshot of it all: In the beginning everything was created, and, in the grand scheme of things, it took no time at all. Either that or everything just decided to show up, which, we suspect, it’s done before. Whichever way, those of us who have better things to do aren’t best pleased. Firstly, because the manual hasn’t been put back yet and, secondly, because everyone knows it would have been better just to make cake. Especially information cake.

No, honestly, that’s it. No more, no less. The rest of this book will try to explain why.

As you might have guessed, this unconventional insight is written in the style of Douglas Adams, the much-loved author of The Hitchhiker’s Guide to the Galaxy[420]. Most would know him as the whimsical storyteller who brought us the catchphrase “Don’t Panic!”, but he did more than that. Adams was a deep thinker with a rare talent for seeing the world differently. He could take an idea and explain it in ways so annoyingly obvious that a deep-seated humor would be released. “Flying is learning how to throw yourself at the ground and miss” [13], is one of his. Funny how it takes rocket scientists years of study to come to the same conclusion.

If Adams and Albert Einstein had ever met, no doubt they would’ve become friends. Both had an insatiable, boyish curiosity, and both understood the true value of imagination. Indeed, you can almost hear Einstein coaching Adams in his famous quote:

“Imagination is more important than knowledge. For knowledge is limited to all we now know and understand, while imagination embraces the entire world, and all there ever will be to know and understand”[14].

Magically, he and Adams applied the power of imagination to prise open the security of our understanding. They looked at the world and told us what they saw, not what they were expected to see. For them, convention was less about rules and more about an invitation to explore the unknown. In Einstein’s case, he taught us how to see the world differently, while Adams did the same, although through the lens of a fictional alternative universe. What matters though, is that their perspectives were unique and useful. They simply told us their story. How blissfully joyful is that.

That’s all we will do here. We’ll tell a story from a new perspective, a perspective that focuses on a somewhat unexpected theme. The strange thing is that you’ll probably already be familiar with it.

The Age of Information

Instinctively, we think we know what information is. And so we should — we’re surrounded by it. Smartphones, social media, TV, advertising, even that annoying voice on your GPS nagging at you to turn left or right. At that level, information is very familiar to us. We depend upon it continually and exchange it regularly. We communicate, coordinate, and control using it. Ask anyone what information means to them and you’re likely in for a long sitting. How they keep in touch with their family. How they get their news. How they schedule their lives. How they this… How they that…

The arrival of the internet hasn’t helped. Like some catalyst, it’s hastened our addiction. Now we no longer use information simply to aid and advise. Today, we covet and crave it; we cherish and caress it. We are as much a part of it as it is of us. It’s not only essential to our existence but to our very identity as human beings. In so many ways, it is us.

Many would recognize this picture and acknowledge the significance that information plays in our lives. Younger readers might even see this as the norm and wonder how the world got by without computers and the Web. But we did, and those who can think back to before 1997 might remember a very different world.

The World Wide Web

1997 was notable for several reasons, but few would expect that its events would prove critical to the story of information. True, one or two milestones were reached that year that might seem linked, like Microsoft® becoming the world’s most valuable company [15], but these are really just distractions. 1997 was actually a tipping point for the world’s interaction with technology. It was the year that the World Wide Web saw mass take-up.

The number of internet users worldwide had skyrocketed since the birth of the Web in 1989. When Sir Tim Berners-Lee submitted his original proposal for what would become the World Wide Web [16] in March of that year, he merely saw it as an attempt to keep track of the growing mess of electronic documents and data at CERN — the European Laboratory for Particle Physics in Geneva. To this day he’s still shocked his proposal was accepted, let alone the whirlwind of change it would bring. But accepted it was, and on December 20, 1990 the world’s first Web server sprang into life. The first ever website was published on August 6th, 1991 [17] and, in 1993, CERN made the World Wide Web available on a royalty-free basis to the general public.

Adoption of the Web was slow at first and although the internet had 14 million users by 1993, only 130 websites had ever been registered. Widespread use didn’t really start until around 1995 [18] when studies suggest the Web grew by around 758%. Growth the following year was even more staggering, topping out at 996%. But most experts acknowledge that the increase of 334% in 1997 really saw the Web come into its own. That’s probably not just due to the number of online users, but also the effect of early adopters spreading the word and setting up the Web’s skeleton of foundational services, like search engines. They helped lower barriers to access and offered enough motivation for inexperienced users to experiment online.

The fuse of the Web’s explosive growth was likely ignited in September 1994, when the National Center for Supercomputing Applications launched its Mosaic™ browser. Dubbed the internet’s killer application, it was the first program to display images in line with website text. Big names like IBM® and Microsoft entered the market the coming year and followed suit, shortly after being joined by the Netscape Communications Corporation — formed when one of Mosaic’s creators left to set up his own company. Over the next few years, a vicious battle broke out for market dominance, speeding up innovation and sucking in creativity. Alongside, many household names began to emerge.

Amazon™ started trading in July 1995, for instance, followed by eBay™ in September of that year. Early search engines, like Yahoo™ and AltaVista™, also came online between 1994 and 1995, with Google™ starting somewhat late in 1998.

1994 was also key. Berners-Lee left CERN and founded the World Wide Web Consortium, or W3C® for short, a group of technical freethinkers who had the foresight to recognize what the Web would become. They cared passionately about the Web’s neutrality and the openness of its underlying construction. As a result, while on the outside the Web evolved organically, its center was protected by a Jedi-like council of technical specialists. This combination of evolution and leadership helped nurse the Web as it emerged. The Consortium is still very much alive today, happy to hide behind the scenes, ever so gently nudging the Web in the right direction.

But the modern-day story of information doesn’t end in 1997. By the turn of the millennium, the Web had grown to just over three million websites and, by the time Wikipedia™ was born in 2001, it had reached nearly 30 million. Around that time, and long before websites like Facebook™ took off, several pioneers were already talking seriously about the idea of a social Web. In truth, social interaction had always been at the Web’s core, but the early innovators built on previous experience to bring social networks to life. For example, as early as 2000, a team at the W3C had started work on the Friend of a Friend project [19], aimed at bringing like-minded individuals together online. Likewise, several similarly themed websites blinked into existence and then faded. But soon our need as social beings spilled over and the rush of social network sites stuck. History would have it that their success was serendipitous, but in fact they were a very obvious consequence of the Web’s need to evolve into something new. Websites like Friends Reunited™ led the charge in June 2000, then Facebook, and later Twitter™ in March 2006. These not only changed the way we interact, but they dramatically sped up change in most forms of natural language. Because of the need to type conversations, rather than speak them, online users tapped in underused characters like “@” and “#” and re-purposed them to shorten the writing process. In truth, it was more of a borrowing from computer lingo than a re-purposing, but the point is that no one suggested or dictated it. Rather, it emerged through common consensus. It was a consequence of the digitalization of the world in which we now live. It was just as much a product of natural selection as each one of us is ourselves.

Hardware, Humans, and Universal Darwinism

As the Web’s software coiled itself around the human condition, the hardware supporting it didn’t stand still. Computer processors became more powerful while shrinking in size and cost. And all the time, the network of cables connecting the internet kept expanding. Smothering. Becoming umbilical in its support of our increasing greed for technological advance.

That meant two things. First, the reach of global communications networks began to erode any meaningful notion of social boundaries, be they geographic, ethnic, political, or whatever. Second, the price of more than powerful enough computers easily fell within the reach of interested individuals. It was about commoditization lowering barriers. It was a freedom thing, a revelation and revolution in one, and in the hands of the common people. Where once it would have taken a roomful of gadgetry to connect to the internet, by the start of the 1990s a suitcase-sized machine would do. By the end of that decade, laptops had taken over, but the real shift came when hardware could comfortably fit into the hand. Then, in the early years of the millennium, mobile phones started to dominate and a cascade of unstoppable change had begun.

Similarly, the history of speech over the airwaves is long and eventful, but its most recent chapter probably starts with the introduction of a mobile handset by Motorola® in April 1973. Back then such devices were unwieldy, with Motorola’s prototype weighing in at 1.1 kg and measuring a massive 23 cm long [20]. Other manufacturers joined in and by the late 1990s a vibrant market was up and running. As processors continued to shrink in size and increase in power, handsets not only became more manageable but gained in functionality too. First came trivial widgets like calculators and simple games, but by 1998, mobiles were accessing the Web. That opened opportunities for untold technical creativity, a fact not lost on those at the Apple™ corporation. In 2007, they changed the game with the introduction of the iPhone™. Not so much a communications device as an intimate personal computer, it was an instant success. It not only went on to become the most profitable consumer product in history [21], but transformed the mobile phone industry. Customers loved it, not least because Apple cleverly placed the iPhone at the center of an ecosystem primed for expansion and growth. Its small packages of handy software were named apps, and each was purpose-built to lessen the burden of a small piece of the user’s average daily workload. It became like having a digital servant in your pocket. iPhones were built to be indispensable and Apple knew it. The idea was a winner from the start. Ingeniously, most apps didn’t interact directly with other apps across the network, instead demanding that their users be in on the act. At once, that created a network of dependence between people, phones, software, and developers. Overnight, a race to the top had started, born out of nothing more than pure digital gluttony. It was like letting some techno-genie out of the bottle, and an almost perfect catalyst to speed up both technical adoption and social change.

At that point, and under normal rules, mainstream science would have become interested. Experts would have consulted their textbooks and no doubt looked up various tables. But in the end, they’d all have agreed. This would have been something familiar to them. They’d have seen similar empirical evidence before and allowed the biologists, zoologists, and ecologists to push to the front. This had to be evolution, and everyone knew it. But the evolution of what, and how could the teaching of Charles Darwin be involved? Under all recognized definitions, evolution needs some form of life present to take place at all, and certainly at the rates being seen. The guidance demanded biology at the very least. That was the baseline, the God-given condition at any rate. Talk about evolution and you were discussing the very essence of life itself. That had to be impossible. How could technology so clearly change in this way? It was about copper wires and silicon chips, not flesh and blood.

The establishment seemed confused. Something had to be wrong.

But no; there was nothing wrong at all. It was just that science itself needed to refocus to see the obvious. Biology was there in the mix all along, but in an unfamiliar way. To the sociologists it sort of made sense, but they didn’t quite have a way to explain it. It wasn’t about flesh and blood, DNA, or genes per se. It was about ideas spreading from person to person, group to group. It was about technology acting as a spark for social change, and it needed a new branch of science, a sociotechnical science, to explain it. This was evolution taking place across bodies, not within them. Society was starting to apply technology to advance humanity. The ideas of Charles Darwin had just stepped up to planet-scale.

Before long, the academics and engineers began to catch up, but those involved in the birth of modern sociotechnical science still often argue about its exact inception. Some refer to a specific conversation in a pub in Edinburgh in 2006,[1] while others believe that the germ of the idea came about in a meeting at an airport hotel in Boston, Massachusetts in 2005.[2] But whichever way, the right minds soon came together and through their collaborative insight things became clearer.

In the end, all agreed that information transfer is at the very core of the evolutionary process itself. Earlier experts, like Richard Dawkins, had already said as much [22]. The fact that networks like the Web don’t look like the double helix of DNA is irrelevant. The Web is merely a collection of concepts, connections, languages, and protocols that don’t contain any notion of physical embodiment. Its deliberate separation from its internet infrastructure sees to that, and by such a disjoint it’s possible to argue that the Web is simply a huge, entangled map of connections.

But does it really have the qualities needed for life and evolution?

Don’t be fooled. Classical Darwinian theories might well be bound to the realities of biology, but more recently accepted ideas are not. In 1976, Dawkins presented a far more ethereal update to accepted thinking. In this, he outlined the idea of selfishness in genes [22], promoting the notion that they act only for themselves and therefore only replicate for their own good. He also introduced the important distinction between replicators and their vehicles. In the most obvious sense one might expect such replicators to be genes themselves, but that’s not necessarily always the case. It’s the information held within them that is more the real replicator, and the gene’s physical structure simply its carrier. It’s the information that’s important and not the mechanism used to support it. A replicator is therefore anything of which copies can be made, and that includes completely virtual capital like ideas, concepts, and even purely abstract information. A vehicle is therefore any entity that interacts with the environment to undertake or assist the copying process [23]. So, in a modern-day sociotechnical sense, any concepts embodied in Web-like content can be seen as replicators, and both human beings and software as their vehicles. Ideas on the wire are all that matters for sociotechnical evolution to take place.

Genes are, of course, replicators in a more general sense; selfish replicators that drive evolution in the biological world. But Dawkins believed that there’s a more fundamental principle at work. He suggested that wherever it arises, anywhere in the universe, be that real or virtual, “all life evolves by the differential survival of replicating entities”[22]. This is the foundation for the idea of Universal Darwinism, which encompasses the application of Darwinian thinking beyond the confines of biological evolution.

At the end of his book, The Selfish Gene, Dawkins asked an obvious, yet provocative, question: Are there any other replicators on our planet? The answer, he claimed, was “yes”. Staring us in the face, although still drifting clumsily about in its primeval soup of culture and technology, is another replicator — a unit of imitation [22].