Analytical Network and System Administration E-Book

Mark Burgess

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 642

Veröffentlichungsjahr: 2012

Beschreibung

Network and system administration usually refers to the skill of keeping computers and networks running properly.  But in truth, the skill needed is that of managing complexity.  This book describes the science behind these complex systems, independent of the actual operating systems they work on. 

It provides a theoretical approach to systems administration that:

• saves time in performing common system administration tasks.
• allows safe utilization of untrained and trained help in maintaining mission-critical systems.
• allows efficient and safe centralized network administration.

Managing Human-Computer Networks:

• Will show how to make informed analyses and decisions about systems, how to diagnose faults and weaknesses
• Gives advice/guidance as to how to determine optimal policies for system management
• Includes exercises that illustrate the key points of the book

The book provides a unique approach to an old problem and will become a classic for researchers and graduate students in Networking and Computer Science, as well as practicing system managers and system administrators.

Leseprobe

Contents

Cover

Half Title page

Title page

Copyright page

Foreword

Preface

Chapter 1: Introduction

1.1 What is system administration?

1.2 What is a system?

1.3 What is administration?

1.4 Studying systems

1.5 What’s in a theory?

1.6 How to use the text

1.7 Some notation used

Chapter 2: Science and its methods

2.1 The aim of science

2.2 Causality, superposition and dependency

2.3 Controversies and philosophies of science

2.4 Technology

2.5 Hypotheses

2.6 The science of technology

2.7 Evaluating a system—dependencies

2.8 Abuses of science

Chapter 3: Experiment and observation

3.1 Data plots and time series

3.2 Constancy of environment during measurement

3.3 Experimental design

3.4 Stochastic (random) variables

3.5 Actual values or characteristic values

3.6 Observational errors

3.7 The mean and standard deviation

3.8 Probability distributions and measurement

3.9 Uncertainty in general formulae

3.10 Fourier analysis and periodic behaviour

3.11 Local averaging procedures

3.12 Reminder

Chapter 4: Simple systems

4.1 The concept of a system

4.2 Data structures and processes

4.3 Representation of variables

4.4 The simplest dynamical systems

4.5 More complex systems

4.6 Freedoms and constraints

4.7 Symmetries

4.8 Algorithms, protocols and standard ‘methods’

4.9 Currencies and value systems

4.10 Open and closed systems: the environment

4.11 Reliable and unreliable systems

Chapter 5: Sets, states and logic

5.1 Sets

5.2 A system as a set of sets

5.3 Addresses and mappings

5.4 Chains and states

5.5 Configurations and macrostates

5.6 Continuum approximation

5.7 Theory of computation and machine language

5.8 A policy-defined state

Chapter 6: Diagrammatical representations

6.1 Diagrams as systems

6.2 The concept of a graph

6.3 Connectivity

6.4 Centrality: maxima and minima in graphs

6.5 Ranking in directed graphs

6.6 Applied diagrammatical methods

Chapter 7: System variables

7.1 Information systems

7.2 Addresses, labels, keys and other resource locators

7.3 Continuous relationships

7.4 Digital comparison

Chapter 8: Change in systems

8.1 Renditions of change

8.2 Determinism and predictability

8.3 Oscillations and fluctuations

8.4 Rate of change

8.5 Applications of the continuum approximation

8.6 Uncertainty in the continuum approximation

Chapter 9: Information

9.1 What is information?

9.2 Transmission

9.3 Information and control

9.4 Classification and resolution

9.5 Statistical uncertainty and entropy

9.6 Properties of the entropy

9.7 Uncertainty in communication

9.8 A geometrical interpretation of information

9.9 Compressibility and size of information

9.10 Information and state

9.11 Maximum entropy principle

9.12 Fluctuation spectra

Chapter 10: Stability

10.1 Basic notions

10.2 Types of stability

10.3 Constancy

10.4 Convergence of behaviour

10.5 Maxima and minima

10.6 Regions of stability in a graph

10.7 Graph stability under random node removal

10.8 Dynamical equilibria: compromise

10.9 Statistical stability

10.10 Scaling stability

10.11 Maximum entropy distributions

10.12 Eigenstates

10.13 Fixed points of maps

10.14 Metastable alternatives and adaptability

10.15 Final remarks

Chapter 11: Resource networks

11.1 What is a system resource?

11.2 Representation of resources

11.3 Resource currency relationships

11.4 Resource allocation, consumption and conservation

11.5 Where to attach resources?

11.6 Access to resources

11.7 Methods of resource allocation

11.8 Directed resources: flow asymmetries

Chapter 12: Task management and services

12.1 Task list scheduling

12.2 Deterministic and non-deterministic schedules

12.3 Human–computer scheduling

12.4 Service provision and policy

12.5 Queue processing

12.6 Models

12.7 The prototype queue M/M/1

12.8 Queue relationships or basic ‘laws’

12.9 Expediting tasks with multiple servers M/M/k

12.10 Maximum entropy input events in periodic systems

12.11 Miscellaneous issues in scheduling

Chapter 13: System architectures

13.1 Policy for organization

13.2 Informative and procedural flows

13.3 Structured systems and ad hoc systems

13.4 Dependence policy

13.5 System design strategy

13.6 Event-driven systems and functional systems

13.7 The organization of human resources

13.8 Principle of minimal dependency

13.9 Decision-making within a system

13.10 Prediction, verification and their limitations

13.11 Graphical methods

Chapter 14: System normalization

14.1 Dependency

14.2 The database model

14.3 Normalized forms

Chapter 15: System integrity

15.1 System administration as communication?

15.2 Extensive or strategic instruction

15.3 Stochastic semi-groups and martingales

15.4 Characterizing probable or average error

15.5 Correcting errors of propagation

15.6 Gaussian continuum approximation formula

Chapter 16: Policy and maintenance

16.1 What is maintenance?

16.2 Average changes in configuration

16.3 The reason for random fluctuations

16.4 Huge fluctuations

16.5 Equivalent configurations and policy

16.6 Policy

16.7 Convergent maintenance

16.8 The maintenance theorem

16.9 Theory of back-up and error correction

Chapter 17: Knowledge, learning and training

17.1 Information and knowledge

17.2 Knowledge as classification

17.3 Bayes’ theorem

17.4 Belief versus truth

17.5 Decisions based on expert knowledge

17.6 Knowledge out of date

17.7 Convergence of the learning process

Chapter 18: Policy transgressions and fault modelling

18.1 Faults and failures

18.2 Deterministic system approximation

18.3 Stochastic system models

18.4 Approximate information flow reliability

18.5 Fault correction by monitoring and instruction

18.6 Policy maintenance architectures

18.7 Diagnostic cause trees

18.8 Probabilistic fault trees

Chapter 19: Decision and strategy

19.1 Causal analysis

19.2 Decision-making

19.3 Game theory

19.4 The strategic form of a game

19.5 The extensive form of a game

19.6 Solving zero-sum games

19.7 Dominated strategies

19.8 Nash equilibria

19.9 A security game

19.10 The garbage collection game

19.11 A social engineering game

19.12 Human elements of policy decision

19.13 Coda: extensive versus strategic configuration management

Chapter 20: Conclusions

Appendix A: Some Boolean formulae

A.1 Conditional probability

A.2 Boolean algebra and logic

Appendix B: Statistical and scaling properties of time-series data

B.1 Local averaging procedure

B.2 Scaling and self-similarity

B.3 Scaling of continuous functions

Appendix C: Percolation conditions

C.1 Random graph condition

C.2 Bi-partite form

C.3 Small-graph corrections

Bibliography

Index

Analytical Network and System Administration

Copyright © 2004John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England

Telephone [phone]

Email (for orders and customer service enquiries): [email protected] our Home Page on www.wileyeurope.com or www.wiley.com

All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1T 4LP, UK, without the permission in writing of the Publisher. Requests to the Publisher should be addressed to the Permissions Department, John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England, or emailed to [email protected], or faxed to ([phone].

This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the Publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought.

Other Wiley Editorial Offices

John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA

Jossey-Bass, 989 Market Street, San Francisco, CA 94103-1741, USA

Wiley-VCH Verlag GmbH, Boschstr. 12, D-69469 Weinheim, Germany

John Wiley & Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australia

John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop #02-01, Jin Xing Distripark, Singapore 129809

John Wiley & Sons Canada Ltd, 22 Worcester Road, Etobicoke, Ontario, Canada M9W 1L1

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.

British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library

ISBN 0-470-86100-2

Foreword

It is my great honor to introduce a landmark book in the field of network and system administration. For the first time, in one place, one can study the components of network and system administration as an evolving and emerging discipline and science, rather than as a set of recipes, practices or principles. This book represents the step from ‘mastery of the practice’ and ‘scientific understanding’, a step very similar to that between historical alchemy and chemistry.

As recently as ten years ago, many people considered ‘network and system administration’ to comprise remembering and following complex recipes for building and maintaining systems and networks. The complexity of many of these recipes—and the difficulty of explaining them to non-practitioners in simple and understandable terms—encouraged practitioners to treat system administration as an ‘art’ or ‘guild craft’ into which practitioners are initiated through apprenticeship.

Current master practitioners of network and system administration are perhaps best compared with historical master alchemists at the dawn of chemistry as a science. In contrast to the distorted popular image of alchemy as seeking riches through transmutation of base metals, historical research portrays alchemists as master practitioners of the subtle art of combining chemicals towards particular results or ends. Practitioners of alchemy often possessed both precise technique and highly developed observational skills. Likewise, current master practitioners of network and system administration craft highly reliable networks from a mix of precise practice, observational skills and the intuition that comes from careful observation of network behaviour over long time periods. But both alchemists and master practitioners lack the common language that makes it easy to exchange valuable information with others: the language of science.

Alas, the alchemy by which we have so far managed our networks is no longer sufficient. When networks were simple in structure, it was possible to maintain them through the use of relatively straightforward recipes, procedures and practices. In the post-Internet world, the administrator is now faced with managing and controlling networks that can dynamically adapt to changing conditions and requirements quickly and, perhaps, even unpredictably. These adaptive networks can exhibit ‘emergent properties’ that are not predictable in advance. In concert with adapting networks to serve human needs, future administrators must adapt themselves to the task of management by developing an ongoing, perpetually evolving, and shared understanding.

In the past, it was reasonable to consider a computer network as a collection of cooperating machines functioning in isolation. Adaptive networks cannot be analysed in this fashion; their human components must also be considered. Modern networks are not communities of machines, but rather communities of humans inextricably linked by machines; what the author calls ‘cooperating ecologies’ of users and machines. The behaviour of humans must be considered along with the behaviour of the network for making conclusions about network performance and suitability.

These pressures force me to an inescapable conclusion. System administrators cannot continue to be alchemist-practitioners. They must instead develop the language of science and evolve from members of a profession to researchers within a shared scientific discipline. This book shows the way.

Though we live thousands of miles apart, the author and I are ‘kindred spirits’—forged by many of the same experiences, challenges and insights. In the late 1980s and early 1990s, both of us were faculty, managing our own computer networks for teaching and research. Neither of us had access to the contemporary guilds of system administration (or each other), and had to learn how to administer networks the hard way—by reading the documentation and creating our own recipes for success. Both of us realized (completely independently) that there were simple concepts behind the recipes that, once discovered, make the recipes easy to remember, reconstruct and understand. Concurrently and independently, both of us set out to create software tools that would avoid repeated manual configuration.

Although we were trained in radically differing academic traditions (the author from physics and myself from mathematics and computer science), our administrative tools, developed completely in isolation from one another, had very similar capabilities and even accomplished tasks using the same methods. The most striking similarity was that both tools were based upon the same ‘principles’. For the first time, it very much looked like we had found an invariant principle in the art of system and network administration: the ‘principle of convergence’. As people would say in the North Carolina backwoods near where I grew up, ‘if it ain’t broke, don’t fix it’.

The road from alchemy to discipline has many steps. In the author’s previous book, Principles of Network and System Administration, he takes the first step from practice (’what to do’) to principles (’why to do it’). Recipes are not created equal; some are better than others. Many times the difference between good and poor recipes can be expressed in terms of easily understood principles. Good recipes can then be constructed top–down, starting at the principles. Practitioners have approached the same problem bottom-up, working to turn their tested and proven recipes into sets of ‘best practices’ that are guaranteed to work well for a particular site or application. Recently, many practitioners have begun to outline the ‘principles’ underlying their practices. There is remarkable similarity between the results of these two seemingly opposing processes, and the author’s ‘principles’, and the practitioners’ ‘best practices’ are now quickly meeting on a common middle ground of principles.

In this book, for the first time, the author identifies principles of scientific practice and observation that anyone can use to become proficient ‘analysts’ of network and system administration practices. This will not make one a better practitioner, but rather will allow one to discuss and evaluate the practice with others in a clear and concise manner. The reader will not find any recipes in this book. The reader will not find principles of practice. Rather, the book explains the principles behind the science and chemistry of cooking, so that one can efficiently derive one’s own efficient and effective recipes for future networks. Proficient system administrators have always been capable of this kind of alchemy, but have found it challenging to teach the skill to others. This book unlocks the full power of the scientific method to allow sharing of analyses, so that future administrators can look beyond recipe, to shared understanding and discipline. In this way, now-isolated practitioners can form a shared scientific community and discipline whose knowledge is greater than the sum of its parts.

Looking at the table of contents, one will be very surprised to note that the traditional disciplines of ‘computer science’ and ‘computer engineering’—long considered the inseparable partners of system administration—are not the basis of the new science. Rather, experimental physics has proven to be the Rosetta Stone that unlocks the mysteries of complex systems. To understand why, we must examine the fundamental differences in economics between the disciplines of computer science and engineering and the disciplines of network and system administration.

Traditional computer science and engineering (and, particularly, the sciences involved in building the systems that system administrators manage) are based upon either an operational or axiomatic semantic model of computing. Both models express ‘what a program does’ in an ideal computing environment. Software developers build complex systems in layers, where each subsequent layer presumes the correct function of layers upon which it is built. Program correctness at a given layer is a mathematical property based upon axioms that describe the behaviour of underlying layers. Fully understanding a very complex system requires understanding of each layer and its interdependencies and assumptions in dealing with other layers.

System administrators have a differing view of the systems they manage compared to that of the developers who designed the systems. It is not economically feasible to teach the deep knowledge and mathematical understanding necessary to craft and debug software and systems to large populations of human system administrators. System administrators must instead base their actions upon a high-level set of initial experimental hypotheses called the ‘system documentation’. The documentation consists of hypotheses to be tested, not axioms to be trusted. As administrators learn how to manage a system, they refine their understanding top-down, by direct observation and ongoing evaluation of hypotheses.

Turning system and network administration into a discipline requires one to learn some skills, previously considered far removed from the practice. Evaluating hypotheses requires a rudimentary knowledge of statistics and the experimental method. These hypotheses are built not upon operational or axiomatic semantic models of computing, but upon specialized high-level mathematical models that describe behaviour of a complex system. With this machinery in hand, several advanced methods of analysis—prevalent in experimental physics and other scientific disciplines—are applied to the problem of understanding management of complex systems.

Proficient system administrators are already skilled experimental scientists; they just do not acknowledge this fact and cannot effectively communicate their findings to others. This book takes a major step towards understanding the profession of system and network administration as a science rather than as an art. While this step is difficult to take, it is both rewarding and necessary for those pioneers who will manage the next generation of networks and services. Please read on, and seek to understand the true nature of networking—as a process that involves connecting humans, not just computers.

Alva CouchTufts University, USA

Preface

This is a research document and a textbook for graduate students and researchers in the field of networking and system administration. It offers a theoretical perspective on human–computer systems and their administration. The book assumes a basic competence in mathematical methods, common to undergraduate courses. Readers looking for a less theoretical introduction to the subject may wish to consult (Burgess (2000b)).

I have striven to write a short book, treating topics briefly rather than succumbing to the temptation to write an encyclopædia that few will read or be able to lift. I have not attempted to survey the literature or provide any historical context to the development of these ideas (see Anderson et al. (2001)). I hope this makes the book accessible to the intelligent lay reader who does not possess an extensive literacy in the field and would be confused by such distractions. The more advanced reader should find sufficient threads to follow to add depth to the material. In my experience, too much attention to detail merely results in one forgetting why one is studying something at all. In this case, we are trying to formulate a descriptive language for systems.

A theoretical synthesis of system administration plays two roles: it provides a descriptive framework for systems that should be available to other areas of computer science and proffers an analytical framework for dealing with the complexities of interacting components. The field of system administration meets an unusual challenge in computer science: that of approximation. Modern computing systems are too complicated to be understood in exact terms.

In the flagship theory of physics, quantum electrodynamics, one builds everything out of two simple principles:

1. Different things can exist at different places and times.

2. For every effect, there must be a cause.

The beauty of this construction is its lack of assumptions and the richness of the results. In this text, I have tried to synthesize something like this for human–computer systems. In order to finish the book, and keep it short and readable, I have had to compromise on many things. I hope that the result nevertheless contributes in some way to a broader scientific understanding of the field and will inspire students to further serious study of this important subject.

Some of this work is based on research performed with my collaborators Geoff Canright, Frode Sandnes and Trond Reitan. I have benefited greatly from discussions with them and others. I am especially grateful for the interest and support of other researchers, most notably Alva Couch for understanding my own contributions when no one else did. Finally, I would like to thank several for reading the draft versions of the manuscript and commenting: Paul Anderson, Lars Kristiansen, Tore Jonassen, Anil Somayaji and Jan Bergstra.

Mark Burgess

Chapter 1

Introduction

Technology: the science of the mechanical and industrial arts.[Gk. tekhne art and logos speech].

—Odhams dictionary of the English language

1.1 What is system administration?

System administration is about the design, running and maintenance of human–computer systems. Human–computer systems are ‘communities’ of people and machines that collaborate actively to execute a common task. Examples of human–computer systems include business enterprises, service institutions and any extensive machinery that is operated by, or interacts with human beings. The human players in a human–computer system are often called the users and the machines are referred to as hosts, but this suggests an asymmetry of roles, which is not always the case.

System administration is primarily about the technological side of a system: the architecture, construction and optimization of the collaborating parts, but it also occasionally touches on softer factors such as user assistance (help desks), ethical considerations in deploying a system, and the larger implications of its design for others who come into contact with it. System administration deals first and foremost with the system as a whole, treating the individual components as black boxes, to be opened only when it is possible or practical to do so. It does not conventionally consider the design of user-tools such as third-party computer programs, nor does it attempt to design enhancements to the available software, though it does often discuss meta tools and improvised software systems that can be used to monitor, adjust or even govern the system. This omission is mainly because user-software is acquired beyond the control of a system administrator; it is written by third parties, and is not open to local modification. Thus, users’ tools and software are treated as ‘given quantities’ or ‘boundary conditions’.

For historical reasons, the study of system administration has fallen into two camps: those who speak of network management and discuss its problems in terms of software design for the management of black box devices by humans (e.g. using SNMP), and those who speak of system administration and concern themselves with practical strategies of machine and software configuration at all levels, including automation, human–computer issues and ethical considerations. These two viewpoints are complementary, but too often ignore one another. This book considers human–computer systems in general, and refers to specific technologies only by example. It is therefore as much about purely human administrative systems as it is about computers.

1.2 What is a system?

A system is most often an organized effort to fulfil a goal, or at least carry out some predictable behaviour. The concept is of the broadest possible generality. A system could be a mechanical device, a computer, an office of workers, a network of humans and machines, a series of forms and procedures (a bureaucracy) etc. Systems involve themes, such as and between different actors, the use of to represent information or to promote efficiency, and the laws of Within any mechanism, of the parts is required to build significant innovation; it is only through strategy of divide and conquer that significant problems can be solved. This implies that each division requires a special solution.