An Introduction to Network Modeling and Simulation for the Practicing Engineer E-Book

Jack L. Burbank

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Serie: IEEE ComSoc Pocket Guides to Communications Technologies
• Sprache: Englisch

Seitenzahl: 333

Veröffentlichungsjahr: 2011

Beschreibung

This book provides the practicing engineer with a concise listing of commercial and open-source modeling and simulation tools currently available including examples of implementing those tools for solving specific Modeling and Simulation examples. Instead of focusing on the underlying theory of Modeling and Simulation and fundamental building blocks for custom simulations, this book compares platforms used in practice, and gives rules enabling the practicing engineer to utilize available Modeling and Simulation tools. This book will contain insights regarding common pitfalls in network Modeling and Simulation and practical methods for working engineers.

Leseprobe

Table of Contents

Cover

Series page

Title page

Copyright page

PREFACE

ACKNOWLEDGMENTS

ABOUT THE AUTHORS

CHAPTER 1 Introduction

1.1 ADVANTAGES AND DISADVANTAGES OF MODELING AND SIMULATION

1.2 COMPARISON OF “HOMEBREW” MODELS AND SIMULATION TOOLS

1.3 COMMON PITFALLS OF MODELING AND SIMULATION AND RULES OF THUMB

1.4 AN OVERVIEW OF COMMON M&S TOOLS

1.5 AN OVERVIEW OF THE REST OF THIS BOOK

CHAPTER 2 Modeling and Simulation for RF Propagation

2.1 THE FADING CHANNEL

2.2 THE ITU M.1225 MULTIPATH FADING PROFILE FOR MOBILE WiMAX

2.3 PRACTICAL FADING MODEL IMPLEMENTATIONS—WiMAX EXAMPLE

2.4 RF PROPAGATION SIMULATORS

2.5 PROPAGATION AND FADING SIMULATIONS—LESSONS LEARNED

CHAPTER 3 Physical Layer Modeling and Simulation

3.1 INCORPORATING INTERFERENCE INTO A MODEL

3.2 THE IMPORTANCE OF A PREAMBLE

3.3 PRACTICAL WIRELESS PHY MODEL IMPLEMENTATIONS

3.4 WIRELESS NETWORK SIMULATION LESSONS LEARNED AND COMMON PITFALLS—PHY LAYER

CHAPTER 4 Medium Access Control Modeling and Simulation

4.1 MODELING AND SIMULATION OF WIRED MACs

4.2 WIRELESS NETWORK MAC SIMULATION

4.3 PRACTICAL MAC MODEL IMPLEMENTATIONS

4.4 NETWORK SIMULATION LESSONS LEARNED AND COMMON PITFALLS—MAC LAYER

CHAPTER 5 Modeling and Simulation for Higher Layer Protocols

5.1 NETWORK LAYER

5.2 TRANSPORT AND APPLICATION LAYERS

5.3 EXAMPLE OF HIGHER LAYER MODELING: TRANSPORT LAYER PERFORMANCE ANALYSIS

5.4 EXAMPLE OF HIGHER LAYER MODELING: DETAILED NETWORK LAYER MODELING

CHAPTER 6 Hardware-in-the-Loop Simulations

6.1 ADVANTAGES AND DISADVANTAGES OF HITL APPROACHES

6.2 NETWORK M&S HITL APPROACHES

6.3 HITL EXAMPLES

6.4 COMMON PITFALLS FOR HITL APPROACHES

6.5 NETWORK-LAYER HITL-READY NETWORK SIMULATION PLATFORMS

6.6 HITL CONCLUSION

CHAPTER 7 Complete Network Modeling and Simulation

7.1 COMPLETE NETWORK M&S PLATFORMS

7.2 IEEE HLA (1516)

7.3 COMPLETE NETWORK SIMULATION EXAMPLES

CHAPTER 8 Other Vital Aspects of Successful Network Modeling and Simulation

8.1 VERIFICATION AND VALIDATION

8.2 DATA VISUALIZATION AND INTERPRETATION

CHAPTER 9 Network Modeling and Simulation: Summary

REFERENCES

Index

IEEE Press

445 Hoes Lane

Piscataway, NJ 08854

IEEE Press Editorial Board

Lajos Hanzo, Editor in Chief

R. AbhariM. El-HawaryO. P. MalikJ. AndersonB-M. HaemmerliS. NahavandiG. W. ArnoldM. LanzerottiT. SamadF. CanaveroD. JacobsonG. Zobrist

Kenneth Moore, Director of IEEE Book and Information Services (BIS)

Technical Reviewers

Nim K. Cheung

Richard Lau

A volume in the IEEE Communications Society series:

The ComSoc Guides to Communications Technologies

Nim K. Cheung, Series Editor

Thomas Banwell, Associate Editor

Richard Lau, Associate Editor

Next Generation Optical Transport: SDH/SONET/OTN

Huub van Helvoort

Managing Telecommunications Projects

Celia Desmond

WiMAX Technology and Network Evolution

Edited by Kamran Etemad, Ming-Yee Lai

An Introduction to Network Modeling and Simulation for the Practicing Engineer

Jack Burbank, William Kasch, Jon Ward

Copyright © 2011 by Institute of Electrical and Electronics Engineers. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax [phone], or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, [phone], fax [phone], or online at http://www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at [phone], outside the United States at [phone] or fax [phone].

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data is available.

ISBN: 978-0-470-46726-8

oBook ISBN: 978-1-118-06365-1

ePDF ISBN: 978-1-118-06363-7

ePub ISBN: 978-1-118-06364-4

PREFACE

This book provides an overview of the current state-of-the-art in modeling and simulation (M&S) tools and discusses many of the pitfalls most commonly encountered by network engineers. A bottom-up approach is taken in describing network M&S, following the Transport Control Protocol / Internet Protocol (TCP/IP) modified Open System Interconnect (OSI) stack model. While applicable to network M&S in general, there is particular emphasis placed on wireless network M&S. This book first decomposes the wireless network M&S problem into a set of smaller scopes: 1) radio frequency (RF) propagation M&S (Chapter 2), 2) physical layer (PHY) M&S (Chapter 3), 3) Medium Access Control (MAC) layer (Chapter 4), and 4) higher layer M&S (Chapter 5). After considering each of these smaller scopes somewhat independently, the book then revisits the overall problem of how to conduct M&S of a wireless networking system in its entirety.

No specific assumptions are made on the type of network being modeled in any particular layer of the protocol stack. Instead, the building blocks are presented to address the common challenges of modeling any wireless network. The reader is also directed to resources that provide more detail on specific topics. Resources are chosen from generic studies of wireless networks and from the Mobile Ad Hoc Network (MANET) and ad hoc sensor network communities. This book is written with particular emphasis placed on specific topics at the different layers of the protocol stack, with the intention of bridging gaps between the computer science and electrical engineering communities. Historically, the higher layers of the protocol stack are often considered research subjects for computer scientists and the lower layers for electrical engineers. In fact, accurate simulations must capture the cross-layer interactions and higher layer simulations must consider the impacts of the lower layer conditions on results. The authors hope that this book will educate the reader in simulation topics that may have not otherwise been considered and will ultimately lead to improved simulation results in the wireless networking research community.

This book can improve the reader’s background knowledge on the key components of successful wireless network simulations. But, ultimately, the reader must learn to validate his or her own simulation since they alone will know all specific details and assumptions that lead to a specific result. In general, the output of a simulation should not be a surprise to the designer, and, it if is, sufficient research into the underlying protocol must be conducted to explain any unanticipated results. Because there are so many variables present in a model and therefore so many potential locations where errors are introduced, a model output should not be taken as ground truth without other methods of verification. Results may be compared with results from other researchers, but as some papers [1–4] note, results between two equivalent scenarios simulated on two different simulators may not match. In this case, the designer must not only validate whether or not his or her simulation is correct, but also what led to results not matching the other simulation. Results should not be published until the simulation designer has confidence in the model, the results have been validated to the best of the designer’s ability, and, once published, should contain all model parameters, assumptions, and simulation source code.

In this book only a select set of simulators have been considered as the most popular commonly used by academic and industrial researchers. These include OPNET, NS-2, GloMoSim, and QualNET. There is no single, all- purpose simulator that is best for all scenarios. Additionally, budget constraints often force researchers to choose open-source simulators over commercial solutions. Custom simulation solutions (i.e., homebrew simulations) are certainly too numerous to be considered. Note that the risk of citing specific simulators is that these tools are continually evolving. This means that statements about a given product’s current capabilities may no longer be valid, as subsequent releases enhance a tool’s capabilities. Care has been taken by the authors to focus on principles and practices that assist the simulation designer in improving wireless network simulations while remaining independent of a particular simulator, and hence topics and results are not as limited to an expiration date.

JACK BURBANK

WILLIAM KASCH

JON WARD

To view color versions of the figures in this book, please visit http://booksupport.wiley.com.

ACKNOWLEDGMENTS

We would like to acknowledge the numerous individuals who have helped make this book a reality. First and foremost, we would like to acknowledge Brian Haberman and Julia Andrusenko for their assistance in writing this book, contributing their expertise and understanding of network simulation tools and radio frequency propagation tools, respectively.

We would like to thank Robert Nichols for his long-time support of our activities in this field.

We would like thank all of our friends and family for their patience and support during the writing of this book.

ABOUT THE AUTHORS

Jack L. Burbank ([email protected]) received his B.S. and M.S. degrees in electrical engineering from North Carolina State University (NCSU) in 1994 and 1998, respectively. As part of the Communications and Network Technologies Group of The Johns Hopkins University Applied Physics Laboratory (JHU/APL), he works with a team of engineers focused on assessing and improving the performance of wireless networking technologies through test, evaluation, and technical innovation. His primary expertise is in the areas of wireless networking and modeling and simulation, focusing on the application and evaluation of wireless networking technologies in the military context. He has published numerous technical papers and book chapters on topics of wireless networking, and regularly acts as a technical reviewer for journals and magazines. He teaches courses on the topics of networking and wireless networking in the Johns Hopkins University Part Time Engineering Program, and is a member of the IEEE and the ASEE.

William T.M. Kasch ([email protected]) received a B.S. in electrical engineering from the Florida Institute of Technology in 2000 and an M.S. in electrical and computer engineering from Johns Hopkins University in 2003. His interests include various aspects of wireless networking technology, including MANETs, IEEE 802 standards, and cellular. He participates actively in both the IEEE 802 standards organization and the Internet Engineering Task Force (IETF).

Jon R. Ward, PE ([email protected]) graduated from NCSU in 2005 with an M.S. degree in electrical engineering. He works at JHU/APL on projects focusing on wireless network design and interference testing of standards-based wireless technologies such as IEEE 802.11, IEEE 802.15.4, and IEEE 802.16. He has experience in wireless network modeling and simulation (M&S) and test and evaluation (T&E) of commercial wireless equipment. He is currently a student at the University of Maryland, Baltimore County (UMBC), pursuing a Ph.D. degree in electrical engineering.

CHAPTER 1

Introduction

Communications systems continue to evolve rapidly. Users continue to demand more high-performance networking capabilities. Service providers respond to this demand by rapid expansion of their network infrastructure. Network researchers continue to develop revolutionary new communications techniques and architectures to provide new capabilities commensurate with evolving demands. Equipment vendors continue to release new devices with ever-increasing capability and complexity. Technology developers rapidly develop next-generation replacements to existing capabilities to keep up with demand. These rapid developments in the network industry lead to a large, complex landscape.

The network designer and developer wants (and needs) to satisfy the demands of the users. This is difficult, as it is often complicated for the typical network engineer to fully understand this rapidly evolving communications landscape. This challenge is exacerbated by the nature of emerging technologies and techniques that are often extremely complex compared with their legacy counterparts. This leaves the typical network engineer with more questions than answers. The network engineer tasked with maintaining an operational network might ask the following: What is the right approach to solving my problem? Do I buy the latest device from company X that claims to solve all my problems? Do I replace the underlying technology of my system with the latest generation? How do I know whether a technology is mature enough to survive the rigors of my application? How do I know how my already existing network system will respond if I add this device? The network engineer researching next-generation networking techniques might ask: How do I know how this new approach will interact with already-existing protocols? or How do I build confidence in the utility of this approach without producing and deploying the technology? The network engineer developing a particular product might ask: How do I ensure that this design will satisfy requirements before I go to production? or How can I assess the utility of a design choice compared to its envisioned cost? This book aims to help answer these questions.

There are many tools available to the network engineer that can assist in answering these questions, including analysis, prototype implementation and empirical testing, trial field deployments, and modeling and simulation (M&S). It should be stated now that no one tool is typically sufficient in understanding the performance of a network; unfortunately, there is no “silver bullet” answer to all our questions. The complex nature of emerging systems also introduces significant complexity into the effective evaluation of these systems and how these various tools can be employed. Evaluation is often conducted through the coordinated usage of analysis, M&S, and trial deployments in closely monitored environments. Due to the costs and complexities of deployments, analysis and M&S are often used to determine the most sensitive performance areas that are then the focus of trial deployments. This limits the scope of the trial deployment to a realistic level while focusing on the important cases to consider.

Because of the increasingly interconnected nature of communications systems, and the resulting interdependencies of individual subsystems to operate as a whole, it will often be the case that individual subsystems cannot be tested in isolation. Rather, multiple systems must be evaluated in concert to verify system-level performance requirements. This increases the required scale of trial deployments and adds significant complexity as now several different types of measurements will often be required in several different locations simultaneously. This increases the required support for a deployment in terms of required resources, including personnel and measurement equipment, further limiting the realistic amount of trial deployments. Thus, this will place a premium on analysis and M&S to perform requirements verification and to form the basis of any performance evaluation. In many cases, M&S may provide the only viable method for providing insight into the behavior of the eventual system prior to full-scale deployment.

Once the importance of M&S is established, many additional questions still arise: How does the network engineer properly employ M&S? What are the most appropriate M&S tools to employ? While networking technologies continue to evolve rapidly, so too do M&S tools intended to evaluate their performance. The M&S landscape is indeed a complicated space with a multitude of tools with a variety of capabilities and pitfalls. Furthermore, there is often a poor understanding of the proper role and application of M&S and how it should fit within the overall evaluation strategy. There is even confusion surrounding the term M&S itself. Before we continue, let us provide some basic definitions that will be used throughout the book.

Modeling and simulation (M&S) are often combined as a single term. However, a model is quite different than a simulation. This book defines these two entities as:

Model: A logical representation of a complex entity, system, phenomena, or process. Within the context of communications and networking, a model is often an analytical representation of some phenomena (e.g., a mathematical representation for the output of a system component) or a state machine representation. This analytical representation can either be in a closed form or an approximation obtained through assumptions.

Simulation: An imitation of a complex entity, system, phenomena, or process meant to reproduce a behavior. Within the context of a communications network, a simulation is most often computer software that to some degree of accuracy functionally reproduces the behavior of the real entity or process, often through the employment of one or more models over time.

Emulation: An imitation of a real-world, complex entity or process meant to perfectly reproduce a behavior or process. Emulation can be thought of as perfect simulation of something such that it is equivalent to the original entity.

To illustrate the difference between a model and a simulation, consider a simple signal detection circuit. A simulation of this device would imperfectly mimic the various actions of the detection circuit to determine a likely outcome for a given input. A model of this same device would generally take the form of a mathematical algorithm that would produce (either perfectly or imperfectly) an output for a given input.

Unfortunately, the terms model and simulation are often incorrectly used interchangeably. Generally speaking, the term simulation has wider scope than the term model, where a simulation is typically a compilation of models and algorithms of smaller components of the larger overall entity or process. This book generally uses the combined term M&S to generically refer to the employment of models, simulations, and emulators to approximate the behavior of an entity or process.

There are numerous types of computer models and simulations. A computer model or simulation can generally be classified according to several key characteristics:

Stochastic vs. Deterministic: Deterministic models are those that have no randomness. A given input will always produce the same output given the same internal state. Deterministic models can be defined as a state machine. Deterministic models are the most common type of computer model. A stochastic model does not have a unique input-to-output mapping and is generally not widely employed, as it leads to unpredictability in execution. A simulation can be made to act in a pseudo-random manner through the employment of random number generators to represent random events. However, the particular models governing the behavior of each component within the simulation are generally deterministic.Steady-state vs. Dynamic: Steady-state models attempt to find the input-to-output relationship of a system or entity once that system is in steady-state equilibrium. A dynamic simulation represents changes to the system in response to changing inputs. Steady-state approaches are often used to provide a simplified model prior to dynamic simulation development.Continuous vs. Discrete: A discrete model considers only discrete moments in time that correspond to significant events that impact the output or internal state of the system. This is also referred to as a discrete-event (DE) model or DE simulation. This requires the simulation to maintain a clock so that the current simulation time can be monitored. Jumps between discrete points in time are instantaneous; nothing happens between discrete points in time corresponding to interesting events. Continuous simulations consider all points in time to the resolution of the host’s hardware limitations (all computer simulations are discrete to some extent because of the fact that it is running on a digital platform with a finite speed clock). DE methods are the most commonly used for network M&S.Local or Distributed: A distributed simulation is such that multiple computer platforms that are interconnected through a computer network work together, interacting with one another, to conduct the simulation. A local simulation resides on a single host platform. Historically, local simulations have been the most common. But the increasing complexity of simulations have increased the importance of distributed simulation approaches.

In general, a simulation can be thought of as a piece of software residing on a computer platform that implements a set of algorithms and routines and takes a set of inputs to produce a set of outputs that represent the behavior of the system of interest. This is depicted in Figure 1-1.

FIGURE 1-1. A block diagram of a wireless communications system simulation.

The typical inputs that are important to consider when simulating a wireless network are summarized in Table 1-1. The typical outputs that are often of interest are summarized in Table 1-2.

TABLE 1-1. Typical Inputs to a Wireless Network Simulation

ParameterExplanationSignal powerThis will influence the received power level and consequently the Bit Error Rate (BER) and Packet Error Rate (PER) performance of the wireless link.Waveform typeThis will influence the BER and PER performance of the wireless link in a given channel.Forward error control coding (FEC) methodThis will influence the BER and PER performance of the wireless link in a given channel.Retransmission protocolThis will affect the throughput and delay performance of the wireless link.Contention methodThis will influence BER, PER, throughput, and delay performance of the wireless link in a given channel.Channel modelThis will determine the performance of a given wireless link in terms of received power level, BER, and PER.Mobility modelThis will impact the performance of the MAC layer protocol and of the higher layers (e.g., IP routing).Traffic modelThis will impact the performance of the MAC layer protocol and of the higher layers (e.g., IP routing).Network topologyThis will impact the performance of the MAC layer protocol and of the higher layers (e.g., IP routing).

TABLE 1-2. Typical Outputs from a Wireless Network Simulation

ParameterExplanationBERThe fundamental performance metric of a digital communications link.PEROften considered the most important performance metric in a packet-switched network.ThroughputThe data rate supportable by the wireless network.GoodputThe useful data rate supported by the wireless network (i.e., data rate as available by the application).LatencyThe end-to-end delay that an application or user will experience across the wireless network.

1.1 ADVANTAGES AND DISADVANTAGES OF MODELING AND SIMULATION

As is the case with any tool, M&S has both advantages and disadvantages. This section provides a tradeoff framework for the designer or developer to consider when choosing to employ M&S. In the following section, M&S is often compared with empirical testing. For the purposes of this book, empirical testing refers to real-world testing of equipment (e.g., physical hardware devices) deployed in a physical environment.

1.1.1 Breadth of Operational Scenario

First and foremost, M&S provides the ability to exercise a wide range of operational scenarios. Empirical testing will exercise a much smaller portion of the possible scenario space than will M&S. This includes the ability to evaluate greatly increased network scale (e.g., number of network nodes), not easily achieved in empirical activities, and more dynamic choice of environmental conditions (e.g., wireless environment). Because of the ability to exercise a wide variety of scenarios, M&S has a clear advantage in this aspect.

1.1.2 Cost

Generally, another advantage of M&S is reduced cost compared with empirical testing and trial deployments. Extensive empirical testing carries a high cost, to the point where extensive empirical-only approaches are largely impossible in the modern wireless networking landscape; however, this advantage is dependent on the scope placed on the M&S development effort.

1.1.3 Confidence in Result

A less obvious advantage of M&S is the amount of precision and control that can be exerted over the scenario in question. In the empirical scenario, measurements are taken and then those measurements are analyzed and understood for their ramifications. However, due to the uncontrolled nature of empirical testing, there are often many variables that affect the measurement. And often the number of uncertain variables is so great that it is impossible to isolate the source of any behavior or to correlate a measurement to its source (i.e., map the effect to the cause). This limits the scientific utility of such measurements, and makes it difficult to associate a high degree of confidence to the measurement. The “the data is what it is” philosophy is rarely justified if the phenomena under observation are not understood. Note, this is much more the case for over-the-air (OTA) empirical activities. Other empirical activities are much more highly controllable (e.g., direct radiofrequency (RF) chain testing).

The primary, and most obvious, disadvantage of M&S is that it is not real. It is a representation of the system, rather than the system itself. There are several assumptions that will be built into any M&S tool. Some of these assumptions will be necessitated by real-world complexities that are not easily represented. Others are necessitated by a lack of information available about the system in question. This will naturally lead to inaccuracies. Consequently, this leads to a decreased confidence in results. This confidence decrease is manageable, however, through verification and validation activities, often in conjunction with empirical activities to improve confidence in such models.

A higher degree of confidence is almost always associated with empirical methods, regardless of the methodology or practices employed during those empirical activities. Unfortunately, this confidence can be ill placed. The common belief is that M&S-based methods are more subject to error because software-based “bugs” could introduce unforeseen inaccuracies. And while that is definitely true, the same applies to the empirical-based approach. Any empirical measurement will have error associated with it (e.g., imperfections in hardware employed to make a measurement, misconfiguration of test equipment). Also, human interpretation must at some point be applied to understand an empirical measurement. This human interpretation can be influenced by assumptions, biases, and preconceived opinions.

Another issue is that of statistical significance. Even if measurement error has been minimized, there are several factors that can influence the significance of that measurement. Take, for example, the measurement of an antenna pattern, which is a key characteristic that will impact wireless network performance. This antenna pattern will vary across antenna population due to manufacturing variation, differences in platform, and differences in age and condition. Furthermore, the RF propagation environment characteristics will be temporal in nature. Thus, a particular measurement is somewhat insignificant in the overall sense. In fact, to make empirical activities truly significant from a statistical standpoint is often cost prohibitive.

With all these factors considered, an empirical approach is still considered to have an advantage, especially if issues such as measurement error and uncertainty are built into empirical activities. However, the proper application of verification and validation practices can help minimize this difference.

1.1.4 Perception

Even if a model is highly accurate, and from a scientific perspective is highly regarded, there is the issue of perception. Many individuals will still remain skeptical of the results from a computer model. This is due to sociological and psychological phenomena that are well beyond the scope or timeframe of any particular M&S activity. Rather, this reality must be accepted and factored into the overall evaluation approach. An empirical-based evaluation method has the overwhelming advantage in this area. In fact, this advantage is so strong that some degree of empirical testing is likely required to give credibility to the findings of the overall M&S activity.

1.1.5 The Need for Verification and Validation

While not considered a disadvantage, certainly a burden associated with M&S is the need to conduct verification and validation (V&V) activities. Such activities are generally required to both verify the accuracy and consistency of model output and validate output relative to other models, empirical tests, and theory. While V&V activities are mandated by good software engineering principles and must be adhered to, the formality of a V&V process can levy significant resource requirements on a project. This partially negates the cost advantage of M&S over empirical testing.

In some sense, M&S is disadvantaged in this regard compared with other tools available to the network engineer. As mentioned previously, there is typically less scrutiny placed on empirical measurements and, consequently, there is typically a greater “burden of proof” placed on an M&S developer as compared with the empirical tester.

1.2 COMPARISON OF “HOMEBREW” MODELS AND SIMULATION TOOLS

Custom simulations, or “homebrew” solutions, are those in which the implementer does not rely on any existing tools but rather develops the simulation in its entirety. The advantages of homebrew simulations include:

The implementer knows exactly what has been implemented.Homebrew solutions can have significant performance benefits.

The disadvantages of homebrew simulations include:

They can be costly to develop.They can be difficult to upgrade.There is a real risk of these custom simulations not being widely adopted, even within your organization (resulting in perpetual “homebrew” solutions).

Other than small-scale efforts that are supporting analysis, homebrew approaches are generally discouraged. With the ever-increasing complexity of wireless networking systems, the feasibility of a meaningful homebrew solution is dwindling. Even for cases where there are no existing implementations of a particular networking technology and code development is inevitable, it is recommended that this new custom simulation be developed within existing tools/environments so that it can be integrated with and leverage existing simulation libraries.

1.3 COMMON PITFALLS OF MODELING AND SIMULATION AND RULES OF THUMB

There are many potential pitfalls that face those who embark on a network simulation development effort. This section discusses some of those most commonly seen.

1.3.1 Model Only What You Understand

It can be said that the utility of a given model is only as good as the degree to which it represents the actual system being modeled. Indeed, a system—whether a wireless network or otherwise—can only be modeled once it is sufficiently understood. While this is a simple tenant, it is one that is certainly not adhered to universally by M&S designers. One may ask why M&S designers develop invalid models. There are many reasons, the first of which is that high-fidelity model development requires a significant investment of time and effort. This statement is not meant to offend developers or to imply carelessness on their part. The fact is that many designers are under time constraints to deliver results. Consequently, a careful understanding of the underlying system being modeled and rigorous validation of the model is not always an option.

While understandable, this is at the same time unacceptable. It is highly unlikely that a simulation developer can provide a meaningful result when they did not understand the system they were intending to model. While the timeline might have been met, the result was likely meaningless. Worse yet, the result was likely wrong and might have adversely affected larger design or business decisions. Model only what you understand! If you don’t have a fundamental understanding of a technology, there is no way you can effectively model or simulate that technology. This step cannot be skipped in a successful M&S effort. If this step cannot be completed, it is better to not proceed down the path of M&S development.

1.3.2 Understand Your Model

It is quite common for the network engineer to utilize off-the-shelf tools, either commercial or open source. This approach typically lends itself to a faster M&S development cycle; however, it is imperative that the network engineer has a full understanding of the tools being used. Most simulations are likely to have errors—even commercial tools. New simulation implementations almost always contain errors. Simulation implementations can make assumptions that may not accurately reflect the exact performance metric of interest. If the simulation developer utilizes existing simulation implementations, it is imperative to allocate the proper amount of time to closely examine that implementation to fully understand what that code is doing and what it is not doing. There is no better way to lose credibility than to not be able to answer questions about one’s own results. Understand what you have modeled! There are resources available to help with this, including technical support for commercial tools, online newgroups and user forums for open source tools, and in some cases the simulation designer can contact the author directly (e.g., a contributed simulation to an open source project).

1.3.3 Make Your Results Independently Repeatable

Many academic papers such as [1–3] have discussed the lack of independent repeatability in wireless network simulation results due to improper documentation of the simulator being utilized, model assumptions, and inputs and outputs. There are subtle parameters and assumptions embedded in simulators such as NS-2 and GloMoSim that certainly can impact all results. Often default simulator parameters are chosen that may not capture the intended network conditions for a given scenario [2]. Perhaps the larger problem is that simulation results are often presented as ground truth and not as a relative ranking of a new idea compared to existing ideas. That is, the literature survey component must always be present in wireless network research and simulation results should be compared to existing results to demonstrate advantages and disadvantages of new ideas. Moreover, new simulation results must be compared with results in existing literature using the same simulator, underlying assumptions, and parameter conditions.

1.3.4 Carefully Define M&S Requirements

This is an activity that is too often ignored or given superficial treatment. The authors would argue that network engineers all too often rush into an M&S effort without a clear idea of what they are hoping to accomplish. This is a surefire recipe for failure.

The first step is to clearly understand the metrics of interest that would be generated by a simulation. Is overall network throughput the metric of interest? Is BER the metric of interest? End-to-end delay? Not all simulation tools necessarily lend themselves to the same types of output metrics, so it is important to define these metrics so that tool selection is an informed process.

The next step is to clearly define the required performance of the simulation to be developed. This book contends that there are four primary dimensions of performance:

Cost: The overall investment in resources towards the development and maintenance of the M&S activity. This includes not only original platform costs, but also development time, upgrade and maintenance costs, and troubleshooting.Scalability: The total complexity of the system to be simulated. There are two factors that must be considered: network size in terms of number of nodes, and network traffic model in terms of number of messages per unit time. These two factors will drive the computational complexity of the simulation and will ultimately be the limiting factors in the size of the network that can be simulated. This is generally governed by software complexity and hardware capability.Execution Speed: For a given simulation scenario, how quickly can that simulation complete and provide the desired output metrics? This is generally governed by software complexity and hardware capability.Fidelity: For a given simulation scenario, how accurately does the simulation’s output metrics reflect the performance of the real system.

Note that these dimensions of performance are contradictory; not all per­formance dimensions can be achieved simultaneously. If you desire a highly scalable simulation with fast execution speed, then the fidelity is likely going to be lower. Do you want high fidelity and scalability with reasonable execution speed? Then the cost will likely be very high. In general, you can pick any three of these metrics.

A common pitfall is to begin an M&S effort with unrealistic expectations. Is it really feasible to model the entire Internet down to every platform with bit-level fidelity? Probably not. Is it possible to model the entire Internet down to every platform with many simplifying assumptions? Probably, but it is unlikely to be useful.

When defining requirements and expectations for an M&S effort it is recommended to begin by choosing the required fidelity. How accurate of an output metric is required? A successful effort will always begin with this metric because, without a meaningful degree of fidelity, any M&S activity is meaningless, despite its scalability or execution speed. Once the required fidelity is established, one can then begin placing limitations on simulation capabilities accordingly. Cost is generally bound by an allocation of resources. So given a known cost constraint and a known fidelity requirement, we can then begin building a conceptual model for the simulation. The target fidelity will mandate the inclusion of particular system characteristics with great detail and inputs with particular degrees of accuracy, and also allow for relaxation on other system details and input accuracy. Note that this exercise requires a strong understanding of the system being modeled and on the underlying concepts of wireless networking. Remember, model only what you understand! Once a conceptual model is designed, the hardware platform can be chosen in accordance with cost constraints to maximize scalability and execution speed performance.

1.3.5 Model What You Need and No More

One of the first decisions that the simulation designer must face is to determine what he or she is attempting to demonstrate through simulation and what is the most simplistic model that captures all necessary components. The engineering tradeoff is that increased detail can provide higher-fidelity output from the model, but at the cost of complexity—potentially introducing error and certainly increasing debugging time and execution time. The designer must also realize that a model is always an abstraction from the real world. Wireless networking devices not only have variables within the standards to which their underlying protocols comply, but there is variability introduced into each manufacturer’s products. At least a subset of the key variables should be included: transmission power, antenna type and gain, receiver sensitivity, and dynamic range should be considered in the model, but the extent of modeling detail required depends on the particular system and desired output for a given scenario. Regardless of the level of detail included, a simulation will always be an approximation of the real system; an arbitrarily high degree of fidelity is generally not possible. Also, the cost of increased fidelity at some point becomes greater than the marginal utility of the additional fidelity. This is illustrated in Figure 1-2. It is imperative to understand the limitations of M&S techniques and to understand the relationship between cost and fidelity so that an M&S effort does not become an over-engineered effort in futility.

FIGURE 1-2. The cost of simulation fidelity.

How much detail is sufficient in a simulation to capture the essence of the real-world network being modeled? Unfortunately, the answer to this question is that it depends on the particular simulation scenario. The reader should first decide exactly what is the problem that he or she seeks to address through simulation. What are the inputs and the outputs of the model? Some outputs may be independent of specific details in the model, while others may be correlated and therefore seriously affected if those components are abstracted. Simulation always takes the form of an abstraction of a system to allow the designer to gain some insight from investigating various operating scenarios of the system. In many cases the simulation allows the user access to knobs and switches that may not be available on the actual system. Consider reliability testing for a consumer networking product that must be tested under as many operating conditions as possible, where the prevention of erratic behavior in a consumer product translates to significant savings for a company. Yet in other cases the researcher desires to investigate a system’s reaction to a single condition that may be unlikely to occur in real life. Perhaps testing the actual system under this condition could be harmful and simulation is the only way to examine the problem.

The next step is to decide how much of the system must be implemented for the simulation results to be valid. Ultimately, the reader is going to have to decide the level of detail required in his or her simulation, but this book is intended to guide the reader towards formulating a more educated decision. First, the reader must consider the engineering tradeoffs between adding more detail to a model and increased computational time, increased complexity, and increased debugging time. This simulation trade space is illustrated in Figure 1-3. A more complex simulation may attempt to capture an actual system’s complete behavior, but at that point the simulation is generally inflexible to scenario modifications, more prone to errors, and more computationally intensive. A more abstract approach that focuses only on the basic behavior of a system is generally very flexible, easier to debug, and has a shorter execution time. But, it may not capture the behavior of interest.

FIGURE 1-3. Illustration of the simulation trade space.

1.3.6 Avoid M&S When M&S Does Not Make Sense