Distributed Storage Networks E-Book

Thomas C. Jepsen

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 369

Veröffentlichungsjahr: 2013

Beschreibung

The worldwide market for SAN and NAS storage is anticipated to grow from US $2 billion in 1999 to over $25 billion by 2004. As business-to-business and business-to-consumer e-commerce matures, even greater demands for management of stored data will arise.

With the rapid increase in data storage requirements in the last decade, efficient management of stored data becomes a necessity for the enterprise. A recent UC-Berkeley study predicts that 150,000 terabytes of disk storage will be shipped in 2003. Most financial, insurance, healthcare, and telecommunications institutions are in the process of implementing storage networks that are distributed to some degree. For these institutions, data integrity is critical, and they will spend much time and money on planning.

One of the primary obstacles to implementing a storage network cited by enterprise IT managers is a lack of knowledge about storage networking technology and the specific issues involved in extending a Storage Area Network (SAN) or Network Attached Storage (NAS) over the Metropolitan Area Networks (MAN) or Wireless Area Networks (WAN). Distributed Storage Networks: Architecture, Protocols and Management addresses the "terminology gap" between enterprise network planners and telecommunications engineers, who must understand the transport requirements of storage networks in order to implement distributed storage networks. Jepsen comprehensively provides IT managers, planners, and telecommunications professionals with the information they need in order to choose the technologies best suited for their particular environment.

• Addresses a hot topic that will become increasingly important in the coming years
• Enables high-level managers and planners to make intelligent decisions about network needs
• Includes example network configurations providing solutions to typical user scenarios
• Fills the "terminology gap" between enterprise network managers and telecommunications engineers who must understand the transport requirements of storage networks in order to implement distributed storage area networks

A fundamental resource for all network managers, planners and network design engineers, as well as telecommunications engineers and engineering, computer science, and information technology students.

Leseprobe

Contents

Foreword

Preface

Acknowledgements

List of Figures

List of Tables

1 Introduction to Storage Networking

1.1 Overview

1.2 Evolution of Storage Networking

1.3 Terminology

1.4 Storage Concepts

1.5 SAN Applications

1.6 Summary

2 Applications for Distributed Storage Networking

2.1 Storage Integration

2.2 Remote Backup/Restoral

2.3 Disk Mirroring

2.4 Data Migration

2.5 Business Continuity/Disaster Recovery

2.6 Remote Operation of Peripheral Devices

2.7 Mainframe/Open Systems Connectivity

2.8 Network Attached Storage (NAS)

2.9 Summary

3 Distance Considerations for Storage Networks

3.1 Physical Layer

3.2 Protocol Considerations

3.3 Caching

3.4 Summary

4 Architectures for Distributed Storage Networking

4.1 Storage Networking in the Business Park

4.2 Storage Networking in the Metro Network

4.3 Storage Networking in the Wide Area Network

4.4 Summary

5 Protocols for Distributed Storage Networking

5.1 Small Computer Systems Interface (SCSI)

5.2 Enterprise Systems Connection (ESCON)

5.3 Fiber Connection (FICON)

5.4 Fibre Channel (FC)

5.5 Gigabit Ethernet (GigE) and 10 Gigabit Ethernet (10 G Ethernet)

5.6 Protocol Summary

6 InfiniBand

6.1 Applications

6.2 Standards

6.3 Network Topology

6.4 Addressing

6.5 Data Framing

6.6 Physical Layer

6.7 Summary

7 MAN/WAN Protocols for Distributed Storage Networking

7.1 Wavelength Division Multiplexing (WDM)

7.2 Time Division Multiplexing and SONET

7.3 Asynchronous Transfer Mode (ATM)

7.4 Generic Framing Procedure (GFP)

7.5 Summary –WAN/MAN Protocols

8 Storage Over IP

8.1 Overview

8.2 Internet SCSI (iSCSI)

8.3 Fibre Channel and IP Networks

8.4 Summary

9 Storage Management Requirements

9.1 Overview

9.2 Standards

9.3 The SNIA Shared Storage Model

9.4 The SNIA Management Model

9.5 The SNIA Prototype

9.6 Distance Considerations for Storage Management

9.7 Summary

10 Security Considerations

10.1 Overview

10.2 Physical Security

10.3 User Login and Administration

10.4 Management Interfaces

10.5 Firewalls

10.6 Encryption and IPSec

10.7 Public Key Infrastructure and Digital Certificates

10.8 Virtual Private Networks (VPNs)

10.9 SAN Security

10.10 NAS Security

10.11 Analyzing Network Architecture for Security Issues

10.12 Summary

11 Planning Distributed Storage Networks

11.1 Selecting an Architecture

11.2 Optical Fiber Planning

11.3 Security Planning

11.4 Outsourcing Storage

11.5 Return on Investment (ROI)

11.6 Summary

12 Glossary of Terms

Bibliography

Index

Copyright © 2003

John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England

Telephone [phone]

Email (for orders and customer service enquiries): [email protected]

Visit our Home Page on www.wileyeurope.com or www.wiley.com

All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1T 4LP, UK, without the permission in writing of the Publisher. Requests to the Publisher should be addressed to the Permissions Department, John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England, or emailed to [email protected], or faxed to ([phone].

This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the Publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought.

Other Wiley Editorial Offices

John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA

Jossey-Bass, 989 Market Street, San Francisco, CA 94103-1741, USA

Wiley-VCH Verlag GmbH, Boschstr. 12, D-69469 Weinheim, Germany

John Wiley & Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australia

John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop #02-01, Jin Xing Distripark, Singapore 129809

John Wiley & Sons Canada Ltd, 22 Worcester Road, Etobicoke, Ontario, Canada M9W 1L1

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.

British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library

ISBN 0-470-85020-5

Foreword

Having been a technical reviewer for Marc Farley’s “Building Storage Networks” and Tom Clark’s “Designing Fibre Channel Storage Area Networks”, as well as making my own small contribution (“Storage Area Network Essentials,” co-authored with Richard Barker, and published by John Wiley & Sons), I feel very much a part of the growing body of storage networking literature. I was therefore particularly gratified to be asked to contribute a foreword to Tom Jepsen’s “Distributed Storage Networks.”

Tom has made a unique contribution to the literature. Today, most large enterprises operate mixtures of mainframe, UNIX, and Windows servers in their data centers. All of these platforms are moving inexorably toward a networked storage concept, but they are taking their own technology directions and moving at their own speeds. It’s possible to find sources of information about open system storage networks and other sources that deal with ESCON and FICON, but IT designers, implementers, and managers who have to cope with the reality of mixed environments are hard put to acquire the unified body of knowledge that allows them to deal with the different capabilities and requirements of the storage networks for which they are responsible. Tom’s book does a masterful job of presenting this much-needed unified body of knowledge in four areas:

it covers both mainframe and open storage network technologies side by side so that readers can develop a broad architectural view as they learn the details

it covers both technology and implementation, explaining both the whats and whys of the networks themselves, and the hows of deploying them

it deals both with storage networking within the data center and with long-distance networks, and their unique challenges and benefits

it deals with all these topics both at the overview level (Chapters 1 and 2) and in depth (Chapters 3–11), providing real value to both executives and hands-on workers.

”Distributed Storage Networks” is also unique in that it’s designed both to be read and to be referred to. The conceptual information, particularly in the chapters that cover applications and implementation, is presented at just the right level of detail for architects and implementers–enough to stimulate the thinking processes, but not the mind-numbing detail that only a standards engineer could love. Particularly trenchant in today’s uncertain world is Tom’s advice on security–”build it in; don’t count on adding it on later.” The many charts and tables are a handy reference for the inevitable “speeds and feeds” part of storage network design.

Whether you’re an architect, designing your organization’s first or tenth storage network, an application implementer who’s concerned with reliability, performance, and security, an administrator needing an overview of the bundle of technologies for which you’re responsible, or a CIO trying to make some sense of the brave new world of distributed storage, Distributed Storage Networks has something to offer you. It’s my candidate for 2003’s “If you can only have one book on storage networking, make it this one” award.

Paul Massiglia

Technical Director, VERITAS

Software Corporation

Director, VERITAS Publishing

Preface

Stored data is the most valuable asset that any organization has, and the accuracy and reliability of that data are key factors in staying competitive in today’s world. The globalization of business creates a need to link together geographically separate databases in a manner that is transparent to the end user. New government regulations in the healthcare and financial industries create requirements to provide secure and reliable data backups at remote locations without impacting system performance, and to provide strategies for business continuity in the event of data loss.

Implementing these functions requires a new mindset on the part of enterprise planners and managers. The familiar computer-centric model of data processing is rapidly being replaced by a storage-centric model in which storage is geographically distributed over a large area, perhaps even the entire world. This book is intended to provide an overview of the architectures, protocols, and management functions needed to understand this new environment.

The concept behind this book emerged as a result of two presentations I gave on storage networks in 2001, at EntNet@Supercom in Atlanta, Georgia, USA, and at the Asia-Pacific Network Operations and Management Symposium in Sydney, Australia. I would like to thank the individuals who made those presentations possible, and provided me with the germ of an idea for a book: Lundy Lewis, Aprisma; Shri Goyal, Verizon; and Masayoshi Ejiri, Fujitsu Ltd. Their encouragement and support are greatly appreciated. I would also like to thank the individuals and publications who granted me permission to reprint copyright material: Monica Vago and Rose Maginniss, ANSI/INCITS; Jacqueline Hansson, IEEE; Andrea Westerinen, Distributed Management Task Force; Arnold Jones, Storage Networking Industry Association; Casper Yu, InfiniBand Trade Association; and the IBM Journal of Research and Development. And lastly, I would like to thank Paul Massiglia for his enthusiastic and supportive comments.

Acknowledgements

The author would like to acknowledge the use of the following source material.

Tables 5.6 to 5.13, 5.27, 5.28, 5.25 and 5.26 are reprinted with permission from, ANSI INCITS 296-1997 SBCON, ANSI INCITS 230-1994 (R1999) Fibre Channel, ANSI INCITS 342-2001, Fibre Channel Back Bone (FC-BB).

Figures 7.8 and 7.16 are reprinted with permission from, ANSI INCITS 342-2001, Fibre Channel Back Bone (FC-BB).

From IEEE Std 802.3-2002*, Copyright © 2002, by IEEE. All rights reserved.

Tables 5.30 to 5.33 reprinted with permission from IEEE Std 802.3-2002*, “Carrier sense multiple access with collision detection (CSMA/CD) access method and physical layer specifications”, Section 3, Table 38-3, Table 38-4, Table 38-7, Table 38-8 and Table 38-10. Copyright © 2002, by IEEE. The IEEE disclaims any responsibility or liability resulting from the placement and use in the described manner.

From IEEE Std 802.3ae-2002*, Copyright © 2002, by IEEE. All rights reserved.

Tables 5.34 to 5.40 reprinted with permission from IEEE Std 802.3ae-2002*, “Part 3, Amendment: Media: Access Control (MAC) Parameters, Physical Layers, and Management Parameters for 10 Gb/s Operation”, Copyright © 2002, by IEEE. The IEEE disclaims any responsibility or liability resulting from the placement and use in the described manner.

Figures 6.4 and 6.5, along with Tables 6-1 to 6-9. Infiband Architecture Specification, Vol. 1, Rel 1.1, Infiband Architecture Specification, Vol. 2, Rel 1.1

The charts and tables referenced above from the Infiband Architecture Specification, Vol. 1, Rel 1.1 and Infiband Architecture Specification, Vol. 2, Rel 1.1, is provided “AS IS” and without warranty of any kind, including, without limitation, any express or implied warranty of non-infringment, merchantability or fitness for a particular purpose.

In no event shall IBTA or any member of IBTA be liable for any direct, indirect, special, exemplary, punitive, or consequential damages, including, without limitation, lost profits, even if advised of the possibility of such damages.

Figures 8.1 to 8.4, IETF draft-ietf-ips-iscsi20, 19/01/2003, Figures 8.8 and 8.9, IETF draft-ietf-ips-fcovertcpip, 12/08/2002 and Figure 8.13, draft-ietf-ips-ifcp- 14/12/2002.

Copyright © The Internet Society (2002/2003). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restrictions of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works.

List of Figures

1.1

Mainframe Storage Network Architecture (1960s)

1.2

Traditional Computer Network Architecture

1.3

Computer Network Architecture with Storage Network

1.4

Point to Point Topology

1.5

Arbitrated Loop Topology

1.6

Arbitrated Loop Topology – Hub Configuration

1.7

Switched Fabric Topology

1.8

Redundant Fabric Switch Configuration

1.9

Two Tier Layered Architecture

1.10

Core/Edge Backbone Architecture

1.11

Network Attached Storage (NAS)

1.12

SAN versus NAS - The User Viewpoint

1.13

Types of Data Sharing

1.14

Storage Virtualization

1.15

RAID Architecture

1.16

LAN-Based Backup

1.17

SAN-Based Backup

1.18

Processor-Centric Disk Mirroring

1.19

Storage Centric Disk Mirroring

2.1

Extending SAN Functions Across the MAN or WAN

2.2

Distributed SAN for Storage Sharing and Integration

2.3

Remote Backup/Restore Using Distributed Storage Network

2.4

Processor-Centric Disk Mirroring Using Remote Mirror

2.5

Storage Centric Remote Disk Mirroring

2.6

Split Mirror Copy Using Three Mirrors

2.7

Use of Storage Network for Data Migration

2.8

Distributed SAN for Disaster Recovery/Business Continuity

2.9

Using a Storage Network for Mainframe/Open Systems Connectivity

2.10

NAS Head Architecture

3.1

Bus/Tag Architecture

3.2

SCSI Bus Topology

3.3

Parallel Bus Signal Impairments

3.4

Computing Loss Budgets for Optical Links

3.5

Differential Mode Delay (DMD)

3.6

Use of Wavelength Division Multiplexing

3.7

Storage Protocol Data Transfer Functions

3.8

CCW Pipelining in FICON

3.9

Effect of Data Droop in SAN/WAN Applications

3.10

Effect of Data Droop on I/O Performance

3.11

Distance Limitations for Common Storage Protocols (Native Mode)

4.1

Distance Versus Protocol for Storage Protocols

4.2

Using Native Mode ESCON or Fibre Channel to Provide Storage Connectivity in the Business Park (<10km)

4.3

Extending ESCON/Fibre Channel Links in the MAN Using Link Extenders

4.4

Extending ESCON/Fibre Channel Links in the MAN Using Pt-to-Pt WDM

4.5

Extending ESCON/Fibre Channel Links in the MAN Using Mesh WDM

4.6

Extending ESCON/Fibre Channel Links in the MAN Using a WDM Ring Configuration

4.7

Extending ESCON/Fibre Channel Links across the WAN

4.8

Storage Protocols in Enterprise, MAN, and WAN Environments

5.1

SCSI Specification Roadmap

5.2

SCSI Network Topology

5.3

Bus Arbitration and Selection Phases

5.4

Information Transfer Phase – Typical READ Operation

5.5

ESCON/SBCON Network Architecture

5.6

ESCON Parallel Processing Configuration

5.7

Link Control Frame Format

5.8

Link Control Frame Header Format

5.9

Device Frame Format

5.10

Device Frame Link Header Format

5.11

Device Header Format

5.12

DIB Format for Data, Command, Status and Device Control Frames

5.13

Initiation of I/O Operation - Non-Immediate Command

5.14

I/O Initiation for Immediate Command

5.15

Ending I/O Operations

5.16

FICON Configurations

5.17

FICON CCW Pipelining

5.18

Fibre Channel Arbitrated Loop Network Topology

5.19

Fibre Channel Switched Network Topology

5.20

Mixed Fabric/Loop Configuration

5.21

Fibre Channel Flow Control Mechanisms

5.22

Fibre Channel Protocol Architecture

5.23

Fibre Channel Frame Format

5.24

Fibre Channel Frames, Sequences and Exchanges

5.25

FLOGI/PLOGI Payload Used for Login

5.26

NPort Login Process

5.27

Class 1 Data Transfer Operation

5.28

Class 2 Service Frame Flow

5.29

Class 2 Data Transfer Operation

5.30

Class 3 Data Transfer Operation

5.31

Conventional Ethernet LAN Topology

5.32

CSMA/CD Protocol

5.33

Point-to-Point Gigabit Ethernet Topology

5.34

Relationship of Ethernet Standards to OSI Protocol Stack

5.35

TCP Connection State Diagram

5.36

TCP Connection Establishment Sequence

5.37

IP Datagram Format (IPv4)

5.38

Ipv6 Header Format

5.39

MAC Frame Data Encapsulation and Protocol Layers

5.40

Ethernet Frame Format

6.1

InfiniBand Architecture - Single Processor Configuration

6.2

Multi-Processor InfiniBand Configuration

6.3

InfiniBand Work and Completion Queues

6.4

Relationship of IBA to OSI Protocol Model

6.5

IBA Packet Framing

7.1

WDM Principle of Operation

7.2

SONET Point-to-Point Architecture

7.3

SONET Unidirectional Path Switched Ring (UPSR) Configuration

7.4

SONET 2-Fiber Bidirectional Link Switched Ring (BLSR) Configuration

7.5

SONET 4-Fiber Bidirectional Link Switched Ring (BLSR) Configuration

7.6

SONET STS-1 Frame Format

7.7

Protocol Layering for SONET/SDH Transport of Fibre Channel Packets

7.8

Encapsulation of Fibre Channel Frames using HDLC

7.9

ATM Protocol Architecture

7.10

ATM Cell Format

7.11

CPCS and SAR Sublayers for AAL5

7.12

ATM VCIs, VPIs, and Connections

7.13

SVC Connection Setup

7.14

TC and PMD Layer Functions

7.15

Protocol Layering for ATM Transport of Fibre Channel Packets

7.16

Encapsulation of Fibre Channel Frames using ATM AAL5 CPCS-PDU

7.17

GFP Frame

7.18

GFP-F and GFP-T Payload Format

8.1

iSCSI Conceptual Architecture

8.2

iSCSI Protocol Layering Concept

8.3

iSCSI Session State Transitions-Normal Data Transfer

8.4

iSCSI PDU Format

8.5

iSCSI PDU Basic Header Segment Format

8.6

iSNS Registration and Query Operation

8.7

Fibre Channel Frame Encapsulation Format

8.8

FCIP Protocol Layering

8.9

FCIP Conceptual Architecture

8.10

FCIP Entity Architecture

8.11

FCIP Encapsulation Format

8.12

FCIP Special Frame (FSF) Format

8.13

Example iFCP Fabric Implementation

8.14

iFCP Fibre Channel Frame Encapsulation

9.1

SNIA Shared Storage Model

9.2

Integrated File/Record and Block Architectures

9.3

SNIA Model for Multi-Site Storage Networking

9.4

CIM Core Model

9.5

CIM Object Model for Disk Resource Manager Prototype

9.6

Web Based Enterprise Management (WBEM) Architecture

9.7

Example XML Representation of CIM Object

9.8

SNIA Prototype Application

10.1

Protocol Architecture of TLS1.0

10.2

Security Checkpoints for a Storage Network

11.1

Channel Extension Architecture Planning Considerations

11.2

Sysplex Architecture Planning Considerations

11.3

ESCON to FICON Migration Options

11.4

SAN Planning Considerations

11.5

Planning Considerations for NAS

List of Tables

3.1

Common Optical Fiber Types and Applications

3.2

Loss Budget for Common Optical Storage Protocols