Network Programming and Automation Essentials E-Book

Claus Topke

• Herausgeber: Packt Publishing
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 382

Veröffentlichungsjahr: 2023

Beschreibung

Unleash the power of automation by mastering network programming fundamentals using Python and Go best practices

Purchase of the print or Kindle book includes a free PDF eBook

Key Features

• Understand the fundamentals of network programming and automation
• Learn tips and tricks to transition from traditional networking to automated networks
• Solve everyday problems with automation frameworks in Python and Go

Book Description

Network programming and automation, unlike traditional networking, is a modern-day skill that helps in configuring, managing, and operating networks and network devices. This book will guide you with important information, helping you set up and start working with network programming and automation.

With Network Programming and Automation Essentials, you'll learn the basics of networking in brief. You'll explore the network programming and automation ecosystem, learn about the leading programmable interfaces, and go through the protocols, tools, techniques, and technologies associated with network programming. You'll also master network automation using Python and Go with hands-on labs and real network emulation in this comprehensive guide.

By the end of this book, you'll be well equipped to program and automate networks efficiently.

What you will learn

• Understand the foundation of network programming
• Explore software-defined networks and related families
• Recognize the differences between Go and Python through comparison
• Leverage the best practices of Go and Python
• Create your own network automation testing framework using network emulation
• Acquire skills in using automation frameworks and strategies for automation

Who this book is for

This book is for network architects, network engineers, and software professionals looking to integrate programming into networks. Network engineers following traditional techniques can use this book to transition into modern-day network automation and programming. Familiarity with networking concepts is a prerequisite.

Leseprobe

Network Programming and Automation Essentials

Get started in the realm of network automation using Python and Go

Claus Töpke

BIRMINGHAM—MUMBAI

Network Programming and Automation Essentials

Copyright © 2023 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Mohd Riyan Khan

Publishing Product Manager: Mohd Riyan Khan

Senior Editor: Romy Dias

Technical Editor: Nithik Cheruvakodan

Copy Editor: Safis Editing

Project Coordinator: Ashwin Kharwa

Proofreader: Safis Editing

Indexer: Hemangini Bari

Production Designer: Vijay Kamble

Marketing Coordinator: Agnes D’souza

First published: March 2023

Production reference: 1100323

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80323-366-6

www.packtpub.com

To all engineers that work hard to connect humans to network infrastructure by automating them.

Contributors

About the author

Claus Töpke is a product developer and founder of Telcomanager. He has worked with large network service providers, such as Telstra, NBN Australia, NZ Telecom, AWS Australia, AWS US, and Embratel. He has also worked in conjunction with large network technology corporations, such as Nokia, Amazon, Juniper, and Cisco. He has been able to experience different job titles, passing through fields such as network engineering, network performance, product development, and software engineering. His experience with network automation has led to the construction of several products and systems for different companies. He also worked on network performance for his master’s thesis and wrote a book about service providers.

A special thanks to my wife and my whole family who have always supported me throughout the journey of this book. In addition, I must mention my little son Daniel, who tried very hard to get me away from the computer screen to play, an endeavor that was successful most of the time but was gunpowder for my motivation and inspiration. Also, a big thanks to Telcomanager for the support.

About the reviewers

Johan Lahti is a technology enthusiast with almost 15 years of experience working with computer networks. He has gained a wide range of experience including designing, building, and maintaining data center, enterprise, and service provider networks and automating those for various customers, ranging from ISPs to the public sector and large enterprises. He also teaches network and network automation for different educational institutes.

He is the founder and CEO of Acebit AB and works primarily as a senior consultant helping many different customers with design and architecture for network and infrastructure automation.

Radoslaw Majkut has worked for over 20 years in the computing technology field, including 13 years at Amazon and 3 years with his current employer, Google, in roles such as customer support, systems admin, SRE, and software engineer. He worked with Claus at Amazon on a team that developed a large-scale network simulator system.

Table of Contents

Preface

Part 1: Foundations for Network Automation

1

Network Basics for Development

Reviewing protocol layers, network device types, and network topologies

Protocol layers

LAN, WAN, internet, and intranet

Describing network architecture and its components

Diagrams

Network node names

The last-mile network

The physical architecture

The routing architecture

Types of failure

Failure detection techniques

Control plane and forwarding plane

Graceful restart

Illustrating network management components, network bastions, and more

ACL

Management system and managed elements

In-band and out-of-band management

Network telemetry

Management information base

Network bastions

FCAPS

Network planning

Network security

Summary

2

Programmable Networks

Exploring the history of programmable networks and looking at those used in the present day

Active networking

NodeOS

Data and control plane separation

Virtual network technologies

Virtual private networks

The Virtual Router Redundancy Protocol

The Virtual Extensible Local Area Network

Open vSwitch

Linux Containers

Virtual machines

SDNs and OpenFlow

History of OpenFlow

SDN architecture

OpenFlow and its future

Understanding cloud computing

Commercial cloud computing

The OpenStack Foundation

Cloud Native Computing Foundation

Using OpenStack for networking

OpenStack Neutron

The Neutron API

Summary

3

Accessing the Network

Working with the CLI

The command prompt

Serial access

Remote insecure access

Remote secure access

Using SNMP

SNMP agents and managers

An SNMP MIB

SNMP versions

SNMP primitive methods

SNMP security issues

Employing NETCONF

Motivation

OpenConfig

YANG

NETCONF

RESTCONF

Adopting gRPC

The letter g

Motivation

Overview

Protobuf

gRPC and network telemetry

Code examples using gRPC

Operating with gNMI

Protocol layers

The data model

The communication model

Service definition

gNMI-gateway

Summary

4

Working with Network Configurations and Definitions

Technical requirements

Describing the configuration problem

Source of truth

The startup configuration and the running configuration

Configuration states and history

Deployment pipeline

Network diagrams and automation

Using network definitions to aid automation

The router configuration render

Using configuration templates

Using Python engine templates

Using Go engine templates

Creating network definitions

Nested and hierarchical definitions

IP allocation considerations

Using files for definitions

File format

Names

Exploring different file types

XML files

JSON files

YAML files

Summary

Part 2: Network Programming for Automation

5

Dos and Don’ts for Network Programming

Coding topics

Peer review

Life cycle

Refactoring

Copying code and licensing

Code quality and perception

Architecture and modeling

Applying best practices in coding

Follow the standards

Mindful code writing

Making it extremely readable

Commenting your code

Use IP libraries

Follow naming conventions

Don’t shorten variable names

Avoid complex loops

Don’t repeat code

Coding formatters

Python Black

Python isort

Python YAPF

Go gofmt

Go golines

Go golangci-lint

Versioning and concurrent development

clone

checkout

commit

Mainline

Branching

Merging

Testing your code

Unit testing

Integration testing

E2E testing

Other testing

Summary

6

Using Go and Python for Network Programming

Technical requirements

Looking into the language runtime

What are compiled and interpreted languages?

Python interpreter

Go compiler

Pros and cons of programming runtimes

Using third-party libraries

Adding Python libraries

Adding Go libraries

Accessing network devices using libraries

Libraries to access the network via a CLI

Libraries to access networks using SNMP

Libraries to access networks using NETCONF or RESTCONF

Libraries to access networks using gRPC and gNMI

Summary

7

Error Handling and Logging

Technical requirements

Writing code for error handling

Adding error handling in Go

Adding error handling in Python

Logging events

Severity levels

Adding logging to your code

Adding event logging in Go

Add event logging in Python

Summary

8

Scaling Your Code

Technical requirements

Dealing with multitasking, threads, and coroutines

Multiprocessing

Multithreading

Coroutines

Adding schedulers and job dispatchers

Using classical schedulers and dispatchers

Working with big data

Using microservices and containers

Building a scalable solution by example

Summary

Part 3: Testing, Hands-On, and Going Forward

9

Network Code Testing Framework

Technical requirements

Using software for testing

Differences between emulation and simulation

Using device emulation

Scaling up emulation with containers

Connecting devices for testing

Using physical wires to connect

Using software to connect

Building a hybrid lab

Using advanced testing techniques

Using time dilation

Using monkey testing

Using artificial intelligence

Using network simulation

Using traffic control

Summary

10

Hands-On and Going Forward

Technical requirements

Using a network lab

Building our network lab

Launching the lab host

Checking the lab host

Connecting the devices

The OOB management network

Looking at the topology

Creating the connections between devices

Automating the connections

Looking into the automation program

Checking the connections manually

Adding automation

Link connection check automation

IP configuration automation

Additional network lab automation

Going forward and further study

Checking popular platforms and tools

Joining the network automation community

Summary

Index

Other Books You May Enjoy

Part 1: Foundations for Network Automation

The first part is dedicated to refreshing you on some of the network fundamentals and jargon, as well as discussing some important aspects of network automation that should be used as the foundation of your work. Additional bases for network automation, such as the methods and protocols used to access the network and how we should use the network configuration and definitions, are also discussed in this part.

This part has the following chapters:

Chapter 1, Network Basics for DevelopmentChapter 2, Programmable NetworksChapter 3, Accessing the NetworkChapter 4, Working with Network Configurations and Definitions

1

Network Basics for Development

This chapter is focused on explaining the basics and jargon used in computer networking. The idea is to build a good foundation to be used throughout the book.

If you are a network engineer or have experience in this field, you might want to skip it, or perhaps skim through it.

If you are a software developer with little network experience, this chapter is for you. It will help you build a solid base on network jargon that will be useful when writing code for network automation.

The following are the topics that we will cover in this chapter:

Reviewing protocol layers, network device types, and network topologiesDescribing network architecture and its componentsIllustrating network management components, network bastions, and more

Reviewing protocol layers, network device types, and network topologies

We have lots to talk about here. But due to the size restraints of this book, I have organized a summary with the most important aspects of today’s network jargon and explained them briefly. I hope you can find some new information to help your automation work.

Protocol layers

It’s important to note that there are several different standards for protocol layers, and the most academic one is the ISO organization called OSI model, which defines seven layers. But we are going to consider only five defined in the TCP/IP protocol stack, which is used on the internet. Here is a short summary of each of the layers:

Physical layer: In this layer are the technologies involved in the physical connection itself where the bits and bytes are transformed into the physical medium, such as the light in fiber optics, electricity in a cable, and radio waves in antennas. At this layer, physical checks can be implemented on the node input, such as power levels, collision, noise, and signal distortion, among other types of checks.Data link layer: Here, the information is called a frame, and it contains a delimited size, known as the maximum transmission unit (MTU). The reason is that a frame is a data representation in bytes that has to move from one node to another one and in a reliable manner without interruption. At this level, frame queues are present; the queues are used to place the frames on the physical layer in sequential order or in priority order. Some data link devices can prioritize certain types of frames, jumping to the front of the queue. At the data link layer, some checks are done, but within the frame itself, such as CRC or checksum. In addition, source and destination addresses can be added to the frame to differentiate destinations on a shared media. The information on the frame is normally used locally within the same organization. This layer is also known as the Ethernet layer.Network layer: This is also known as the IP layer, or the router layer. Here, the information is called a packet, and it contains the information that goes between nodes that are beyond the layer 2 domain (or the previous Ethernet layer). This level is where the routing protocols are used, the network address translation (NAT) does its job, some access control lists (ACLs) are present, and the control packets are, among other functions. The packet on this level has enough information to know where it came from and where it has to go. This layer is also responsible for fragmenting the packet into multiple packets if the frame MTU is smaller than the IP packet. The main information carried in the packet is the IP address and has source and destination addresses.Transport layer: The transport layer deals with data information that is called a segment. On today’s internet, only two types of protocols are used here, the User Data Protocol (UDP) and the Transmission Control Protocol (TCP). The idea is one provides more confirmation and control than the other. TCP has traffic flow control, packet loss detection, and packet retransmission, among other functions. UDP, on the other hand, is just the IP packet plus a little more information. The idea behind having TCP is to enhance communication on the unreliable internet, so the application has a guaranteed transport method. TCP has more overhead, with an additional header field, and might be slower in some cases than UDP. The transport layer adds a port number to the segment, which is carried inside every packet in the IP layer. The port number is used for two reasons: to designate which application is using the transport layer, such as port 80 for HTTP communication, and to associate it with a communication socket in the host. The port number is required for the source and destination, which will be used to designate the correct socket to communicate with the host.Application layer: This is the top of the layers, normally referred to by my professor as the cherry on the cake. An application layer is used to associate a socket on the host where data will be sent and received. The application normally handles the content of the data, such as page requests on HTTP. The software that we are producing in this book uses this layer to automate the network.

LAN, WAN, internet, and intranet

LAN, or local area network, is used to refer to networks that are local. Nowadays, it means networks that use the data link layer as the main communication, such as Ethernet. The reason why the name is more related to the communication layer than the geography is that technology has evolved, allowing Ethernet switches to communicate over thousands of kilometers. So, a LAN normally designates a topology inside the same organization using Ethernet, but not necessarily geographically in the same location.

WAN, or wide area network, is used to refer to networks that are remotely connected, or technologies that allow nodes to be far apart, such as extinct technologies such as X.25, Frame Relay, and Asynchronous Transfer Mode (ATM). Now, the term WAN is normally used to designate interfaces or networks that are connected to different networks, or in other words, networks that are not in the same organization, data link layer, or Ethernet domain.

Information

For more information about ATM, please refer to the article Technology and Applications in SSRN Electronic Journal, June 1998, by Jeffrey Scott Ray.

The internet is what you know, this gigantic network interconnecting everybody worldwide.

The term intranet was used when corporations were using the internet protocols to communicate internally on their network. The reason is that other technologies were competing with the internet TCP/IP protocol at that time, such as SNA and IPX. So, when the term intranet was used, it was simply to state that the corporate network uses TCP/IP. Nowadays, intranet refers to a network that is within the same organization and not connected to external nodes. Therefore, the network is safe from external interference.

Point-to-point connections

A point-to-point (P2P) connection is used to interconnect two nodes. A link between two nodes is normally a P2P connection (as shown in Figure 1.1), unless using media such as satellite or broadcast antennas. This connection can either be back to back or not. The term back to back is normally used to indicate that the nodes are connected directly without any other physical layer between them, such as repeaters. Therefore, back-to-back connections have limited distances due to the noise and distortion introduced in the connection as the wiring gets longer. Depending on the speed and the technology used, the distances are limited to within the same room or building.

Figure 1.1 – A P2P connection

Star or hub-spoke topologies

Star or hub-spoke topologies are used in small and medium companies, where one office is the main distributor and the other locations are consumers. The topology looks like a star, and network elements are smaller and simpler at the remote locations, while being larger and complex at the main distributor (see the example in Figure 1.2).

Normally, these types of topologies can scale up to hundreds of nodes, but depending on the traffic, the requirements can scale to thousands. Let’s look at two examples that illustrate the scale of these topologies.

For instance, in a bank, the automated teller machines are distributed in remote locations, where the main computer is located in the main branch. This can scale to thousands of remote machines as the traffic requirements are small in terms of byte transfer on a teller machine.

On the other hand, if you have a supermarket chain using a star topology, it won’t scale to thousands of remote machines, as each supermarket requires a large amount of data transfer to handle all transactions and employees.

So, the use of star topologies is limited to the amount of traffic it can handle in the central node. In the star topology, we have two device functions, a device that will be either at the remote location or in the main office.

Network capacity planning is trivial when dealing with star topologies, as the main office node is updated as it grows.

Figure 1.2 – A star topology

Hierarchical or tree topologies

Hierarchical topologies are used to optimize traffic, where larger nodes are used to aggregate traffic to smaller nodes in a hierarchical matter (see the example in Figure 1.3). These topologies can scale to thousands of nodes; however, because of the number of nodes in the path, the topologies can cause undesirable latency and extra node costs.

An internet service provider normally uses a hierarchical topology to concentrate customer traffic in certain remote locations before aggregating even more in other locations.

There is no limit on the number of nodes on this type of topology, and it’s one of the foundations of the internet global infrastructure.

In the hierarchical topologies, we have multiple device functions, the customer premises equipment (CPE), aggregators, distributors, core, and peering, among others.

Depending on the size of this topology, it can introduce a longer path, which will add significant latency. For instance, in Figure 1.3, A1 has to cross five hosts to reach A7.

Network capacity planning is focused on the aggregation points, and augmenting the network is not that difficult.

Figure 1.3 – A hierarchical or tree topology

Clos topologies

This type of topology is also known as a Clos network or fabric. This topology is used to increase the number of ports without compromising latency and throughput and is often used in data centers. This topology is composed of at least three stages. Note that there is no oversubscription or aggregation like in the hierarchical topologies. The Clos topology provides the same amount of available bandwidth on the input and output. The stage names are normally spines and leafs. The spines are always in the center and only have connections to the Clos nodes. Leaves are used to connect to external devices or networks.

Figure 1.4 shows an example of a 16-port Clos network. Note that normally, all connections between a spine node to a leaf node are back to back:

Figure 1.4 – A Clos topology

Why are these topologies used? To increase the number of ports available without compromising throughput. This kind of topology is also used inside a router to provide connectivity between interface cards. Some companies use small devices to increase the number of ports that are offered without raising the cost as smaller devices are normally cheaper.

Important note

One additional characteristic of the Clos network is that it has the same distance between any two external ports (in terms of nodes in the path), therefore the latency in normal conditions is the same. For instance, in Figure 1.4, the latency between an external port on node L1 to an external port on L4 or E1 is the same.

Important note

More information on Clos networks can be found in an interesting paper from Google called Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google’s Datacenter Network – ACM SIGCOMM Computer Communication Review, Volume 45, Issue 4, October 2015.

Mixed topologies

A mixed topology is used in large corporations where latency and traffic are both important to care of. Normally, star topologies and P2P are used to shorten paths and reduce latency, whereas hierarchical topologies are used to optimize and aggregate traffic, and finally, Clos networks to increase the number of ports.

Modern cloud service providers are migrating to a more complex topology, where there are connections between elements where latency matters and aggregate device functions where traffic matters.

Network capacity planning is normally harder because connections are not totally hierarchical and aggregation points are not necessarily part of all traffic paths. An example of this kind of mixed topology is shown in Figure 1.5:

Figure 1.5 – A mixed topology

Interface speeds

A very important point that some engineers get confused about is the interface speed representation. 1 KB in memory representation is 2^10 or 1,024 bytes and 1 GB is 2^30, which is 1,073,741,824 bytes. For interface speeds, the same does not apply, and 1 Kbps is actually 1,000 bits/second, while 1 Gbps is 1,000,000,000 bits/second (more details can be found at https://en.wikipedia.org/wiki/Data-rate_units).

Device types and functions

Network devices used to have specific functions as CPU and memory were scarce and expensive. Nowadays, network devices can have multiple functions when required. In large networks, devices have fewer functions as they tend to get overloaded easier when traffic demands increase. Here are some of the functions that a device can have:

Hub: This is a very old term to designate a device that only repeats the physical signal.Switch: A device that works only on the data link layer. It is normally used in LANs, and it works by switching frames. The most common protocol used on these devices to control paths is the Spanning Tree Protocol (STP).Router: A device that works only on the network layer or IP Layer. It is used to interconnect multiple LANs or create long-haul remote connections. Internally, a router routes packets using a routing protocol to exchange route information with other routers. Some routers can also switch frames or work as a switch.NAT: NAT is devices that replace source and destination IP addresses to allow the use of private IP addresses or to isolate internal traffic from external traffic.Firewall: Normally, devices that control the traffic that passes through it by looking into the content of the frame or the packet. There are several different types of firewalls, and some might be super complex, which includes encrypting and decrypting traffic.Load balancers: When servers can’t handle too many clients because of hardware limitations, load balancers can be used to deal with the client demands by sharing the client request between several servers. Those devices also look into the packet content to determine which server would get the traffic.Network server: A computer used to provide some sort of service to the network, for instance, an authentication server, an NTP server, or a Syslog collector.

Oversubscription

In network jargon, this term is used to describe nodes or links in the network that aggregate traffic from other parts of the network and statistically use it to their advantage. For instance, they have a 1 Gbps interface to connect to the internet and 1,000 customers with 10 Mbps interfaces to use the service, which is an oversubscription of 1 to 10. This practice is quite normal and is only possible to use because of the characteristics of the client’s traffic that allow such aggregation without degradation. There are lots of mathematical models and papers on the internet describing this behavior and how to use it in your favor.

But some traffic can’t be aggregated without being degraded. In a data center, the traffic that can’t be oversubscribed is the traffic between servers, such as remote disk, data transfers, and database replicas. In this scenario, the best solution is to interconnect them without oversubscription using a solution such as non-blocking Clos topologies.

Browsing web pages, watching videos, and receiving messages from most of the traffic on the internet, which easily allows the aggregation technique without degradation.

Important note

More information on oversubscription can be found in the paper Evaluating Impacts of Oversubscription on Future Internet Business Models by A. Raju, V. Gonçalves, and P. Ballon – Published in Networking Workshops, 25 May 2012 – Computer Science.

In this section, we went over the basic components of computer networks, including protocols, topology types, interface speeds, and device types. By now, you should be able to identify these terms more easily and will be familiar with their meanings, because we are going to use these terms throughout this book. Moving on, we are going to review more terms related to network architecture.

Describing network architecture and its components

The term network architecture was introduced in the early 2000s, mimicking roles in the construction industry, where architects design and civil engineers build. Different companies use the term differently, but in this book, network architecture will be used to refer to the design of the network and its functions.

For a good network architecture, it is desirable to have a document describing in detail the first three layers of the network, from the physical layer to the routing layer. With this documentation, it is easy for the engineers to understand the physical connections, the Ethernet domains, and the routing protocols used.

Diagrams

A network diagram is mostly like a map, where the cities are the nodes and the roads are the links that connect them. For a network engineer, diagrams are crucial to describe how nodes are connected, and they also can group and demarcate important areas. A good diagram is easy to interpret and follow how data flows.

There are up to three types of diagrams; they can be integrated on the same page and graph, or they can be separated onto different pages. The main diagrams are one to show the physical connections, which can include the technology involved in the data link layer, and the switching and routing diagrams.

In Figure 1.6, we can see an example of a network diagram:

Figure 1.6 – Example of a network diagram

Figure 1.7 shows examples of network diagram symbols:

Figure 1.7 – Network diagram symbols

Network node names

A network node is a device that is essentially used to interconnect and serve as a transport of the data in the network. It can be either a hub, a switch, or a router. To help network engineers identify the node function, names are used to describe their main function. Here are some of them:

Transit router: These are routers that have interfaces with other service providers. These links are normally used as a service to access other networks, therefore they have a cost because they are normally connected to other big carriers.Peer router: These routers have interfaces with other networks in a peer configuration, meaning none of the parts pay to use it. In these links, only the traffic between the peer companies is exchanged, and the traffic destinated to outside networks are not allowed. Accessing external networks would be the case when using transit routers.Core router: These are nodes that are in the center of the network. They normally handle a large amount of traffic and have high-speed interfaces. Their throughput capacity is the highest in the network, but they have fewer interfaces as they concentrate the traffic of the network.Distribution router: These are nodes that normally connect to the core and aggregation routers. They normally interconnect different locations of the network. They don’t have many interfaces and their throughput capacity is high, but not as high as the core router.Aggregation router: These routers normally aggregate the traffic from the access routers. They are normally located in the same area or location as the access routers, and they have fewer interfaces compared to the access routers.Access router: Some architects add a node that connects all last-mile networks or CPE nodes. These routers are located closer to the customer and have more interfaces than any other router.Top of the rack (TOR): TOR refers to nodes that can be either a switch or a router, depending on the architecture. They are responsible for connecting the servers in the rack to the rest of the network.Clos rack: A Clos network, as described before, is a technique to add connectivity to multiple servers using small devices. A Clos rack is seen by the rest of the network as a single unique block, and in terms of architecture, it acts as a single node, normally used as a single router with a large number of interfaces.CPE: CPE is the node that is installed at the customer’s location. It normally has one interface connecting to the last-mile network and one local interface that can be an Ethernet or a wireless Ethernet. These devices can also implement NAT, firewall and, in some cases, they have multiple local interfaces, which can act as a switch and a router. These nodes are cheap and small with very low throughput capacity compared to the other nodes.

The last-mile network

This term is used to describe the architecture used to connect the customer to the network. Normally, this term is only used for ISPs, but some corporations also use it to interconnect their branches.

The last-mile network has a range of coverage and normally doesn’t cross the 1 km mark but depends on the type of technology used. Here are some of the most common last-mile networks:

Cable TV: There are several technologies used here to provide data communication using the cable TV that the customer has installed. The most used one is DOCSIS, which in 2017 was upgraded to version 4. This solution uses a single cable that is shared to several premises.Digital subscriber line (DSL): DSL uses the old telephone line to pass data communication. For that, there are lots of standards, and the most common ones are VDSL and ADSL. The DSL solutions don’t share the same media as cable TV does, and there is one cable for each customer.Fiber to the premises (FTTP): FTTP is when an optical cable arrives at a customer’s premises. Like cable TV, the most common implementation is a single fiber that crosses several different customers in a sharable manner. The most common technology is a passive optical network (PON) or, more specifically, the Gigabit Ethernet PON (GPON) (or G.984).

Important note

Further details on GPON networks can be found in the paper GPON in Telecommunication Network – November 2010 – Paper from the International Congress on Ultra Modern Telecommunications and Control Systems (ICUMT) conference, 2010.

Wi-Fi: Normally, this technology is used privately inside a company or a home, but some ISPs use the Wireless Ethernet standards (IEEE 802.11 family) to provide the last mile to customers using omnidirectional antennas. This particular use is different depending on each country and it depends on the government’s legislation. They are normally advertised as Ethernet hotspots (https://en.wikipedia.org/wiki/Hotspot_(Wi-Fi)).Satellite: For data communication using satellites, there are two methods: one using geostationary satellites and the other using constellation satellites. The difference between them is the latency, as geostationary orbits very far from earth. The constellation method has low latency but has handover challenges as the satellites keep moving, normally having very low data throughput. The most famous technology using geostationary is VSAT. Internet using VSAT adds around 250 ms every time it has to travel from earth to the satellite, therefore it is a 500 ms round trip. But the dark ages of high latency might be over as SpaceX has announced they have finally solved the handover problem using the constellation method. This new service is called Starlink and has promised to have high capacity, low latency, and high availability using low orbit satellites.

Important note

A good discussion on the Starlink network can be found in the paper Starlink Analysis – July 15, 2021 – Research group ROADMAP-5G at the Carinthia University of Applied Sciences.

Power line communication (PLC) or HomePlug: PLC, or broadband over power lines (BoPL), uses the power cables to communicate data. This is achieved by modulating high frequencies on the wire. Most transformers won’t be able to pass through the information as they act as a low-frequency cut filter, so it has to be contained within a house or between posts without a transformer. The most common technologies here are the HomePlug AV2 and IEEE 1901-2010 (https://ieeexplore.ieee.org/document/5678772).Mobile: Definitely the most popular network is the mobile last mile. Today, they use 5G technology, but other old networks are still in use, such as 4G (LTE), 3G, and GPRS.

Important note

More information on mobile technologies can be found at Evolution of Mobile Communication Technology towards 5G Networks and Challenges by A. Agarwal, K. Agarwal, S. Agarwal, and G. Misra – American Journal of Electrical and Electronic Engineering, 2019, Vol. 7, No. 2, pp. 34-37.

The physical architecture

The physical architecture is sometimes not necessarily the description of the cables or the fibers that will connect the devices but the infrastructure used by the network as a physical layer defined in the TCP/IP stack. This means we can reuse other foreign networks as a physical layer even though they have their own protocol stacks. Here are some of the possible physical technologies used in the architecture:

Dark fiber: When connecting nodes, the term dark fiber means the nodes that are connected will be using a fiber that does not contain a repeater or underlying infrastructure. In the case of a connection between two nodes using dark fiber, if one node loses power, the other will not receive any light from the fiber. In this scenario, a fiber cut is perceived in both ends immediately, and interfaces go down instantaneously with a fiber cut. Only the packets in the output interface queue are discarded when a failure occurs.Synchronous Transport Module (STM): STM was initially created to multiplex digital phone lines, but later started to be used for data communication. The most common one was STM-1, which was 155 Mbps. Routers used to have an interface that could encapsulates STM frames toward an STM network. The STM network would just switch the frames from one end to the other. A cut in the fiber using this technology might not be perceived quickly enough, causing a huge amount of packet loss. As we will describe later, bidirectional forwarding detection (BFD) needs to be used here to avoid drastic problems.Dense wavelength-division multiplexing (DWDM): DWDM is an evolution of STM. The DWDM network is a switch network that also has a frame and time and wave division for each of the packets of data carried, similar to STM but enhanced. Similarly, BFD is necessary because a cut in the fiber here would not be perceived quickly enough, causing a huge amount of packet loss.Back to back: As explained before, the term back to back is normally used to designate the nodes that are connected directly without any other physical layer in between, such as repeaters.Network tunnels: Network tunnels are points of the network that are used to encapsulate the traffic and travel in a different network. Tunnels can be either Layer 2 or Layer 3 and are implemented to abstract the network that is being carried. In some network architectures, they are meant to reach a distant part of the network using a foreign infrastructure.VPN tunnels: These are like network tunnels. VPN tunnels normally add encryption.

The routing architecture

It’s important to define how the traffic will flow in the network. For that, we need to have a proper design in terms of routing distribution. This is necessary so failure remediation, redundant paths, load balancing, routing policies, and traffic agreements can be implemented. The architecture would have to include an internal routing protocol and an external routing protocol if connected outside. Here is a summary:

Interior gateway protocol (IGP): IGP is a routing protocol that runs in a delimited area or location, normally internally within the same organization, as the name says. In the IGP domain, routers exchange path information by announcing and receiving topology updates. The most common IGPs use link state information to build the routing path topology. If an interface goes down, the update has to be propagated to the entire IGP domain. Isolated areas are used to avoid having to update a too-large topology and cause instability. Historically, the popular IGPs were RIP and EIGRP, but today, only