Mastering Go – Third Edition E-Book

Mihalis Tsoukalos

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 800

Veröffentlichungsjahr: 2021

Beschreibung

Mastering Go is the essential guide to putting Go to work on real production systems. This freshly updated third edition includes topics like creating RESTful servers and clients, understanding Go generics, and developing gRPC servers and clients.

Mastering Go was written for programmers who want to explore the capabilities of Go in practice. As you work your way through the chapters, you’ll gain confidence and a deep understanding of advanced Go concepts, including concurrency and the operation of the Go Garbage Collector, using Go with Docker, writing powerful command-line utilities, working with JavaScript Object Notation (JSON) data, and interacting with databases. You’ll also improve your understanding of Go internals to optimize Go code and use data types and data structures in new and unexpected ways.

This essential Go programming book will also take you through the nuances and idioms of Go with exercises and resources to fully embed your newly acquired knowledge.

With the help of Mastering Go, you’ll become an expert Go programmer by building Go systems and implementing advanced Go techniques in your projects.

Leseprobe

Mastering Go

Third Edition

Harness the power of Go to build professional utilities and concurrent servers and services

Mihalis Tsoukalos

BIRMINGHAM—MUMBAI

Mastering Go

Third Edition

Copyright © 2021 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Producer: Dr. Shailesh Jain

Acquisition Editor – Peer Reviews: Saby Dsilva

Project Editor: Amisha Vathare

Content Development Editor: Edward Doxey

Copy Editor: Safis Editing

Technical Editor: Aniket Shetty

Proofreader: Safis Editing

Indexer: Rekha Nair

Presentation Designer: Pranit Padwal

First published: April 2018

Second edition: August 2019

Third edition: August 2021

Production reference: 1200821

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80107-931-0

www.packt.com

Contributors

About the author

Mihalis Tsoukalos is a UNIX Systems Engineer who enjoys technical writing. He is the author of Go Systems Programming and Mastering Go, both first and second editions. He holds a BSc in Mathematics from the University of Patras and an MSc in IT from University College London, UK. He has written more than 300 technical articles for magazines including Sys Admin, MacTech, Linux User and Developer, Usenix ;login:, Linux Format, and Linux Journal. His research interests include time series, databases, and indexing.

You can reach him at https://www.mtsoukalos.eu/ and @mactsouk.

As writing a book is a team effort, I would like to thank the people at Packt Publishing for helping me write this book. This includes Amit Ramadas for answering all my questions, Shailesh Jain for convincing me to write the third edition of Mastering Go, Edward Doxey for the helpful comments and suggestions, and Derek Parker, the technical reviewer, for his good work.

Last, I would like to thank you, the reader of the book, for choosing this book. I hope you find it helpful.

About the reviewer

Derek Parker is a Software Engineer at Red Hat. He is the creator of the Delve debugger for Go and a contributor to the Go compiler, linker, and standard library. Derek is an open-source contributor and maintainer and has worked on everything from front-end JavaScript to low-level assembly code.

I would like to thank my wife, Erica, and our two incredible children for lending me the time to work on this project.

Contents

Preface

Who this book is for

What this book covers

To get the most out of this book

Get in touch

A Quick Introduction to Go

Introducing Go

The history of Go

Why UNIX and not Windows?

The advantages of Go

The go doc and godoc utilities

Hello World!

Introducing functions

Introducing packages

Running Go code

Compiling Go code

Using Go like a scripting language

Important formatting and coding rules

Important characteristics of Go

Defining and using variables

Printing variables

Controlling program flow

Iterating with for loops and range

Getting user input

Reading from standard input

Working with command-line arguments

Using error variables to differentiate between input types

Understanding the Go concurrency model

Developing the which(1) utility in Go

Logging information

log.Fatal() and log.Panic()

Writing to a custom log file

Printing line numbers in log entries

Overview of Go generics

Developing a basic phone book application

Exercises

Summary

Additional resources

Basic Go Data Types

The error data type

Numeric data types

Non-numeric data types

Strings, Characters, and Runes

Converting from int to string

The unicode package

The strings package

Times and dates

A utility for parsing dates and times

Working with different time zones

Go constants

The constant generator iota

Grouping similar data

Arrays

Slices

About slice length and capacity

Selecting a part of a slice

Byte slices

Deleting an element from a slice

How slices are connected to arrays

The copy() function

Sorting slices

Pointers

Generating random numbers

Generating random strings

Generating secure random numbers

Updating the phone book application

Exercises

Summary

Additional resources

Composite Data Types

Maps

Storing to a nil map

Iterating over maps

Structures

Defining new structures

Using the new keyword

Slices of structures

Regular expressions and pattern matching

About Go regular expressions

Matching names and surnames

Matching integers

Matching the fields of a record

Improving the phone book application

Working with CSV files

Adding an index

The improved version of the phone book application

Exercises

Summary

Additional resources

Reflection and Interfaces

Reflection

Learning the internal structure of a Go structure

Changing structure values using reflection

The three disadvantages of reflection

Type methods

Creating type methods

Using type methods

Interfaces

The sort.Interface interface

The empty interface

Type assertions and type switches

The map[string]interface{} map

The error data type

Writing your own interfaces

Using a Go interface

Implementing sort.Interface for 3D shapes

Working with two different CSV file formats

Object-oriented programming in Go

Updating the phone book application

Setting up the value of the CSV file

Using the sort package

Exercises

Summary

Additional resources

Go Packages and Functions

Go packages

Downloading Go packages

Functions

Anonymous functions

Functions that return multiple values

The return values of a function can be named

Functions that accept other functions as parameters

Functions can return other functions

Variadic functions

The defer keyword

Developing your own packages

The init() function

Order of execution

Using GitHub to store Go packages

A package for working with a database

Getting to know your database

Storing the Go package

The design of the Go package

The implementation of the Go package

Testing the Go package

Modules

Creating better packages

Generating documentation

GitLab Runners and Go

The initial version of the configuration file

The final version of the configuration file

GitHub Actions and Go

Storing secrets in GitHub

The final version of the configuration file

Versioning utilities

Exercises

Summary

Additional resources

Telling a UNIX System What to Do

stdin, stdout, and stderr

UNIX processes

Handling UNIX signals

Handling two signals

File I/O

The io.Reader and io.Writer interfaces

Using and misusing io.Reader and io.Writer

Buffered and unbuffered file I/O

Reading text files

Reading a text file line by line

Reading a text file word by word

Reading a text file character by character

Reading from /dev/random

Reading a specific amount of data from a file

Writing to a file

Working with JSON

Using Marshal() and Unmarshal()

Structures and JSON

Reading and writing JSON data as streams

Pretty printing JSON records

Working with XML

Converting JSON to XML and vice versa

Working with YAML

The viper package

Using command-line flags

Reading JSON configuration files

The cobra package

A utility with three commands

Adding command-line flags

Creating command aliases

Creating subcommands

Finding cycles in a UNIX file system

New to Go 1.16

Embedding files

ReadDir and DirEntry

The io/fs package

Updating the phone book application

Using cobra

Storing and loading JSON data

Implementing the delete command

Implementing the insert command

Implementing the list command

Implementing the search command

Exercises

Summary

Additional resources

Go Concurrency

Processes, threads, and goroutines

The Go scheduler

The GOMAXPROCS environment variable

Concurrency and parallelism

Goroutines

Creating a goroutine

Creating multiple goroutines

Waiting for your goroutines to finish

What if the number of Add() and Done() calls differ?

Creating multiple files with goroutines

Channels

Writing to and reading from a channel

Receiving from a closed channel

Channels as function parameters

Race conditions

The Go race detector

The select keyword

Timing out a goroutine

Timing out a goroutine – inside main()

Timing out a goroutine – outside main()

Go channels revisited

Buffered channels

nil channels

Worker pools

Signal channels

Specifying the order of execution for your goroutines

Shared memory and shared variables

The sync.Mutex type

What happens if you forget to unlock a mutex?

The sync.RWMutex type

The atomic package

Sharing memory using goroutines

Closured variables and the go statement

The context package

Using context as a key/value store

The semaphore package

Exercises

Summary

Additional resources

Building Web Services

The net/http package

The http.Response type

The http.Request type

The http.Transport type

Creating a web server

Updating the phone book application

Defining the API

Implementing the handlers

Exposing metrics to Prometheus

The runtime/metrics package

Exposing metrics

Creating a Docker image for a Go server

Exposing the desired metrics

Reading metrics

Putting the metrics in Prometheus

Visualizing Prometheus metrics in Grafana

Developing web clients

Using http.NewRequest() to improve the client

Creating a client for the phone book service

Creating file servers

Downloading the contents of the phone book application

Timing out HTTP connections

Using SetDeadline()

Setting the timeout period on the client side

Setting the timeout period on the server side

Exercises

Summary

Additional resources

Working with TCP/IP and WebSocket

TCP/IP

The nc(1) command-line utility

The net package

Developing a TCP client

Developing a TCP client with net.Dial()

Developing a TCP client that uses net.DialTCP()

Developing a TCP server

Developing a TCP server with net.Listen()

Developing a TCP server that uses net.ListenTCP()

Developing a UDP client

Developing a UDP server

Developing concurrent TCP servers

Working with UNIX domain sockets

A UNIX domain socket server

A UNIX domain socket client

Creating a WebSocket server

The implementation of the server

Using websocat

Using JavaScript

Creating a WebSocket client

Exercises

Summary

Additional resources

Working with REST APIs

An introduction to REST

Developing RESTful servers and clients

A RESTful server

A RESTful client

Creating a functional RESTful server

The REST API

Using gorilla/mux

The use of subrouters

Working with the database

Testing the restdb package

Implementing the RESTful server

Testing the RESTful server

Testing GET handlers

Testing POST handlers

Testing the PUT handler

Testing the DELETE handler

Creating a RESTful client

Creating the structure of the command-line client

Implementing the RESTful client commands

Using the RESTful client

Working with multiple REST API versions

Uploading and downloading binary files

Using Swagger for REST API documentation

Documenting the REST API

Generating the documentation file

Serving the documentation file

Exercises

Summary

Additional resources

Code Testing and Profiling

Optimizing code

Benchmarking code

Rewriting the main() function for better testing

Benchmarking buffered writing and reading

The benchstat utility

Wrongly defined benchmark functions

Profiling code

Profiling a command-line application

Profiling an HTTP server

The web interface of the Go profiler

The go tool trace utility

Tracing a web server from a client

Visiting all routes of a web server

Testing Go code

Writing tests for ./ch03/intRE.go

The TempDir function

The Cleanup() function

The testing/quick package

Timing out tests

Testing code coverage

Finding unreachable Go code

Testing an HTTP server with a database backend

Fuzzing

Cross-compilation

Using go:generate

Creating example functions

Exercises

Summary

Additional resources

Working with gRPC

Introduction to gRPC

Protocol buffers

Defining an interface definition language file

Developing a gRPC server

Developing a gRPC client

Testing the gRPC server with the client

Exercises

Summary

Additional resources

Go Generics

Introducing generics

Constraints

Creating constraints

Defining new data types with generics

Using generics in Go structures

Interfaces versus generics

Reflection versus generics

Exercises

Summary

Additional resources

Appendix A – Go Garbage Collector

Heap and stack

Garbage collection

The tricolor algorithm

More about the operation of the Go garbage collector

Maps, slices, and the Go garbage collector

Using a slice

Using a map with pointers

Using a map without pointers

Splitting the map

Comparing the performance of the presented techniques

Additional resources

Other Books You May Enjoy

Index

Landmarks

Cover

Index

Preface

The book you are reading right now is Mastering Go, Third Edition, which is all about helping you become a better Go developer! If you have the second edition of Mastering Go, do not throw it away—Go has not changed that much, and the second edition is still useful. However, Mastering Go, Third Edition is better than the second edition in many aspects.

There exist many exciting new topics in this edition, including writing RESTful services, working with the WebSocket protocol, and using GitHub Actions and GitLab Actions for Go projects, as well as an entirely new chapter on generics and the development of lots of practical utilities. Additionally, I tried to make this book smaller than the second edition and structure it in a more natural way to make it both easier and faster to read, especially if you are a busy professional.

I also tried to include the right amount of theory and hands-on content—but, only you, the reader, can tell whether I succeed! Try to do the exercises located at the end of each chapter and do not hesitate to contact me about ways or ideas that can make future editions of this book even better!

Who this book is for

This book is for intermediate Go programmers who want to take their Go knowledge to the next level. It will also be helpful for experienced developers in other programming languages who want to learn Go without going over programming basics.

What this book covers

Chapter 1, A Quick Introduction to Go, begins by talking about the history of Go, the important characteristics of Go, and the advantages of Go, before describing the godoc and go doc utilities and explaining how we can compile and execute Go programs. Afterward, the chapter talks about printing output and getting user input, working with command-line arguments, and using log files. Lastly, we develop a basic version of a phone book application that we are going to keep improving in forthcoming chapters.

Chapter 2, Basic Go Data Types, discusses the basic data types of Go, both numeric and non-numeric, as well as arrays and slices that allow you to group data of the same data type. It also considers Go pointers, constants, and working with dates and times. The last part of the chapter is about generating random numbers and populating the phone book application with random data.

Chapter 3, Composite Data Types, begins with maps, before going into structures and the struct keyword. Additionally, it talks about regular expressions, pattern matching, and working with CSV files. Lastly, we improve the phone book application by adding data persistency to it.

Chapter 4, Reflection and Interfaces, is about reflection, interfaces and type methods, which are functions attached to data types. The chapter also includes the use of the sort.Interface interface for sorting slices, the use of the empty interface, type assertions, type switches, and the error data type. Additionally, we discuss how Go can mimic some object-oriented concepts before improving the phone book application.

Chapter 5, Go Packages and Functions, is all about packages, modules, and functions, which are the main elements of packages. Among other things, we create a Go package for interacting with a PostgreSQL database, create documentation for it, and explain the use of the sometimes tricky defer keyword. This chapter also includes information about the use of GitLab Runners and GitHub Actions for automation and how to create a Docker image for a Go binary.

Chapter 6, Telling a UNIX System What to Do, is about systems programming, which includes subjects such as working with command-line arguments, handling UNIX signals, file input and output, the io.Reader and io.Writer interfaces, and the use of the viper and cobra packages. Additionally, we talk about working with JSON, XML, and YAML files, create a handy command-line utility for discovering cycles in a UNIX filesystem, and discuss embedding files in Go binaries as well as the os.ReadDir() function, the os.DirEntry type, and the io/fs package. Lastly, we update the phone book application to use JSON data and convert it into a proper command-line utility with the help of the cobra package.

Chapter 7, Go Concurrency, discusses goroutines, channels, and pipelines. We learn about the differences between processes, threads, and goroutines, the sync package, and the way the Go scheduler operates. Additionally, we explore the use of the select keyword and we discuss the various "types" of Go channels as well as shared memory, mutexes, the sync.Mutex type, and the sync.RWMutex type. The rest of the chapter talks about the context package, the semaphore package, worker pools, how to time out goroutines, and how to detect race conditions.

Chapter 8, Building Web Services, discusses the net/http package, the development of web servers and web services, exposing metrics to Prometheus, visualizing metrics in Grafana, creating web clients and creating file servers. We also convert the phone book application into a web service and create a command-line client for it.

Chapter 9, Working with TCP/IP and WebSocket, is about the net package, TCP/IP, and the TCP and UDP protocols, as well as UNIX sockets and the WebSocket protocol. We develop lots of network servers and clients in this chapter.

Chapter 10, Working with REST APIs, is all about working with REST APIs and RESTful services. We learn how to define REST APIs and develop powerful concurrent RESTful servers as well as command-line utilities that act as clients to RESTful services. Lastly, we introduce Swagger for creating documentation for REST APIs and learn how to upload and download binary files.

Chapter 11, Code Testing and Profiling, discusses code testing, code optimization, and code profiling as well as cross-compilation, benchmarking Go code, creating example functions, the use of go:generate, and finding unreachable Go code.

Chapter 12, Working with gRPC, is all about working with gRPC in Go. gRPC is an alternative to RESTful services that was developed by Google. This chapter teaches you how to define the methods and the messages of a gRPC service, how to translate them into Go code, and how to develop a server and a client for that gRPC service.

Chapter 13, Go Generics, is about generics and how to use the new syntax to write generic functions and define generic data types. Generics is coming to Go 1.18, which, according to the Go development cycle, is going to be officially released during February 2022.

Appendix A, Go Garbage Collector, talks about the operation of the Go garbage collector and illustrates how this Go component can affect the performance of your code.

To get the most out of this book

This book requires a UNIX computer with a relatively recent Go version installed, which includes any machine running macOS X, macOS, or Linux. Most of the presented code also runs on Microsoft Windows machines without any changes.

To get the most out of this book, you should try to apply the knowledge of each chapter in your own programs as soon as possible and see what works and what does not! As I told you before, try to solve the exercises found at the end of each chapter or create your own programming problems.

Download the example code files

The code bundle for the book is hosted on GitHub at https://github.com/mactsouk/mastering-Go-3rd. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://static.packt-cdn.com/downloads/9781801079310_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, folder names, filenames, file extensions, pathnames, and user input. For example, "The star of this chapter will be the net/http package, which offers functions that allow you to develop powerful web servers and web clients."

A block of code is set as follows:

package main import ( "fmt""math/rand""os""path/filepath""strconv""time" )

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are highlighted:

package main import ( "fmt""math/rand""os""path/filepath""strconv""time" )

Any command-line input or output is written as follows:

$ go run www.go Using default port number: :8001 Served: localhost:8001

Bold: Indicates a new term, an important word, or words that you see on the screen, for example, in menus or dialog boxes. For example: "The UNIX logging service has support for two properties named logging level and logging facility."

Warnings or important notes appear like this.

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: Email [email protected], and mention the book's title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book we would be grateful if you would report this to us. Please visit http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit http://authors.packtpub.com.

Share your thoughts

Once you've read Mastering Go, Third Edition, we'd love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we're delivering excellent quality content.