Mastering Go E-Book

Mihalis Tsoukalos

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 867

Veröffentlichungsjahr: 2024

Beschreibung

Mastering Go, now in its fourth edition, remains the go-to resource for real-world Go development. This comprehensive guide delves into advanced Go concepts, including RESTful servers, and Go memory management. This edition brings new chapters on Go Generics and fuzzy Testing, and an enriched exploration of efficiency and performance. As you work your way through the chapters, you will gain confidence and a deep understanding of advanced Go topics, including concurrency and the operation of the Garbage Collector, using Go with Docker, writing powerful command-line utilities, working with JavaScript Object Notation (JSON) data, and interacting with databases.

You will be engaged in real-world exercises, build network servers, and develop robust command-line utilities. With in-depth chapters on RESTful services, the WebSocket protocol, and Go internals, you are going to master Go's nuances, optimization, and observability. You will also elevate your skills in efficiency, performance, and advanced testing.

With the help of Mastering Go, you will become an expert Go programmer by building Go systems and implementing advanced Go techniques in your projects.

Leseprobe

Mastering Go

Fourth Edition

Leverage Go’s expertise for advanced utilities, empowering you to develop professional software

Mihalis Tsoukalos

BIRMINGHAM—MUMBAI

Mastering Go

Fourth Edition

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Senior Publishing Product Manager: Denim Pinto

Acquisition Editor – Peer Reviews: Swaroop Singh

Project Editor: Meenakshi Vijay

Content Development Editor: Deepayan Bhattacharjee

Copy Editor: Safis Editing

Technical Editor: Aniket Shetty

Proofreader: Safis Editing

Indexer: Manju Arasan

Presentation Designer: Ajay Patule

Developer Relations Marketing Executive: Vipanshu Parashar

First published: April 2018

Second edition: August 2019

Third edition: August 2021

Fourth edition: March 2024

Production reference: 1250324

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK.

ISBN 978-1-80512-714-7

www.packt.com

Contributors

About the author

Mihalis Tsoukalos is a UNIX systems engineer and a technical writer. He is the author of Go Systems Programming, Time Series Indexing, and Mastering Go. He holds a BSc in mathematics from the University of Patras and an MSc in IT from University College London, UK. He has written more than 300 technical articles for magazines including Sys Admin, MacTech, Linux User and Developer, Usenix ;login:, Linux Format, and Linux Journal. His research interests include times series, databases, and indexing.

As authoring a book is a team effort, I would like to thank the people at Packt Publishing, including Denim Pinto, Amit Ramadas, Aniket Shetty, and Meenakshi Vijay, for answering all my questions and helping me write this book. I would also like to thank Ricardo Gerardi and Derek Parker, the technical reviewers of the book, for their great work.

Finally, I would like to thank you, the reader of the book, for choosing this book. I hope you find it helpful and that it will keep you good company in your Go journey.

About the reviewers

Derek Parker is the author of Delve, the Go debugger, and is an active open source maintainer, contributor, speaker, and a longstanding member of the Go community.

I would like to thank my amazing wife, Erica Parker, and my children, for allowing me the time to work on this during the review process.

Ricardo Gerardi is an experienced IT professional with over 25 years of experience in the industry. He has worked in both large and small companies in Brazil and Canada, and he is currently serving as a Principal Consultant at Red Hat. Ricardo has been a Linux and open-source enthusiast and contributor for over 20 years. He is currently interested in IT automation, Kubernetes, and exploring innovative solutions using the Go programming language. He's the author of Powerful Command-Line Applications in Go and Automate Your Home Using Go. Ricardo also writes regularly about Go, Linux, and other technologies for several online community publications.

I would like to thank Mihalis Tsoukalos for writing such a great Go reference, and I would like to thank my daughters for allowing me the time to work on this project.

Join our community on Discord

Join our community’s Discord space for discussions with the authors and other readers:

https://discord.gg/FzuQbc8zd6

Contents

Preface

Who this book is for

What this book covers

To get the most out of this book

Get in touch

A Quick Introduction to Go

Introducing Go

The history of Go

The advantages of Go

When to use Go

My personal Go journey

The go doc and godoc utilities

Hello World!

Introducing functions

Introducing packages

Running Go code

Compiling Go code

Using Go like a scripting language

Important formatting and coding rules

What you should know about Go

Defining and using variables

Constants

Global variables

Printing variables

Controlling program flow

Iterating with for loops and range

Getting user input

Reading from standard input

Working with command line arguments

Using error variables to differentiate between input types

Understanding the Go concurrency model

Developing the which(1) utility in Go

Logging information

log.Fatal() and log.Panic()

Writing to a custom log file

Printing line numbers in log entries

Writing to multiple log files

Developing a statistics application

Summary

Exercises

Additional resources

Basic Go Data Types

The error data type

Numeric data types

Avoiding overflows

Non-numeric data types

Strings, characters, and runes

Converting int to string

The unicode package

The strings package

Times and dates

Working with different time zones

Constants

The constant generator iota

Typed and untyped constants

Grouping similar data

Arrays

Slices

About slice length and capacity

Selecting a part of a slice

Byte slices

Deleting an element from a slice

How slices are connected to arrays

Catching out of bounds errors

The copy() function

Sorting slices

Pointers

Converting a slice to an array or an array pointer

Data types and the unsafe package

Generating random numbers

Generating random strings

Generating secure random numbers

Updating the statistics application

Summary

Exercises

Additional resources

Composite Data Types

Maps

How to tell whether a key exists on a map

Storing to a nil map

Iterating over maps

Structures

The type keyword

Defining new structures

Using the new keyword

Slices of structures

Regular expressions and pattern matching

About regexp.Compile and regexp.MustCompile

Go regular expressions

About raw string and interpreted string literals

Matching names and surnames

Matching integers

Improving the statistics application

Working with CSV files

The updated version of the statistics application

Summary

Exercises

Additional resources

Go Generics

An introduction to generics

Hello, generics!

Constraints

Creating constraints

Supporting underlying data types

Supporting slices of any type

Defining new data types with generics

Using generics in Go structures

The cmp package

The slices package

Shallow and deep copies

The maps package

When to use generics

Summary

Exercises

Additional resources

Reflection and Interfaces

Reflection

Understanding the internal structure of a Go structure

Changing structure values using reflection

The three disadvantages of reflection

Type methods

Creating type methods

Value and point receivers

Using type methods

Interfaces

The sort.Interface interface

The empty interface

Type assertions and type switches

The map[string]interface{} map

The error data type

Writing your own interfaces

Using a Go interface

Object-oriented programming in Go

Interfaces versus generics

Reflection versus generics

Updating the statistics application

Summary

Exercises

Additional resources

Go Packages and Functions

Go packages

About go get and go install

Functions

Anonymous functions

Functions that return multiple values

The return values of a function can be named

Functions that accept other functions as parameters

Functions can return other functions

Variadic functions

The defer keyword

Big O complexity

Developing your own packages

The init() function

Order of execution

Using GitHub to store Go packages

A package for working with SQLite

Working with SQLite3 and Go

Storing the Go package

The design of the Go package

The implementation of the Go package

Testing the Go package

Modules

Creating better packages

Generating documentation

Workspaces

Versioning utilities

Summary

Exercises

Additional resources

Telling a UNIX System What to Do

stdin, stdout, and stderr

UNIX processes

File I/O

The io.Reader and io.Writer interfaces

Using and misusing io.Reader and io.Writer

Buffered and unbuffered file I/O

Reading text files

Reading a text file line by line

Reading a text file word by word

Reading a text file character by character

Reading from /dev/random

Reading a specific amount of data from a file

Writing to a file

Working with JSON

Using Marshal() and Unmarshal()

Structures and JSON

Reading and writing JSON data as streams

Pretty printing JSON records

The viper package

Using command line flags

Reading JSON configuration files

The cobra package

A utility with three commands

Adding command line flags

Creating command aliases

Creating subcommands

Important Go features

Embedding files

ReadDir and DirEntry

The io/fs package

Updating the statistics application

The slog package

Sending logs to io.Discard

Using cobra

Storing and loading JSON data

Implementing the list command

Implementing the insert command

Summary

Exercises

Additional resources

Go Concurrency

Processes, threads, and goroutines

The Go scheduler

The GOMAXPROCS environment variable

Concurrency and parallelism

Goroutines

Creating a goroutine

Creating multiple goroutines

Waiting for all goroutines to finish

What if the number of Add() and Done() calls differ?

Creating multiple files with goroutines

Channels

Writing to and reading from a channel

Receiving from a closed channel

Channels as function parameters

Race conditions are bad

The Go race detector

The select keyword

Timing out a goroutine

Timing out a goroutine inside main()

Timing out a goroutine outside main()

Go channels revisited

Buffered channels

nil channels

Worker pools

Signal channels

Specifying the order of execution for your goroutines

Handling UNIX signals

Handling two signals

Shared memory and shared variables

The sync.Mutex type

What happens if you forget to unlock a mutex?

The sync.RWMutex type

The atomic package

Sharing memory using goroutines

Closured variables and the go statement

The context package

About context.WithCancelCause

The semaphore package

Making the statistics application concurrent

Summary

Exercises

Additional resources

Building Web Services

The net/http package

The http.Response type

The http.Request type

The http.Transport type

Creating a web server

Updating the statistics application

Defining the API

Implementing the handlers

Creating a Docker image

Developing web clients

Using http.NewRequest() to improve the client

Using errGroup

Creating a client for the statistics service

Timing out HTTP connections

Using SetDeadline()

Setting the timeout period on the client side

Setting the timeout period on the server side

Summary

Exercises

Additional resources

Working with TCP/IP and WebSocket

TCP/IP

The nc(1) command line utility

The net package

Developing a TCP client

Developing a TCP client with net.Dial()

Developing a TCP client that uses net.DialTCP()

Developing a TCP server

Developing a TCP server with net.Listen()

Developing a TCP server that uses net.ListenTCP()

Developing a UDP client

Developing a UDP server

Developing concurrent TCP servers

Creating a WebSocket server

The implementation of the server

Using websocat

Creating a WebSocket client

Working with RabbitMQ

Running RabbitMQ

Writing to RabbitMQ

Reading from RabbitMQ

How to remove a module

Summary

Exercises

Additional resources

Working with REST APIs

An introduction to REST

Developing RESTful servers and clients

A RESTful server

A RESTful client

Creating a functional RESTful server

The REST API

Using gorilla/mux

The use of subrouters

The Gin HTTP framework

Gin versus Gorilla

Working with the database

Implementing the RESTful server

Testing the RESTful server

Testing GET handlers

Testing POST handlers

Testing the PUT handler

Testing the DELETE handler

Creating a RESTful client

Creating the structure of the command line client

Implementing the RESTful client commands

Using the RESTful client

Working with multiple REST API versions

Summary

Exercises

Additional resources

Code Testing and Profiling

Optimizing code

Rewriting the main() function for better testing

Profiling code

Profiling a command line application

Profiling an HTTP server

The web interface of the Go profiler

The go tool trace utility

Tracing a web server from a client

Visiting all routes of a web server

Testing Go code

Writing tests for ./ch03/intRE.go

Testing UNIX signals

Disabling test caching

The testing.TempDir() function

The Cleanup() function

The testing/quick package

Timing out tests

Testing using testing.T.Fatal() and testing.T.Fatalf()

Table-driven tests

Testing code coverage

Finding unreachable Go code

Testing an HTTP server with a database backend

The govulncheck tool

Installing the tool

Cross-compilation

Using go:generate

Creating example functions

Summary

Exercises

Additional resources

Fuzz Testing and Observability

Fuzz testing

A simple fuzz testing example

An advanced fuzz testing example

Correcting the bug

Observability

The runtime/metrics package

Measuring the execution time of a function

The expvar package

Learning about CPU characteristics

Exposing metrics to Prometheus

Exposing metrics

Creating a Docker image for a Go server

Specifying the metrics to expose

Getting the metrics

Putting the metrics in Prometheus

Visualizing Prometheus metrics in Grafana

Summary

Exercises

Additional resources

Efficiency and Performance

Benchmarking code

A simple benchmark scenario

Benchmarking the number of memory allocations

Initial version

Improving the number of memory allocations

Buffered versus unbuffered file I/O

The benchstat utility

Wrongly defined benchmark functions

Go memory management

Heap and stack

The main elements of the Go memory model

Memory leaks

Slices and memory leaks

Maps and memory leaks

Memory pre-allocation

Working with eBPF

What is eBPF?

About observability and eBPF

Creating an eBPF tool in Go

Summary

Exercises

Additional resources

Changes in Recent Go Versions

About rand.Seed()

What is new in Go 1.21?

The sync.OnceFunc() function

The clear function

What is new in Go 1.22?

Changes in slices

Changes in for loops

The math/rand/v2 package

Summary

Exercises

Additional resources

Appendix: The Go Garbage Collector

Garbage collection

The key characteristics of the Go GC

Learning more about the Go GC

The tri-color algorithm

More about the operation of the Go GC

Maps, slices, and the Go GC

Using slices

Using maps with pointers

Using maps without pointers

Splitting a map

Comparing the performance of the presented techniques

Additional resources

Other Books You May Enjoy

Index

Landmarks

Cover

Index

Preface

Welcome to the fourth edition of Mastering Go! As the Go programming language continues to evolve and gain popularity, I am delighted to present an updated edition of this book.

In the rapidly changing landscape of technology, Go has emerged as a language of choice for building scalable, performant, and maintainable software. Whether you are a seasoned Go developer looking to deepen your expertise or a newcomer eager to master the intricacies of the language, this book is your comprehensive guide.

If you have an older edition of Mastering Go, apart from the first edition, which is now pretty old, do not throw it away just because Mastering Go, Fourth Edition, is out. Go has not changed that much and both the second and third editions can still be useful and relevant. However, Mastering Go, Fourth Edition, is better in many aspects than all previous editions and includes information about the latest Go versions, which you are not going to find in the previous editions of the book.

There exist many exciting topics in this latest edition, including writing RESTful services, writing a statistics application, and working with eBPF, as well as an entirely new chapter on fuzz testing and observability.

I tried to include the right amount of theory and hands-on content—but only you, the reader, can tell if I have succeeded or not. Try to do the exercises located at the end of each chapter and do not hesitate to contact me about ways or ideas to make future editions of this book even better.

Thank you for choosing to pick up Mastering Go, Fourth Edition. Let us dive in and unlock the full potential of Go together. Happy coding!

Who this book is for

This book is for amateur and intermediate Go programmers who want to take their Go knowledge to the next level, as well as developers in other programming languages who want to learn Go.

If this is your first programming book ever, you might have some issues following it and a second reading might be required to fully absorb all the presented knowledge.

Learning by doing is a fundamental principle of studying any programming language. Throughout the book, you will find practical examples and hands-on exercises that illustrate key concepts and encourage you to apply your knowledge to real-world scenarios.

One way or another, prepare to work and learn and fail and then work and learn and fail some more. After all, life is not that different from programming.

What this book covers

Chapter 1, A Quick Introduction to Go, begins by discussing the history of Go, important characteristics of Go, and the advantages of Go, before describing the godoc and go doc utilities and explaining how we can compile and execute Go programs. Then, the chapter discusses about controlling program flow, printing output and getting user input, working with command line arguments, and using log files. In the last part of Chapter 1, we develop a basic version of a statistics application that we are going to keep improving in forthcoming chapters.

Chapter 2, Basic Go Data Types, discusses the basic data types of Go, both numeric and non-numeric; arrays and slices that allow you to group data of the same date; Go pointers; constants; and working with dates and times. The last part of the chapter is about generating random numbers and populating the statistics application with random data.

Chapter 3, Composite Data Types, begins by teaching you about maps, before going into structures and the struct keyword. Additionally, it talks about regular expressions, pattern matching, and working with CSV files. Last, it improves the statistics application by adding data persistency to it.

Chapter 4, Go Generics, is about Generics and how to use the new syntax to write generic functions and define generic data types. This chapter also presents the cmp package, the slices package, and the maps package, which are all implemented using generics to work with as many data types as possible.

Chapter 5, Reflection and Interfaces, is about reflection, interfaces, and type methods, which are functions attached to data types. The chapter also covers the use of the sort.Interface interface for sorting slices, the use of the empty interface, type assertions, type switches, and the error data type. Additionally, we discuss how Go can mimic some object-oriented concepts before improving the statistics application. This chapter also compares generics with interfaces and reflection.

Chapter 6, Go Packages and Functions, is all about packages, modules, and functions, which are the main elements of packages. Among other things, we create a Go package for interacting with a SQLite3 database, create documentation for it, and explain the use of the sometimes-tricky defer keyword. Last, we talk about Workspaces, which is a relatively new Go feature.

Chapter 7, Telling a UNIX System What to Do, is about Systems Programming, which includes subjects such as working with command line arguments, handling UNIX signals, file input and output, the io.Reader and io.Writer interfaces, and the use of the viper and cobra packages. Last, we update the statistics application to use JSON data and convert it into a proper command line utility with the help of the cobra package.

Chapter 8, Go Concurrency, discusses goroutines, channels, and pipelines. We learn about the differences between processes, threads, and goroutines, the sync package, and the way the Go scheduler operates. Additionally, we explore the use of the select keyword and we discuss the various types of Go channels as well as shared memory, mutexes, the sync.Mutex type, and the sync.RWMutex type. The rest of the chapter talks about the context package, the semaphore package, worker pools, how to time out goroutines, and how to detect race conditions.

Chapter 9, Building Web Services, discusses the net/http package, the development of web servers and web services, the creation of web clients, and the timing out of HTTP connections. We also convert the statistics application into a web service and create a command line client for it.

Chapter 10, Working with TCP/IP and WebSocket, is about the net package, TCP/IP, the TCP and UDP protocols, as well as the WebSocket protocol and working with RabbitMQ. We develop lots of practical servers and clients in this chapter.

Chapter 11, Working with REST APIs, is all about working with REST APIs and RESTful services. We learn how to define REST APIs and develop powerful concurrent RESTful servers as well as command line utilities that act as clients to RESTful services.

Chapter 12, Code Testing and Profiling, discusses code testing, code optimization, and code profiling, as well as cross compilation, creating example functions, the use of go:generate, and finding unreachable Go code.

Chapter 13, Fuzz Testing and Observability, talks about fuzz testing, which is a relatively new addition to the Go programming language, and about observability, which refers to the ability to understand, measure, and analyze the internal state and behavior of a system based on its external outputs or observable signals.

Chapter 14, Efficiency and Performance, is about benchmarking Go code, understanding the Go memory model, and eliminating memory leaks. The chapter also includes the development of an eBPF utility—eBPF has become a foundational technology for improving observability, security, and performance in modern Linux systems.

Chapter 15, Changes in Recent Go Versions, is about the language changes, additions, and improvements in the latest Go versions and it will help you understand how Go is evolving over time.

Appendix, The Go Garbage Collector, talks about the operation of the Go Garbage Collector and illustrates how this Go component can affect the performance of your code.

To get the most out of this book

This book requires a modern computer with a relatively recent Go version installed, which includes any machine running Mac OS X, macOS, or Linux, as well as familiarity with your operating system, its filesystem, and git(1). Most of the presented code also runs on Microsoft Windows machines without any changes.

As you embark on your journey to mastering Go, I encourage you to experiment, ask questions, and engage with the material actively. The Go programming language offers a refreshing blend of simplicity and power, and I am confident that this book will provide you with the knowledge and skills needed to become a proficient Go developer.

Download the example code files

The code bundle for the book is hosted on GitHub at https://github.com/mactsouk/mGo4th. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://packt.link/gbp/9781805127147.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. For example: “Mount the downloaded WebStorm-10*.dmg disk image file as another disk in your system”.

A block of code is set as follows:

package main import"fmt"funcdoubleSquare(x int) (int, int) { return x * 2, x * x }

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

fmt.Println("Double of", n, "is", d) fmt.Println("Square of", n, "is", s) anF := func(param int)int {return param * param }

Any command line input or output is written as follows:

$ go run namedReturn.go 1 -2 -2 1 -2 1

Bold: Indicates a new term, an important word, or important information. For instance, words in menus or dialog boxes appear in the text like this. For example: “The wordByWord() function uses regular expressions to separate the words found in each line of the input file”.

Warnings or important notes appear like this.

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: Email [email protected] and mention the book’s title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you reported this to us. Please visit http://www.packtpub.com/submit-errata, click Submit Errata, and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit http://authors.packtpub.com.

Share your thoughts

Once you’ve read Mastering Go, Fourth Edition, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application. 

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781805127147

Submit your proof of purchaseThat’s it! We’ll send your free PDF and other benefits to your email directly