Mastering Julia E-Book

Malcolm Sherrington

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 595

Veröffentlichungsjahr: 2024

Beschreibung

Julia is a well-constructed programming language which was designed for fast execution speed by using just-in-time LLVM compilation techniques, thus eliminating the classic problem of performing analysis in one language and translating it for performance in a second.
This book is a primer on Julia’s approach to a wide variety of topics such as scientific computing, statistics, machine learning, simulation, graphics, and distributed computing.
Starting off with a refresher on installing and running Julia on different platforms, you’ll quickly get to grips with the core concepts and delve into a discussion on how to use Julia with various code editors and interactive development environments (IDEs).
As you progress, you’ll see how data works through simple statistics and analytics and discover Julia's speed, its real strength, which makes it particularly useful in highly intensive computing tasks. You’ll also and observe how Julia can cooperate with external processes to enhance graphics and data visualization. Finally, you will explore metaprogramming and learn how it adds great power to the language and establish networking and distributed computing with Julia.
By the end of this book, you’ll be confident in using Julia as part of your existing skill set.

Leseprobe

Mastering Julia

Enhance your analytical and programming skills for data modeling and processing with Julia

Malcolm Sherrington

Mastering Julia

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Kunal Sawant

Senior Editor: Kinnari Chohan

Technical Editor: Rajdeep Chakraborty

Copy Editor: Safis Editing

Project Coordinator: Manisha Singh

Indexer: Rekha Nair

Production Designer: Joshua Misquitta

Marketing DevRel Coordinator: Sonia Chauhan

First published: July 2015

Second edition: January 2024

Production reference: 2180124

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK

ISBN 978-1-80512-979-0

www.packtpub.com

I would like to dedicate this book to the memory of my late wife, Hazel Sherrington, without whose encouragement and support, my involvement in Julia would not have started but who is no longer here to see the culmination of her vision.

Contributors

About the author

Malcolm Sherrington has been working in computing for over 35 years. He holds degrees in mathematics, chemistry, and engineering and has given lectures at two different universities in the UK as well as worked in the aerospace and healthcare industries. Currently, he is running his own company in the finance sector, with specific interests in High Performance Computing and applications of GPUs and parallelism.

Always hands-on, Malcolm started programming scientific problems in Fortran and C, progressing through Ada and Common Lisp, and recently became involved with data processing and analytics in Perl, Python, and R.

Malcolm is the organizer of the London Julia User Group. In addition, he is a co-organizer of the UK High Performance Computing and the financial engineers and Quant London meetup groups.

About the reviewer

Mattia Nicolò Careddu is a software engineer from Milan, Italy. He graduated in Stastistics and Big Data. He is Senior Software Engineer at Conio inc, previously Lead AI Engineer at nCore HR. He is passioned about Technology, Bitcoin, Artificial Intelligence and Startups. He is an open source contributor and creator of Wasabi.jl, an ORM for Julia.

Table of Contents

Preface

1

The Julia Environment

Julia 101

Overview of Julia

Philosophy

Not only, but also…

What is data science?

Comparison with other languages

Why is Julia fast?

Why use Julia?

Julia is easy to learn

Julia is written (mostly) in Julia

Julia can interface with other languages

Julia has a novel type system

Julia has genuine runtime macros

Getting started with Julia

A first Julia script

Editors and IDEs

A quick look at some (more) Julia

Package management

Listing, adding, and removing packages

Testing a package

Choosing and exploring packages

Machine learning

Final thoughts

Summary

2

Developing in Julia

Technical requirements

Integers, bits, bytes, and Booleans

Integers

Primitive types

Logical and arithmetic operators

Booleans

Big integers

Arrays

Broadcasting and list comprehensions

Computing recursive functions

Implicit and explicit typing

Simple matrix operations

Characters and strings

Characters

Strings

Byte array literals

Complex and rational numbers

Complex numbers

Rationals

A little light relief

The Sieve of Eratosthenes

Bulls and cows

Julia sets

Multi-dimensional arrays

Sparse matrices

Sparse vectors

Sparse diagonal matrices

Data arrays and data frames

Dictionaries, sets, stacks, and queues

Dictionaries

Sets

Stacks and queues

Summary

3

The Julia Type System

More about functions

The do syntax

First-class objects

Closures and currying

Passing arguments

Default and optional arguments

Variable argument list

Keyword arguments

Scope

The Queens problem

Conversion between numbers and strings

Derived and composite types

A look at the Rational type

A composite Vehicle data type

Modularization

typealias and unions

Enumerations

Multidimensional vectors and computing pi (revisited)

Parameterization

Higher dimensional vectors

Summary

4

The Three Ms

Multiple dispatch

Code generation

Metaprogramming

Symbols and expressions

Manipulating the code tree

Macros

Timing macros

Macro hygiene

Macro expansions

MacroTools

Macro reductions

Lazy evaluation

Generated functions

Functions or macros?

Modularity

Loading a module

Modular integers

Methods

Testing

Ordered pairs

Summary

5

Interoperability

C and Fortran

Mapping C types

Calling Fortran routines

Basel and Horner functions in C

C++

Python, R, and Java

Python

Going the other way

Packages with Python wrappings

The R (language)

Java

Working with the OS and pipelines

Running commands

Text processing and pipes

Finding large files

Perl one-liners

A couple of examples

Using process I/O channels

Interfacing with other languages

Perl 6

Ruby

Python

Other languages supported by the JuliaInterop group

Working with the filesystem

Summary

6

Working with Data

Basic I/O

Terminal I/O

Terminal output

Terminal input

Text files

Text processing

Binary files

Structured datasets

CSV and other delimited (DLM) files

HDF5 and JLD files

Julia data format (JLD)

XML files

Time series

DataFrames and statistics

DataFrames

Some simple statistics

Kernel densities

Testing hypothesis

Summary

7

Scientific Programming

Linear algebra

Matrix decomposition

Simultaneous equations

Eigenvalues and eigenvectors

High-order algebraic equations

Signal processing

Frequency analysis

Image convolutions

DEs

ODEs

Simulating a (real) pendulum

Catastrophic equations

A touch of chaos

Stochastic DEs

Calculus

Differentiation

Automatic differentiation

Quadratures

Optimization

JuMP

Stochastic simulations

SimJulia

Summary

8

Visualization

Textual visualization

Simple inline displays

Luxor

Turtle graphics

PGFPlots

Basic graphic packages

PyPlot and PythonPlot

Winston

Gadfly

The Plots API

Creating multiple plots using layouts

Recipes

Backends

Visualization frameworks

Plotly/PlotlyJS

StatsPlots

Makie

Basic image processing

The Images(.jl) family

Summary

9

Database Access

Database preliminaries

Interfacing to databases

Relational databases

Building and loading

Interfacing with a database

SQLite

MySQL

PostgreSQL

JDBC databases

NoSQL databases

KV datastores

Document databases

Interfacing with REST

JSON/BSON formats

Web databases

(The) Queryverse

Querying the stocks dataset

LINQ queries

Vega-Lite

Summary

10

Networks and Multitasking

Sockets and servers

Well-known ports

UDP and TCP sockets

A “looking-glass world” echo server

Working with the web

HTTP methods

Utility functions

TCP servers

Routing

Mux

Web crawlers

Genie

Tasks and remote procedures

Tasks

Remote procedures

Needles and PI(ns)

Distributed arrays and Map-Reduce

Running on multiple machines

Distributed data sources

Summary

11

Julia’s Back Pages

Configuring Julia and the OS

Getting Julia sources

The .julia subdirectory

Julia environments

Startup configuration(s)

Standalone Julia

Scripting

System images

Development tools

Document strings

Performance tips

Debugging

Revision

Profiling

Creating packages

A “funky” module

Creating the layout

Collaborating with Git

Quo Vadis, Julia?

Summary

Index

Other Books You May Enjoy

Preface

The previous incarnation of this book was written when Julia was at v0.2, which scurried up to v0.4 by the time it went to press. Now I have been persuaded that a second version is overdue, and similarly, it was at v1.8.2 when I began and is now v1.9.4 but with a v1.10.x out as a release candidate and even v1.11 in development.

Why so long between editions? Julia in the past has not been reluctant to modify the language and seemingly continues to do so at the present time. There are many pitfalls in ignoring backward compatibility when developing a computing language. An example that probably many readers will be aware of is when Python tried to slither up from version 2 to version 3, a sizeable minority of the users had to be dragged screaming to make that leap and it was only when support for version 2 was effectively withdrawn that this happened.

More alarming was when Perl, which once held the position that Python holds now (circa. 2023) moving from v5 to v6 proved so impossible that Perl 6 was renamed (a couple of times) as a new language, one which is rarely used, as is also Perl 5.

So given the gap between the two versions of the book, it is to be hoped that Julia, now well past v1.0, which was announced at JuliaCon 2018 when it was hosted here in London, might at least have now settled down and done without the dreaded depreciation warnings which eventually turn into errors. Moreover, since many Julia packages are written in 100% native Julia code, the effects of changes made in Julia are widespread and packages need to be promptly maintained or the packages should be retired.

Turning to the content of the book, its philosophy remains the same as earlier. So, it will not start by discussing how to print “Hello World” nor how to compute the result of 1 + 1, although there is a version of the former, a metaphorical wolf in sheep’s clothing, so it can be found!

Again, I’m not discussing the usual programming constructs for looping and branching, but rather highlighting some Julia constructs that may not be familiar to all, such as list comprehensions, broadcasting, etc., and of course, dealing with topics such as the Julia type system. The examples are not overly sophisticated but hopefully detailed enough that they will be interesting and importantly runnable for some time in the future.

The chapter count now has risen from 10 to 11, this is due the some of the material in the Dispatch and types chapter being split out into separate chapters, the second one being enigmatically entitled: 3M’s, viz., referring to multiple dispatch, macros and modules, all of which will have been met already in some to the earlier material in the book. This means that now the first five chapters are mainly aimed at Julia’s programming and the next five at themed topics, all promoted by an extra number, are akin to those previously and with the material in these being brought up to date. The final chapter is somewhat different, another with an enigmatic name?!

What this book covers

Chapter 1: The Julia Environment is intended as a gentle introduction to Julia which covers the steps needed to get a working distribution up and running and the steps needed to acquire packages and run Julia in the REPL, code editors (in particular) VS-Code and the Jupyter and Pluto IDEs, continuing then by illustrating these with some simple examples.

Chapter 2: Developing in Julia is an overview of some of Julia’s basic syntax, discussing how to work with simple numeric and character variables either as scalars or in array aggregates, along with briefly introducing the concept of data frames, which will be met often in the remainder of the book.

Chapter 3: The Julia Type System describes first the use of further features such as the specification of parametric and optional parameters as arguments to functions and then continues by introducing the idea of abstract and concrete data types as a precursor to working with and defining more complex composite data structures. It concludes by considering unions and aliases in Julia and higher dimensional arrays.

Chapter 4: The Three M’s covers the topics of three aspects of Julia which may be a little more unfamiliar to users of a different programming language, namely the concept (and use of) multiple dispatch, the macro system, and Julia’s approach to modularity, which is somewhat different to conventional object-oriented systems.

Chapter 5: Interoperability is a little different as it is concerned with Julia working together with other languages either directly with C, Python, and R (via the shared libraries or APIs) or indirectly by invoking the aid of the operating system, setting up pipelines and redirecting basic input and output.

Chapter 6: Working with Data is the first of the themed chapters, concerned primarily with handling textural and binary data files, and structured datasets, and meeting again in much more detail Julia’s sophisticated structures, i.e., data frames and tables. The chapter concludes by applying some simple statistical analysis to some of the datasets referenced.

Chapter 7: Scientific Programming may have originally been viewed as the jewel in Julia’s crown, the raison d’etre of its purpose. The development of the language encompassed many other aspects but recently its focus on machine learning and system simulations may be heralding a move back in this direction. It covers a wide variety of topics including linear algebra, signal processing, solution of differential equations, optimization, and stochastics, albeit briefly.

Chapter 8: Visualization provides a little light relief turning to the production of graphics and other forms of imagery. Originally Julia was criticized, unfairly, as being deficient in this area but now has an embarrassment of riches. The chapter covers simple (raster) video displays and packages for creating vector graphics & hardcopy output such as PDFs and then now to the comprehensive Plots API. It also touches on the powerful image processing packages and concludes with an overview of some Julia mega graphic frameworks.

Chapter 9: Database Access, as might be inferred from the name, deals with Julia’s methods for interaction with databases. The division between relational (SQL-style) databases and others, often lumped together and NoSQL is described and various examples of each are explored, both disk-based and in-memory types. It continues by describing the use of REST-style web database APIS and finishes with a discussion of the use of the Queryverse and its interaction with VegaLite graphics.

Chapter 10: Networks and Multi-tasking focuses on the methods in Julia for multitasking and working with distributed systems. It introduces the concept of sockets and creates networked services that can be used when working with the web. Then the methods by which parallel processes can be defined and run in Julia are presented, finally, it concludes with a discussion of analyzing distributed datasets.

Chapter 11: Julia’s Back Pages are a little different dealing with topics that should be understood to progress from causal user to serious developer. It covers running scripts without the REPL (or an equivalent IDE) and then onto the major topic of creating Julia packages, by means of techniques such as code profiling and use of the debugger. Finally poses the question of where next for Julia, however, does not attempt to answer it!

Who this book is for

The book is targeted at programmers with some familiarity with Julia or else fluency in another programming language, such as C/C++, Python, or R

It assumes some familiarity with the use of code editors, a popular one currently being Visual Studio Code, and also web-based IDEs such as Jupyter.

It covers a wide range of topics in some chapters where knowledge of working with the underlying operating system(s), developing in other programming languages, and installing and running databases will be beneficial.

To get the most out of this book

The software and OS requirements to get the most out of this book are:

Software/hardware covered in the book

Operating system requirements

Julia 1.X

Windows, macOS, or Linux

Downloading the code accompanying the book.

Packt Publishing distributes the sources on their GitHub website, and for this book, the sources can be found at https://github.com/PacktPublishing/Mastering-Julia-Second-Edition

These can be viewed using a web browser and can be downloaded using the git clone <git-url> command which will create a copy of the sources named Mastering-Julia-Second Edition.

As this is somewhat unwieldy, I have renamed this as MJ2 on my computer(s), and this is used in the text throughout the book and in the Julia scripts.

To clone the sources may require using the built-in git command, although it is possible to use an editor such as Visual Studio Code of a GIT IDE. Whether git is available from the command line depends on which operating system is being used.

macOS doesn’t have a built-in version of Gi; however, it’s included as part of Xcode or else installed via using homebrew or by choosing an IDE such as GitHub Desktop.Most Linux distros have the git command already built in.For Windows binaries can be downloaded from the main git website using https://git-scm.com/download; in fact, this page includes sources for all three operating systems plus links to several popular git clients.

The source accompanying this book is arranged as a set of code folders, one per chapter denoted from Code01 through to Code11, containing all the Julia source code relating to the chapter. For example, the Code01 folder contains the relevant script named as: Chp02.jl. There may be additional sources and in particular those corresponding to Jupyter and Pluto notebooks. The sources in each folder are described in the markdown file(s) README.md or the equivalent README.txt file.

Additionally, there is a Project.TOML and a Manifest.TOML file for each chapter plus a script (setup.jl) which can be used to create these from scratch rather than using the ones provided.

These can be utilized by switching to the specific project via the package manager via

julia> import Pkg; Pkg.activate(".")

Possibly when first cloned it may be sensible to also include: Pkg.instantiate()

It will be found to be necessary to work with individual chapter projects rather than adding all Julia modules to the general registry, which otherwise will get extremely clogged, with probable conflicts arising. However, in addition to building up manifests, the reader may find setup.jl useful in getting started with some of the earlier code samples in the book, rather than just adding them individually in the REPL

Additionally, there are two other directories: Alice and DataSources

Alice has a set of files, from the Alice books and others of Lewis Carroll’s works, which are used in some text processing routines.DataSources is split into subfolders containing datasets, such as those in CSV and TSV formats, and files necessary for the sections working with SQL-based and other databases.

As an example ENV["HOME"]*"/MJ2/DataSources" is used as a reference to the location of the data sources directory, since ENV["HOME"] is an alias to my user home directory and I have added the GitHub sources as a folder directly under it.

Note that on Windows this alias is now ENV["HOMEPATH"]. So the construct joinpath (ENV["HOMEPATH"],"MJ2","DataSources") may be preferred as it does not depend on the directory separator, though in my experience either “\\” or “/” can be used in Julia when on Windows.

Share your thoughts

Once you’ve read Mastering Julia, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781805129790

2. Submit your proof of purchase

3. That’s it! We’ll send your free PDF and other benefits to your email directly