40,81 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Understand the fundamentals of algorithmic trading to apply algorithms to real market data and analyze the results of real-world trading strategies
Key Features
- Understand the power of algorithmic trading in financial markets with real-world examples
- Get up and running with the algorithms used to carry out algorithmic trading
- Learn to build your own algorithmic trading robots which require no human intervention
Book Description
It's now harder than ever to get a significant edge over competitors in terms of speed and efficiency when it comes to algorithmic trading. Relying on sophisticated trading signals, predictive models and strategies can make all the difference. This book will guide you through these aspects, giving you insights into how modern electronic trading markets and participants operate.
You'll start with an introduction to algorithmic trading, along with setting up the environment required to perform the tasks in the book. You'll explore the key components of an algorithmic trading business and aspects you'll need to take into account before starting an automated trading project. Next, you'll focus on designing, building and operating the components required for developing a practical and profitable algorithmic trading business. Later, you'll learn how quantitative trading signals and strategies are developed, and also implement and analyze sophisticated trading strategies such as volatility strategies, economic release strategies, and statistical arbitrage. Finally, you'll create a trading bot from scratch using the algorithms built in the previous sections.
By the end of this book, you'll be well-versed with electronic trading markets and have learned to implement, evaluate and safely operate algorithmic trading strategies in live markets.
What you will learn
- Understand the components of modern algorithmic trading systems and strategies
- Apply machine learning in algorithmic trading signals and strategies using Python
- Build, visualize and analyze trading strategies based on mean reversion, trend, economic releases and more
- Quantify and build a risk management system for Python trading strategies
- Build a backtester to run simulated trading strategies for improving the performance of your trading bot
- Deploy and incorporate trading strategies in the live market to maintain and improve profitability
Who this book is for
This book is for software engineers, financial traders, data analysts, and entrepreneurs. Anyone who wants to get started with algorithmic trading and understand how it works; and learn the components of a trading system, protocols and algorithms required for black box and gray box trading, and techniques for building a completely automated and profitable trading business will also find this book useful.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 443
Veröffentlichungsjahr: 2019
Ähnliche
Learn Algorithmic Trading
Copyright © 2019 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor: Sunith ShettyAcquisition Editor: Ali AbidiContent Development Editor: Nazia ShaikhSenior Editor: Sheetal RaneTechnical Editor: Dinesh ChaudharyCopy Editor: Safis EditingProject Coordinator: Kirti PisatProofreader: Safis EditingIndexer: Pratik ShirodkarProduction Designer: Joshua Misquitta
First published: November 2019
Production reference: 1071119
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78934-834-7
www.packt.com
Packt.com
Subscribe to our online digital library for full access to over 7,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Fully searchable for easy access to vital information
Copy and paste, print, and bookmark content
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and, as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the authors
Sebastien Donadio is the Chief Technology Officer at Tradair, responsible for leading the technology. He has a wide variety of professional experience, including being head of software engineering at HC Technologies, partner and technical director of a high-frequency FX firm, a quantitative trading strategy software developer at Sun Trading, working as project lead for the Department of Defense. He also has research experience with Bull SAS, and an IT Credit Risk Manager with Société Générale while in France. He has taught various computer science courses for the past ten years in the University of Chicago, NYU and Columbia University. His main passion is technology but he is also a scuba diving instructor and an experienced rock-climber.
Sourav Ghosh has worked in several proprietary high-frequency algorithmic trading firms over the last decade. He has built and deployed extremely low latency, high throughput automated trading systems for trading exchanges around the world, across multiple asset classes. He specializes in statistical arbitrage market-making, and pairs trading strategies for the most liquid global futures contracts. He works as a Senior Quantitative Developer at a trading firm in Chicago. He holds a Masters in Computer Science from the University of Southern California. His areas of interest include Computer Architecture, FinTech, Probability Theory and Stochastic Processes, Statistical Learning and Inference Methods, and Natural Language Processing.
About the reviewers
Nataraj Dasgupta is the VP of Advanced Analytics at RxDataScience Inc. He has been in the IT industry for more than 19 years and has worked in the technical & analytics divisions of Philip Morris, IBM, UBS Investment Bank, and Purdue Pharma. He led the Data Science team at Purdue, where he developed the company's award-winning Big Data and Machine Learning platform. Prior to Purdue, at UBS, he held the role of Associate Director, working with high-frequency & algorithmic trading technologies in the Foreign Exchange Trading group. He has authored Practical Big Data Analytics and co-authored Hands-on Data Science with R. Apart from his role at RxDataScience, and is also currently affiliated with Imperial College, London.
Ratanlal Mahanta is currently working as a quantitative analyst at bittQsrv, a global quantitative research company offering quant models for its investors. He has several years of experience in the modeling and simulation of quantitative trading. Ratanlal holds a master's degree in science in computational finance, and his research areas include quant trading, optimal execution, and high-frequency trading. He has over 9 years' work experience in the finance industry, and is gifted at solving difficult problems that lie at the intersection of the market, technology, research, and design.
Jiri Pik is an artificial intelligence architect & strategist who works with major investment banks, hedge funds, and other players. He has architected and delivered breakthrough trading, portfolio, and risk management systems, as well as decision support systems, across numerous industries.
Jiri's consulting firm, Jiri Pik—RocketEdge, provides its clients with certified expertise, judgment, and execution at the speed of light.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
Learn Algorithmic Trading
About Packt
Why subscribe?
Contributors
About the authors
About the reviewers
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
Section 1: Introduction and Environment Setup
Algorithmic Trading Fundamentals
Why are we trading?
Basic concepts regarding the modern trading setup
Market sectors
Asset classes
Basics of what a modern trading exchange looks like
Understanding algorithmic trading concepts
Exchange order book
Exchange matching algorithm
FIFO matching
Pro-rata matching
Limit order book
Exchange market data protocols
Market data feed handlers
Order types
IOC – Immediate Or Cancel
GTD – Good Till Day
Stop orders
Exchange order entry protocols
Order entry gateway
Positions and profit and loss (PnL) management
From intuition to algorithmic trading
Why do we need to automate trading?
Evolution of algorithmic trading – from rule-based to AI
Components of an algorithmic trading system
Market data subscription
Limit order books
Signals
Signal aggregators
Execution logic
Position and PnL management
Risk management
Backtesting
Why Python?
Choice of IDE – Pycharm or Notebook
Our first algorithmic trading (buy when the price is low, and sell when the price is high)
Setting up your workspace
PyCharm 101
Getting the data
Preparing the data – signal
Signal visualization
Backtesting
Summary
Section 2: Trading Signal Generation and Strategies
Deciphering the Markets with Technical Analysis
Designing a trading strategy based on trend- and momentum-based indicators
Support and resistance indicators
Creating trading signals based on fundamental technical analysis
Simple moving average
Implementation of the simple moving average
Exponential moving average
Implementation of the exponential moving average
Absolute price oscillator
Implementation of the absolute price oscillator
Moving average convergence divergence
Implementation of the moving average convergence divergence
Bollinger bands
Implementation of Bollinger bands
Relative strength indicator
Implementation of the relative strength indicator
Standard deviation
Implementing standard derivatives
Momentum
Implementation of momentum
Implementing advanced concepts, such as seasonality, in trading instruments
Summary
Predicting the Markets with Basic Machine Learning
Understanding the terminology and notations
Exploring our financial dataset
Creating predictive models using linear regression methods
Ordinary Least Squares
Regularization and shrinkage – LASSO and Ridge regression
Decision tree regression
Creating predictive models using linear classification methods
K-nearest neighbors
Support vector machine
Logistic regression
Summary
Section 3: Algorithmic Trading Strategies
Classical Trading Strategies Driven by Human Intuition
Creating a trading strategy based on momentum and trend following
Examples of momentum strategies
Python implementation
Dual moving average
Naive trading strategy
Turtle strategy
Creating a trading strategy that works for markets with reversion behavior
Examples of reversion strategies
Creating trading strategies that operate on linearly correlated groups of trading instruments
Summary
Sophisticated Algorithmic Strategies
Creating a trading strategy that adjusts for trading instrument volatility
Adjusting for trading instrument volatility in technical indicators
Adjusting for trading instrument volatility in trading strategies
Volatility adjusted mean reversion trading strategies
Mean reversion strategy using the absolute price oscillator trading signal
Mean reversion strategy that dynamically adjusts for changing volatility
Trend-following strategy using absolute price oscillator trading signal
Trend-following strategy that dynamically adjusts for changing volatility
Creating a trading strategy for economic events
Economic releases
Economic release format
Electronic economic release services
Economic releases in trading
Understanding and implementing basic statistical arbitrage trading strategies
Basics of StatArb
Lead-lag in StatArb
Adjusting portfolio composition and relationships
Infrastructure expenses in StatArb
StatArb trading strategy in Python
StatArb data set
Defining StatArb signal parameters
Defining StatArb trading parameters
Quantifying and computing StatArb trading signals
StatArb execution logic
StatArb signal and strategy performance analysis
Summary
Managing the Risk of Algorithmic Strategies
Differentiating between the types of risk and risk factors
Risk of trading losses
Regulation violation risks
Spoofing
Quote stuffing
Banging the close
Sources of risk
Software implementation risk
DevOps risk
Market risk
Quantifying the risk
The severity of risk violations
Differentiating the measures of risk
Stop-loss
Max drawdown
Position limits
Position holding time
Variance of PnLs
Sharpe ratio
Maximum executions per period
Maximum trade size
Volume limits
Making a risk management algorithm
Realistically adjusting risk
Summary 
Section 4: Building a Trading System
Building a Trading System in Python
Understanding the trading system
Gateways
Order book management
Strategy
Order management system 
Critical components
Non-critical components
Command and control
Services
Building a trading system in Python
LiquidityProvider class
Strategy class
OrderManager class
MarketSimulator class
TestTradingSimulation class
Designing a limit order book
Summary
Connecting to Trading Exchanges
Making a trading system trade with exchanges
Reviewing the Communication API
Network basics
Trading protocols
FIX communication protocols
Price updates
Orders
Receiving price updates
Initiator code example
Price updates
Sending orders and receiving a market response
Acceptor code example
Market Data request handling
Order
Other trading APIs
Summary
Creating a Backtester in Python
Learning how to build a backtester 
In-sample versus out-of-sample data
Paper trading (forward testing)
Naive data storage
HDF5 file
Databases
Relational databases
Non-relational databases
Learning how to choose the correct assumptions
For-loop backtest systems
Advantages
Disadvantages
Event-driven backtest systems
Advantages
Disadvantages
Evaluating what the value of time is
Backtesting the dual-moving average trading strategy
For-loop backtester
Event-based backtester
Summary
Section 5: Challenges in Algorithmic Trading
Adapting to Market Participants and Conditions
Strategy performance in backtester versus live markets
Impact of backtester dislocations
Signal validation
Strategy validation
Risk estimates
Risk management system
Choice of strategies for deployment
Expected performance
Causes of simulation dislocations
Slippage
Fees
Operational issues
Market data issues
Latency variance
Place-in-line estimates
Market impact
Tweaking backtesting and strategies in response to live trading
Historical market data accuracy
Measuring and modeling latencies
Improving backtesting sophistication
Adjusting expected performance for backtester bias
Analytics on live trading strategies
Continued profitability in algorithmic trading
Profit decay in algorithmic trading strategies
Signal decay due to lack of optimization
Signal decay due to absence of leading participants
Signal discovery by other participants
Profit decay due to exit of losing participants
Profit decay due to discovery by other participants
Profit decay due to changes in underlying assumptions/relationships
Seasonal profit decay
Adapting to market conditions and changing participants
Building a trading signals dictionary/database
Optimizing trading signals
Optimizing prediction models
Optimizing trading strategy parameters
Researching new trading signals
Expanding to new trading strategies
Portfolio optimization
Uniform risk allocation
PnL-based risk allocation
PnL-sharpe-based risk allocation
Markowitz allocation
Regime Predictive allocation
Incorporating technological advances
Summary
Final words
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
In modern times, it is increasingly difficult to gain a significant competitive edge just by being faster than others, which means relying on sophisticated trading signals, predictive models, and strategies. In our book Learn Algorithmic Trading, we provide a broad audience withthe knowledge and hands-on practical experience required to build a good understanding of how modern electronic trading markets and market participants operate, as well as how to go about designing, building, and operating all the components required to build a practical and profitable algorithmic trading business using Python.
You will be introduced to algorithmic trading and setting up the environment required to perform tasks throughout the book. You will learn the key components of an algorithmic trading business and the questions you need to ask before embarking on an automated trading project.
Later, you will learn how quantitative trading signals and trading strategies are developed. You will get to grips with the workings and implementation of some well-known trading strategies. You will also understand, implement, and analyze more sophisticated trading strategies, including volatility strategies, economic release strategies, and statistical arbitrage. You will learn how to build a trading bot from scratch using the algorithms built in the previous sections.
By now, you will be ready to connect to the market and start researching, implementing, evaluating, and safely operating algorithmic trading strategies in live markets.
Who this book is for
This book is for software engineers, financial traders, data analysts, entrepreneurs, and anyone who wants to begin their journey in algorithmic trading. If you want to understand how algorithmic trading works, what all the components of a trading system are, the protocols and algorithms required for black box and gray box trading, and how to build a completely automated and profitable trading business, then this book is what you need!
What this book covers
Chapter 1, Algorithmic Trading Fundamentals, explains what algorithmic trading is and how algorithmic trading is related to high frequency or low latency trading. We will discuss the evolution of algorithmic trading, from rule-based to AI. We will look at essential algorithmic trading concepts, asset classes, and instruments. You will learn how to set up your mind for algorithmic decisions.
Chapter 2, Deciphering the Markets with Technical Analysis, covers some popular technical analysis methods and shows how to apply them to the analysis of market data. We will perform basic algorithmic trading using market trends, support, and resistance.
Chapter 3, Predicting the Markets with Basic Machine Learning, reviews and implements a number of simple regression and classification methods and explains the advantages of applying supervised statistical learning methods to trading.
Chapter 4, Classical Trading Strategies Driven by Human Intuition, looks at some basic algorithmic strategies (momentum, trend, mean-reversion), and explains their workings, as well as their advantages and disadvantages.
Chapter 5, Sophisticated Algorithmic Strategies, consolidates the basic algorithmic strategies by looking at more advanced approaches (statistical arbitrage, pair correlation), as well as their advantages and disadvantages.
Chapter 6, Managing Risk in Algorithmic Strategies, explains how to measure and manage risk (market risk, operational risk, and software implementation bugs) in algorithmic strategies.
Chapter 7, Building a Trading System in Python, describes the functional components supporting the trading strategy based on the algorithm created in the preceding chapters. We will be using Python to build a small trading system, and will use the algorithm from the preceding chapters to build a trading system capable of trading.
Chapter 8, Connecting to Trading Exchanges, describes the communication components of a trading system. We will be using the quickfix library in Python to connect the trading system to a real exchange.
Chapter 9, Creating a Backtester in Python, explains how to improve your trading algorithm by running tests with large amounts of data to validate the performance of your trading bot. Once a model is implemented, it is necessary to test whether the trading robot behaves as expected in the trading infrastructure (by checking for implementation-related mistakes).
Chapter 10, Adapting to Market Participants and Conditions, discusses why strategies do not perform as expected when deployed in live trading markets and provides examples of how to address those issues in the strategies themselves or the underlying assumptions. We will also discuss why strategies that are performing well slowly deteriorate in terms of performance and provide some simple examples to explain how to address this.
To get the most out of this book
Readers should have a basic knowledge of finance and Python.
Download the example code files
You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register at
www.packt.com
.
Select the
Support
tab.
Click on
Code Downloads
.
Enter the name of the book in the
Search
box and follow the onscreen instructions.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Learn-Algorithmic-Trading. In case there's an update to the code, it will be updated on the existing GitHub repository.
We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Download the color images
We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: http://www.packtpub.com/sites/default/files/downloads/9781789348347_ColorImages.pdf.
Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in, and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packt.com.
Section 1: Introduction and Environment Setup
In this section, you will be introduced to algorithmic trading and setting up the environment required to perform tasks throughout the book. You will learn the key components of trading and the questions you need to ask before embarking on a robot trading project.
This section comprises the following chapter:
Chapter 1
,
Algorithmic Trading Fundamentals
Algorithmic Trading Fundamentals
Algorithmic trading, or automated trading, works with a program that contains a set of instructions for trading purposes. Compared to a human trader, this trade can generate profits and losses at a higher speed. In this chapter, this will be your first time being exposed to trading automation. We will walk you through the different steps to implement your first trading robot. You will learn the trading world and the technical trading components behind it. We will also go into detail about the tools that you will use and, by the end of this chapter, you will be capable of coding your first native trading strategy in Python. We will cover the following topics in this chapter:
Why are we trading?
Introducing algorithm trading and automation
What the main trading components are
Setting up your first programming environment
Implementing your first native strategy
Why are we trading?
From the Roman era through to the present day, trading is an inherent part of humankind. Buying raw materials when the price is low to resell it when the price is high has been a part of many cultures. In ancient Rome, the rich Romans used the Roman Forum to exchange currencies, bonds, and investments. In the 14th century, traders negotiated government debts in Venice. The earliest form of the stock exchange was created in Antwerp, Belgium, in 1531. Traders used to meet regularly to exchange promissory notes and bonds. The conquests of new worlds entailed a high cost, but also a good return. The Dutch East India Company in 1602 opened their capital for investors to participate in this costly project with a high potential return. During the same time period, a well-known tulip was sold everywhere in the world, creating a profitable market for investors and sellers. A future contract was created for this reason, since many people speculated regarding the price of this flower.
A hundred years later, a French expedition to Louisiana was also attracting many investors, creating the dream of making a lot of money. The Mississippi Company was created to handle all the investments based on potential wealth in Louisiana. Many other investment opportunities arose across the centuries, including the British railroad and the conquest of Latin America.
All these events had a common root: wealthy people willing to make more money. If we want to answer the question Why are we trading?, the answer is to potentially make more money. However, all the previous historical examples ended pretty badly. Investments turned out to be bad investments or, most of the time, the value was over-estimated and traders ended up losing their money. This is actually a good lesson for the readers of this book. Even if trading can sound a profitable business, always keep in mind the ephemeral part of profitability (it can work sometimes, but not always) and also taking into account the inherent risk that goes with investment.
Basic concepts regarding the modern trading setup
This section will cover the basics of trading and what drives market prices, as well as supply and demand.
As we touched upon in the previous section, trading has been around since the beginning of time, when people wanted to exchange goods between one another and make profits while doing so. Modern markets are still driven by basic economic principles of supply and demand. When demand outweighs supply, prices of a commodity or service are likely to rise higher to reflect the relative shortage of the commodity or service in relation to the demand for it. Conversely, if the market is flooded with a lot of sellers for a particular product, prices are likely to drop. Hence, the market is always trying to reflect the equilibrium price between available supply and demand for a particular product. We will see later how this is the fundamental driver of price discovery in today's markets. With the evolution of modern markets and available technology, price discovery becomes increasingly efficient.
Intuitively, you may draw a parallel with the fact that with the advances in online retail businesses, prices of products have become increasingly efficient across all sellers, and the best offers are always the ones that customers are buying because the information (price discovery) is so easily accessible. The same is true for modern trading. With advances in technology and regulations, more and more market participants have access to complete market data that makes price discovery much more efficient than in the past. Of course, the speed at which participants receive information, the speed at which they react, the granularity of the data that they can receive and handle, and the sophistication with which each participant draws trading insights from the data they receive, is where the competition lies in modern trading, and we will go over these in the subsequent sections. But first, let's introduce some basic concepts regarding the modern trading setup.
Market sectors
In this section, we will briefly introduce the concepts of what different types of market sectors are and how they differ from the concept of asset classes.
Market sectors are the different kinds of underlying products that can be traded. The most popular market sectors are commodities (metals, agricultural produce), energy (oil, gas), equities (stocks of different companies), interest rate bonds (coupons you get in exchange for debt, which accrues interest, hence the name), and foreign exchange (cash exchange rates between currencies for different countries):
Asset classes
Asset classes are the different kinds of actual vehicles that are available for trading at different exchanges. For example, cash interest rate bonds, cash foreign exchange, and cash stock shares are what we described in the previous section, but we can have financial instruments that are derivatives of these underlying products. Derivatives are instruments that are built on top of other instruments and have some additional constraints, which we will explore in this section. The two most popular derivatives are futures and options, and are heavily traded across all derivatives electronic exchanges.
We can have future contracts pertaining to underlying commodities, energy, equities, interest rate bonds, and foreign exchanges that are tied to the prices of the underlying instruments, but have different characteristics and rules. A simple way to think of a future contract is that it is a contract between a buyer and a seller in which the seller promises to sell a certain amount of the underlying product at a certain date in the future (also known as the expiry date), and where the buyer agrees to accept the agreed-upon amount at the specific date at the specific price.
For example, a producer of butter might want to protect themselves from a potential future spike in the price of milk, on which the production costs of butter directly depend, in which case, the butter producer can enter into an agreement with a milk producer to provide them with enough milk in the future at a certain price. Conversely, a milk producer may worry about possible buyers of milk in the future and may want to reduce the risk by making an agreement with butter producers to buy at least a certain amount of milk in the future at a certain price, since milk is perishable and a lack of supply would mean a total loss for a milk producer. This is a very simple example of a future contract trade; modern future contracts are much more complex than this.
Similar to future contracts, we can have options contracts for underlying commodities, energy, equities, interest rate bonds, and foreign exchanges that are tied to the prices of the underlying instruments, but have different characteristics and rules. The difference in an options contract compared to a futures contract is that the buyer and seller of an options contract have the option of refusing to buy or sell at the specific amount, at the specific date, and at the specific price. To safeguard both counterparties involved in an options trade, we have the concept of a premium, which is the minimum amount of money that has been paid upfront to buy/sell an options contract.
A call option, or the right to buy, but not an obligation to buy at expiration, makes money if the price of the underlying product increases prior to expiration because now, such a party can exercise their option at expiration and buy the underlying product at a price lower than the current market price. Conversely, if the price of the underlying product goes down prior to expiration, such a party now has the option of backing out of exercising their option and thus, only losing the premium that they paid for. Put options are analogous, but they give the holder of a put contract the right to sell, but not an obligation to sell, at expiration.
We will not delve too deeply into different financial products and derivatives since that is not the focus of this book, but this brief introduction was meant to introduce the idea that there are a lot of different tradeable financial products out there and that they vary significantly in terms of their rules and complexity.
Basics of what a modern trading exchange looks like
Since this book is primarily designed to introduce what modern algorithmic trading looks like, we will focus on trying to understand how a modern electronic trading exchange appears. Gone are the days of people yelling at one another in the trading pits and making hand signals to convey their intentions to buy and sell products at certain prices. These remain amusing ideas for movies, but modern trading looks significantly different.
Today, most of the trading is done electronically through different software applications. Market data feed handlers process and understand market data disseminated by the trading exchanges to reflect the true state of the limit book and market prices (bids and offers). The market data is published in a specific market data protocol previously agreed upon by the exchange and the market participants (FIX/FAST, ITCH, and HSVF). Then, the same software applications can relay that information back to humans or make decisions themselves algorithmically. Those decisions are then again communicated to the exchange by a similar software application (order entry gateways) that informs the exchange of our interest in a specific product and our interest in buying or selling that product at specific prices by sending specific order types (GTDs, GTCs, IOCs, and so on). This involves understanding and communicating with the exchange in an exchange order entry protocol previously agreed upon by the exchange and participants at the exchange (FIX, OMEX, OUCH).
After a match takes place against available market participants, that match is conveyed back to the software application again via the order entry gateways and relayed back to the trading algorithm or the humans, thereby completing a transaction, often wholly electronically. The speed of this round trip varies a lot based on the market, the participant, and the algorithms themselves. This can be as low as under 10 microseconds all the way up to seconds, but we will discuss this in greater detail later.
The following diagram is a descriptive view of the flow of information from an electronic trading exchange to the market participants involved, and the flow of information back to the exchange:
As shown in the preceding diagram, the trading exchange maintains a book of client buy orders (bids) and client ask orders (asks), and publishes market data using market data protocols to provide the state of the book to all market participants. Market data feed handlers on the client side decode the incoming market data feed and build a limit order book on their side to reflect the state of the order book as the exchange sees it. This is then propagated through the client's trading algorithm and then goes through the order entry gateway to generate an outgoing order flow. The outgoing order flow is communicated to the exchange via order entry protocols. This, in turn, will generate further market data flow, and so the trading information cycle continues.
Understanding algorithmic trading concepts
We introduced a lot of new concepts in the previous section, such as exchange order books (consisting of different kinds of orders sent by market participants), exchange matching algorithms, exchange market data protocols, and exchange order entry protocols. Let's formally discuss these in greater detail here.
Exchange order book
The exchange order book maintains all incoming buy and sell orders placed by clients. It tracks all attributes for incoming orders—prices, number of contracts/shares, order types, and participant identification. Buy orders (or bids) are sorted from the highest price (best price) to the lowest price (worst price). Bids with higher prices have a higher priority as far as matching is concerned. Bids at the same price are prioritized depending on the matching algorithm. The simplest FIFO (First In First Out) algorithm uses the intuitive rule of prioritizing orders at the same price in the order in which they came in. This will be important later on when we discuss how sophisticated trading algorithms use speed and intelligence to get higher priorities for their orders and how this impacts profitability. Sell orders (or asks) are sorted from the lowest price (best price) to the highest price (worst price). Again, as regards asks at the same price, the matching prioritization method depends on the matching algorithm adopted by the exchange for the specific product, which we will expand upon in greater detail in the next section. Market participants are allowed to place new orders, cancel existing orders, or modify order attributes such as price and the number of shares/contracts, and the exchange generates public market data in response to each order sent by participants. Using the market data distributed by the exchange, market participants can get an accurate idea of what the order book at the exchange looks like (depending on what information the exchange chooses to hide, but we ignore that nuance for now).
Exchange matching algorithm
When incoming bids are at or above the best (lowest price) ask orders, then a match occurs. Conversely, when incoming asks are at or below the best (highest price) bid orders, then a match occurs. Incoming aggressive orders continue to match against existing passive orders in the book until one of these two conditions is met. Either the new aggressive order is fully matched, or the other possibility is that the remaining orders on the opposite side have prices worse than the incoming order price and, hence, the match cannot occur. This is because of the fundamental rule that an order cannot be matched at a price worse than the limit price it was entered at. Now, as far as orders at the same price level are concerned, the order of matching is dictated by what matching algorithm rules the exchange adopts.
FIFO matching
We briefly described the FIFO algorithm previously, but let's expand on it by showing an example. Assume the following state of an order book when the exchange bid orders A, B, and C were entered at price 10.00 in that order in time. So, at the same price, order A has a higher priority than order B, which has a higher priority than order C. Bid order D is at a worse price, 9.00. Similarly, on the ask side, order X was entered at price 11.00 before order Y, also at price 11.00. Hence, order X has a higher priority than order Y, and then ask order Z was entered at a worse price, 12.00:
BIDS
ASKS
Order A: Buy 1 @ 10.00
Order X: Sell 1 @ 11.00
Order B: Buy 2 @ 10.00
Order Y: Sell 2 @ 11.00
Order C: Buy 3 @ 10.00
Order Z: Sell 2 @ 12.00
Order D: Buy 1 @ 9.00
Assume an incoming sell order K for 4 shares @ 10.00 would match against order A for 1 share, order B for 2 shares, and order C for 1 share, in that order, under FIFO matching. At the end of the matching, order C would still have the remaining size of 2 shares at price 10.00 and will have the highest priority.
Pro-rata matching
Pro-rata matching comes in a variety of flavors and is usually implemented in slightly different ways. For the scope of this book, we provide some intuition behind this matching algorithm and provide a hypothetical matching scenario.
The underlying intuition between pro-rata matching is that it favors larger orders over smaller orders at the same price and ignores the time at which the orders were entered. This changes the market's microstructure quite a bit, and the participants are favored to enter larger orders to gain priority instead of entering orders as fast as possible:
BIDS
ASKS
Order A: Buy 100 @ 10.00
Order X: Sell 100 @ 11.00
Order B: Buy 200 @ 10.00
Order Y: Sell 200 @ 11.00
Order C: Buy 700 @ 10.00
Order Z: Sell 200 @ 12.00
Order D: Buy 100 @ 9.00
Consider a market state as shown earlier. For the sake of this example, the hypothetical order sizes have been raised by a factor of 100. Here, bid orders A, B, and C are at the same price, 10.00. However, when an incoming sell order of size 100 comes in for price 10.00, order C gets a fill for 70 contracts, order B gets a fill for 20 contracts, and order A gets a fill for 10 contracts, proportional to how big they are at that level. This is an overly simplified example that excludes complications related to fractional match sizes and breaking ties between orders with the same size, and so on. Also, some exchanges have a mix of pro-rata and FIFO, where part of the incoming aggressor matches using pro-rata, and part matches in FIFO order. But this should serve as a good basic understanding of how different pro-rata matching is compared to FIFO matching. A detailed examination of pro-rata matching and its impact is beyond the scope of this book, so we exclude it.
Limit order book
A limit order book is very similar in spirit to the exchange order book. The only difference is that this is built by the market participants based on the market data that is being sent out by the exchange in response to market participants sending orders to it. The limit order book is a central concept in all algorithmic trading, and one often found in all other forms of trading as well. The purpose is to collect and arrange bids and offers in a meaningful way to gain insight into the market participants present at any particular time, as well as gain insight regarding what the equilibrium prices are. We will revisit these in the next chapter when we dig deeper into technical analysis. Depending on what information the exchange decides to make available to all market participants via public market data, the limit order book that the client builds can be slightly different from what the order book at the exchange matching engine looks like.
Exchange market data protocols
Exchange market data protocols are not the focus of this book, so a rigorous treatment of this topic is beyond the scope of this book. These market data protocols are outgoing communication streams from the exchange to all market participants that are well-documented for new participants to build their software applications to subscribe, receive, decode, and check for errors and network losses. These are designed with latency, throughput, error tolerance, redundancy, and many other requirements in mind.
Market data feed handlers
Market data feed handlers are software applications that market participants build with a view to interfacing with the specific exchange market data protocol. These are able to subscribe, receive, decode, and check for errors and network losses, and are designed with latency, throughput, error tolerance, redundancy, and many other requirements in mind.
Order types
Most exchanges support a variety of orders that they accept from market participants. We will discuss a few of the most common types in this section.
IOC – Immediate Or Cancel
These orders never get added to the book. They either match against existing resting orders to a maximum of the IOC order size, or the rest of the incoming order gets canceled. If no resting order is available at a price that the IOC can match against, then the IOC is canceled in its entirety. IOC orders have the benefit of not resting in the book post matching and causing additional complexity with order management in trading algorithms.
GTD – Good Till Day
These orders get added to the book. If they match fully against existing resting orders in the book, then they don't get added, otherwise the remaining quantity on the order (which can be the entire original quantity if there's no partial match) gets added to the book and sits as resting orders that the incoming aggressors can match against. The benefits of GTD orders are that they can take advantage of FIFO matching algorithms by having better priorities than orders that just showed up in the book, but require more complex order management in trading algorithms.
Stop orders
Stop orders are orders that aren't in the book until a specific price (called the stop price) is traded in the market, at which point they become regular GTD orders at a pre-specified price. These orders are great as exit orders (either to liquidate a losing position or to realize profit on a winning position). We will revisit these orders after we have explained what having a losing or winning position means and what exiting a position means.
Exchange order entry protocols
Exchange order entry protocols are how market participant software applications send order requests (new, cancels, modifies) and how the exchange replies to these requests.
Order entry gateway
Order entry gateways are the market participant client applications that communicate with the exchange matching engine over the order entry protocols. These have to deal with order flow in a reliable manner, sending orders to the exchange, modifying and canceling those orders, and getting notifications when these orders are accepted, canceled, executed, and so on. Oftentimes, market participants run a second variant of order entry gateways that simply receive order-executed notifications to check consistency against the primary order entry gateway order flow. These are called drop-copy gateways.
Positions and profit and loss (PnL) management
Orders that get executed cause market participants to have positions in the instrument that they got executed, for the amount the order executed, and at the price of the execution (limit orders can match at better prices than they were entered for, but not worse). A buy side execution is called having a long position, while a sell side execution is called having a short position. When we have no position at all, this is referred to as being flat. Long positions make money when market prices are higher than the price of the position, and lose money when market prices are lower than the price of the position. Short positions, conversely, make money when market prices go down from the price of the position and lose money when market prices go up from the price of the position, hence, the well-known ideas of buy low, sell high, and buy high, sell higher, and so on.
Multiple buy executions, or multiple sell executions for different amounts and prices, cause the overall position price to be the volume weighted average of the execution prices and quantities. This is called the Volume Weighted Average Price (VWAP) of the position. Open positions are marked to market to get a sense of what the unrealized Profit and Loss (PnL) of the position is. This means that current market prices are compared to the price of the position; a long position where market prices have gone up is considered unrealized profit, and the opposite is considered unrealized loss. Similar terms apply to short positions. Profit or loss is realized when an open position is closed, meaning you sell to close a long position and you buy to close a short position. At that point, the PnL is given the term realized PnL. The total PnL at any point is the total of the realized PnLs so far and the unrealized PnLs for open positions at the market price.
From intuition to algorithmic trading
Here, we will discuss how trading ideas are born and how they are turned into algorithmic trading strategies. Fundamentally, all trading ideas are driven by human intuition to a large extent. If markets have been moving up/down all the time, you might intuitively think that it will continue to move in the same direction, which is the fundamental idea behind trend-following strategies. Conversely, you might argue that if prices have moved up/down a lot, it is mispriced and likely to move in the opposite direction, which is the fundamental idea behind mean reversion strategies. Intuitively, you may also reason that instruments that are very similar to one another, or loosely dependent on one another, will move together, which is the idea behind correlation-based trading or pairs trading. Since every market participant has their own view of the market, the final market prices are a reflection of the majority of market participants. If your views are aligned with the majority of the market participants, then that particular strategy is profitable in that particular instance. Of course, no trading idea can be right all the time, and whether a strategy is profitable or not depends on how often the idea is correct versus how often it is not correct.
Why do we need to automate trading?
Historically, human traders implemented such rule-based trading to manually enter orders, take positions, and make profits or losses through the day. Over time, with advances in technology, they've moved from yelling in the pits to get orders executed with other pit traders, to calling up a broker and entering orders over the telephone, to having GUI applications that allow entering orders through point and click interfaces.
Such manual approaches have a lot of drawbacks – humans are slow to react to markets so they miss information or are slow to react to new information, they can't scale well or focus on multiple things at a time, humans are prone to making mistakes, they get distracted, and they feel a fear of losing money and a joy of making money. All of these drawbacks cause them to deviate from a planned trading strategy, severely limiting the profitability of the trading strategy.
Computers are extremely good at rule-based repetitive tasks. When designed and programmed correctly, they can execute instructions and algorithms extremely quickly, and can be scaled and deployed across a lot of instruments seamlessly. They are extremely fast at reacting to market data, and they don't get distracted or make mistakes (unless they were programmed incorrectly, which is a software bug and not a drawback of computers themselves). They don't have emotions, so don't deviate from what they are programmed to do. All of these advantages make computerized automated trading systems extremely profitable when done right, which is where algorithmic trading starts.
