Getting Started with Forex Trading Using Python E-Book

Alex Krishtop

• Herausgeber: Packt Publishing
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 573

Veröffentlichungsjahr: 2023

Beschreibung

Algorithm-based trading is a popular choice for Python programmers due to its apparent simplicity. However, very few traders get the results they want, partly because they aren’t able to capture the complexity of the factors that influence the market. Getting Started with Forex Trading Using Python helps you understand the market and build an application that reaps desirable results. The book is a comprehensive guide to everything that is market-related: data, orders, trading venues, and risk. From the programming side, you’ll learn the general architecture of trading applications, systemic risk management, de-facto industry standards such as FIX protocol, and practical examples of using simple Python codes. You’ll gain an understanding of how to connect to data sources and brokers, implement trading logic, and perform realistic tests. Throughout the book, you’ll be encouraged to further study the intricacies of algo trading with the help of code snippets. By the end of this book, you’ll have a deep understanding of the fx market from the perspective of a professional trader. You’ll learn to retrieve market data, clean it, filter it, compress it into various formats, apply trading logic, emulate the execution of orders, and test the trading app before trading live.

Leseprobe

Getting Started with Forex Trading Using Python

Beginner’s guide to the currency market and development of trading algorithms

Alex Krishtop

BIRMINGHAM—MUMBAI

Getting Started with Forex Trading Using Python

Copyright © 2023 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Publishing Product Manager: Apeksha Shetty

Senior Editor: Tazeen Shaikh

Technical Editor: Kavyashree K S

Copy Editor: Safis Editing

Project Coordinator: Farheen Fathima

Proofreader: Safis Editing

Indexer: Sejal Dsilva

Production Designer: Vijay Kamble

Marketing Coordinator: Nivedita Singh

First published: March 2023

Production reference: 2200323

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80461-685-7

www.packtpub.com

To my loving wife, Emily, without whose perpetual kicking I would have never accomplished this book.

And to my son, Daniel, who is making his first steps in the weird world of data science and Python programming.

– Alex Krishtop

Contributors

About the author

Alex Krishtop is a quantitative trader and researcher with 20 years of experience in developing automated trading solutions. He is currently the head of trading and research at Edgesense Technologies and CTO at Econq Ltd. He develops market models and trading algorithms for FX, commodities, and crypto. He was one of the first traders who started using Python as the ultimate environment for quantitative trading and suggested a few approaches to developing trading apps that, today, have become standard among many quant traders. He has worked as a director of education with the Algorithmic Traders Association, where he developed an exhaustive course in systematic and algo trading, which covers the worlds of both quantitative models and discretionary approaches.

About the reviewer

Andrey Lyashenko is a VP quantitative analyst at one of the largest investment banks in Eastern Europe and has around 20 years of experience in information technologies and finance in various roles. He has worked primarily in the development of high-availability and high-performance systems (including trading platforms) and has experience in private equity and leveraged algorithmic trading in financial markets. In trading, he takes a systematic, science-based approach to developing and implementing investment strategies.

Table of Contents

Preface

Part 1: Introduction to FX Trading Strategy Development

1

Developing Trading Strategies – Why They Are Different

Trading strategies – it’s all about how you manage risk

Trading strategies – how we make money in financial markets

Trading app – what can be simpler?

So, what about that risk?

Automated trading – operational risk and regulatory limitations

Retrieving market data – quality and consistency as keys to success

Receiving data – when size does matter

Received data – looking at it from a critical angle

Making actual trading decisions – trading logic and credit risk

Ordering – transactional risk

The general architecture of a trading application

Summary

2

Using Python for Trading Strategies

Technical requirements

The advantages of using Python in trading strategy development

Memory management

Interactive computing

Ease of integration and routing

Native language structures – lists and dictionaries

Libraries

Modeling – forecasting the future doesn’t necessarily mean practical benefits

Paper trading and backtesting – an essential part of a systemic trader’s risk management

What are paper trading and backtesting?

Backtesting and paper trading in Python

The disadvantages of using Python in trading strategy development

Live trading – where Python faces its limits

Summary

3

FX Market Overview from a Developer's Standpoint

Trading venues – where money meets… other money

Organizing chaos – types of trading venues

Auction (open outcry)

Exchange and order book

OTC markets

Liquidity and order limitations

Trade mechanics – again, some terminology

Market makers – comfortable, sophisticated, expensive

Counterparties that provide liquidity

How do market makers make money?

Market risk and how market makers mitigate it

Reasons to trade with market makers

Liquidity providers – the whales that support this planet

Liquidity and volatility – how one transforms into another

ECN – looks like a fair game, but is it?

The organization of ECNs

ECN – not all market participants were created equal

Aggregation – in search of the best price

Trading the FX market – what and how

FX instruments

Naming conventions

How do I place a trade?

Why do I need all this?

Summary

Part 2: General Architecture of a Trading Application and A Detailed Study of Its Components

4

Trading Application: What’s Inside?

Technical requirements

Have your app talk to the world – the gloomy world of communication protocols

FIX – universal but too flexible

Retrieving data – garbage in, garbage out

Tick sequence

Time gaps

Price gaps

Trading logic – this is where a small mistake may cost a fortune

Risk management – your safety belt

Ordering – make sure you are understood correctly

Summary

5

Retrieving and Handling Market Data with Python

Navigating through data

Tick data and snapshots

Timestamps – be careful comparing apples to apples!

Storing and retrieving tick data

Order book (depth of market)

Storing and retrieving order book data

Data compression – keeping the amounts to a reasonable minimum

Retrieving compressed data

Compressing market data in Python

Working with saved and live data – keeping your app universal

Universal data connector

Data compression revisited

Summary

6

Basics of Fundamental Analysis and Its Possible Use in FX Trading

Fundamental analysis – intuitive but of little practical use

Economic news

US non-farm payrolls

Gross domestic product (GDP)

Economic news from a fundamental analysis perspective

Political events

US presidential elections, November 8, 2016

UK Brexit, June 23, 2016

Industry-specific information

Crude oil and the Canadian dollar

Summary

7

Technical Analysis and Its Implementation in Python

Technical analysis – ideal for computing, but missing real market processes

Velocity of the market

Digital filters

Range indicators

Volatility indicators

Key takeaways – what TA indicators are all about and how to use them

Implementation of TA indicators in Python

Sliding windows

Peeking ahead

The ultimate solution to the peek-ahead issue

Sliding window as a queue

Moving average – implementation

Stochastic oscillator – implementation

Summary

8

Data Visualization in FX Trading with Python

Technical requirements

The basics of charting with Python

Graphical backends

Static versus real-time data and related issues

Installing Matplotlib

Using Matplotlib in your code

Simple plots of market data

Visualizing static market data with pandas

Installing pandas

Reading data from CSV files with pandas

Setting index for a dataframe

Creating simple bar charts using pandas and mplfinance

Making slices and subsets in pandas

Plotting market charts with mplfinance

Visualizing live market data

Plotting live tick data

Plotting live bar or candlestick charts

Adding objects to price charts

Summary

Part 3: Orders, Trading Strategies, and Their Performance

9

Trading Strategies and Their Core Elements

Alpha and beta – widely used, widely confused

Alpha – earn from changes in price

Using currency rates as a benchmark

Common FX benchmarks – the USDX

Bank indices – more details, more confusion

Beta – earn on volatility

Options – stable income with unlimited risk

Alpha classics – trend-following, mean reversion, breakout, and momentum

Trend following

Mean reversion

Breakouts

Arbitrage – let’s earn from others’ mistakes

Statistical arbitrage

Event-driven trading strategies

Market-making – profiting on liquidity provision and associated risks

High frequency, low latency – where Python fails

Summary

10

Types of Orders and Their Simulation in Python

Order ticket – what you send is what you get

Market orders – the way to get maximum control over transactional risk

Possible execution issues

Limit orders – guaranteed price, but not execution

Possible execution issues

Time in force – better control over execution

Immediate or cancel

Fill or kill

Good for a day (GTD) and good till canceled (GTC)

Stop orders – maximum uncontrolled risk

Possible execution issues

Compound orders

Summary

11

Backtesting and Theoretical Performance

Trading app architecture – revised and improved

Market data component

Data aggregation component

Trading logic component

Order execution component

Advantages of the modular architecture

Multithreading – convenient but full of surprises

Trading application with live data feed

Backtesting – speeding up the research

Syncing threads using events

Backtesting platform with a historical data feed

Emulating order execution during backtesting

Equity curve and statistics

Summary – where do we go now?

Part 4: Strategies, Performance Analysis, and Vistas

12

Sample Strategy – Trend-Following

Trend-following revisited – trading setup

Determining a trend, part 1 – market model

Determining a trend, part 2 – moving averages

Entry and exit rules

Money management

Choosing the market and preparing data

Compressing data to a daily timeframe

Be careful with time!

Trend-following strategy – implementation

Summary

13

To Trade or Not to Trade – Performance Analysis

Trade analysis

Average trade and trading costs

Spread

Slippage

Commission

Overnight swaps

Measuring performance – alpha and beta revisited

Alpha in investment and trading

Beta

Net profit versus buy and hold

Drawdown

Risk/reward and return on drawdown

The power of leverage – how much do I need to trade it?

Summary

14

Where to Go Now?

Implementing limit and stop orders

The correct way to calculate the number of trades

From trading ideas to implementation – another sample strategy using limit and stop orders

Contingent orders

Money management and multiple entries

Strategy performance revisited – more metrics

More about the risks specific to algo trading

Classical technical trading setups

Optimization – the blessing and the curse of algo trading

Final words

Index

Other Books You May Enjoy

Preface

Forex trading has been popular for the last 20 years and its popularity is only growing. This market attracts new traders mostly because of huge potential profits and the ability to start with a relatively small amount of initial capital. However, at the same time, it is known to be one of the most dangerous markets, as according to the statistics of most regulators, more than 75% of forex traders completely lose their money.

This happens for a number of reasons. Some newcomers to forex tend to treat it the same as stock trading and try to invest in a market that doesn’t have an intrinsic upside potential. Others want to take advantage of high leverage, being inspired by multiple “get rich quick” books to hunt for legendary profits, incorrectly calculating their exposure, and then quickly losing their entire deposit.

But the main problems for all beginner forex traders are a lack of trading ideas and a lack of understanding of why the market may do this or that, when it may be reasonable to place a trade, when it may be problematic, and when to stay away from the market entirely to avoid almost inevitable losses.

Algorithmic trading can provide a solution to this problem because it’s based on rules that can be tested before any real money is put at stake. Beyond that, trading automation helps to mitigate operational risks and provides full control over execution, which is not achievable with manual order placement.

This book bridges the gap between the “easy forex” advertised by many retail brokers and the real forex market, with all its caveats and pitfalls, from the perspective of a professional algo trader. It guides the reader through all the necessary steps in developing automated trading strategies that have at least a chance of survival in this controversial market. It is not a collection of ready-made recipes for you to copy, paste, and run: the markets are constantly changing and you will have to adapt and re-adapt your strategies to them as they change.

The main goal of the book is to provide a clear understanding of what is possible, what is not, and what to realistically expect from algorithmic trading in forex. By the end of the book, you will have the essential knowledge of the forex market at the level of a professional desk trader. At the same time, you will have developed practical skills for implementing trading algorithms in Python. Although this book is only the first step for beginners into the world of forex trading, it can help you not only get started as a retail trader but also eventually find a job in this industry.

Who this book is for

This book is for anyone familiar with Python programming who wants to test the waters of algo trading. This is not a Python tutorial: I assume that you are well familiar with the language and object-oriented programming. No special knowledge in programming is required though, and any time a new type of object or structure is used, it is explained in detail.

No knowledge about markets and trading is required and nor is any prior experience in trading. I tried to explain the contents in the most simple terms but without simplifying the subject itself to help you expand your trading vocabulary and get a clear understanding of the meaning of the most essential terms.

At the same time, Python is used only to provide working examples of the concepts considered in the book. No special programming techniques specific to Python are used. This means that you can easily port the code or develop something on your own in other languages, so the book can also be quite useful for traders who use other platforms, such as MetaTrader.

What this book covers

Chapter 1, Developing Trading Strategies – Why They Are Different, introduces the concept of trading strategies and automated trading and highlights the main components of a trading application, such as receiving market data, making trading decisions, and executing orders.

Chapter 2, Using Python for Trading Strategies, focuses on using Python for market analysis, modeling, and actual trading.

Chapter 3, FX Market Overview from a Developer’s Standpoint, explains how markets operate, focuses on the different kinds of market participants and the way they affect the price, and introduces core terminology essential for any further study.

Chapter 4, Trading Application – What’s Inside?, expands the first draft of a trading app architecture that was proposed in Chapter 1 and provides a more detailed view of its main components.

Chapter 5, Retrieving and Handling Market Data with Python, provides practical examples of manipulating ticks and order book data, explains the use of data compression, and introduces the concept of a universal data connector.

Chapter 6, Basics of Fundamental Analysis and Its Possible Use in FX Trading, explains the difference between fundamental and technical analysis and shows examples of the most important fundamental events and the way they impact the market price.

Chapter 7, Technical Analysis and Its Implementation in Python, considers a number of the most well-known and typical technical studies, explains their relationships with the underlying price time series, and suggests possible implementations in Python.

Chapter 8, Data Visualization in FX Trading with Python, introduces basic plotting, which is used to visualize price time series and other graphical objects, such as equity curves and drawdowns.

Chapter 9, Trading Strategies and Their Core Elements, introduces alpha and beta as the key performance metrics, explains benchmarking in capital management, and discusses the most popular alpha-generating trading strategies.

Chapter 10, Types of Orders and Their Simulation in Python, introduces the concept of a trading order, explains the main types of orders, and emphasizes the risks associated with each of them.

Chapter 11, Backtesting and Theoretical Performance, discusses backtesting as the core of any trading strategy development, reviews the architecture of a trading application, introduces multithreading and provides a detailed use case of it to organize communication between the parts of an app, explains the logic involved, and proposes working code solutions for a trading app using live market data for backtesting.

Chapter 12, Sample Strategy – Trend-Following, carefully considers all the steps necessary to develop a simple trading strategy from scratch and provides some working Python code.

Chapter 13, To Trade or Not to Trade – Performance Analysis, introduces the very basic metrics used to assess the performance of a trading strategy and gives practical examples using the theoretical performance of the strategy developed in Chapter 12.

Chapter 14, Where to Go Now?, offers a collection of trading ideas, technical setups, and code snippets that you can use for your own development.

To get the most out of this book

Please remember that the code printed in the book is interwoven with explanations: this is intentionally done to comment on most of the lines so that no important action remains unexplained. Therefore, it is advised to first read the entire code with explanations, follow the logic, understand how it works, and only afterward copy the code from the book or GitHub and run it. Otherwise, there’s a risk that you will attempt to run only a part of the code, which may not work without the context.

Software/hardware covered in the book

Operating system requirements

Python 3.10 or above

Windows, macOS, or Linux

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Getting-Started-with-Forex-Trading-Using-Python. If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Conventions used

There is a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “Mount the downloaded WebStorm-10*.dmg disk image file as another disk in your system.”

A block of code is set as follows:

html, body, #map { height: 100%; margin: 0; padding: 0 }

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

[default] exten => s,1,Dial(Zap/1|30) exten => s,2,Voicemail(u100) exten => s,102,Voicemail(b100) exten => i,1,Voicemail(s0)

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “Select System info from the Administration panel.”

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Getting Started with Forex Trading Using Python, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781804616857

Submit your proof of purchaseThat’s it! We’ll send your free PDF and other benefits to your email directly

Part 1: Introduction to FX Trading Strategy Development

Success in any software development project is determined by two main factors – how fluent you are with the technology and how proficient you are in the subject domain.

Part 1 gives an essential overview of today’s FX markets, focusing on their peculiarities, which determine the choice of solutions used in developing actual code. By the end of Part 1, you will have gained a conceptual understanding of the market structure, its operations, and the key risks, and at the same time, you will have learned about approaches to designing the architecture of trading applications, which address specific market-related issues.

This part comprises the following chapters:

Chapter 1, Developing Trading Strategies – Why They Are DifferentChapter 2, Using Python for Trading StrategiesChapter 3, FX Market Overview from a Developer’s Standpoint

1

Developing Trading Strategies – Why They Are Different

For many years, I traded in various markets, educated developers in trading, and taught some essential programming and algorithm (algo) development to traders. Eventually, I concluded that the main problem for both groups (the developers and traders) was not a lack of knowledge in programming, but an incorrect or insufficient understanding of the subject domain, that is, financial markets and their mechanics.

This was quite a surprise for me. I thought that coding, debugging, and handling errors should be the main obstacle, but in reality, the problem is in finding useful, correct, and adequate information about markets, their structure, and operations, especially in regard to algo trading. I don’t mean that it’s somehow undisclosed or otherwise hidden from the general audience – not at all. The problem is that it’s really hard to find a source that not only covers everything – from the market structure to money and risk management – but also shows clear connections between very specific features of a market and less-specific features that a trading algorithm should have in order to be successful in this market.

For example, many authors consider entries and exits in a trading strategy as something that can be executed immediately and with equal success at any moment, any time of the day. However, in the real market, this is not the case. I saw lots of strategies that only worked on paper because their developers didn’t take into consideration the fact that they were dealing with a real, complex structure with lots of limitations and shortcomings, and not with an abstract idealistic model.

This example is really primitive, but I hope it explains the most important point that is missed by many researchers and developers: we do not deal with abstract datasets and time series where each data point has an equal value and where any action can be taken at any moment with equal possibility; we work with a very complex structure that has lots of features that make this structure a dynamic non-stationary system. And to be successful in algo trading, these features should be accounted for.

Without knowing how markets are organized and how they operate, our trading algorithms will inevitably fail sooner or later.

If put in a more positive manner, understanding the market will help us in making trading algorithms more robust.

This book is my humble attempt to bridge the gap between the world of quantitative trading (driven by numbers) and the real market (driven by humans). I am convinced that today, any long-term success in trading can be achieved only with good knowledge of both.

Let’s begin with the basics. We will learn about the very basic market terminology and the essential concept of a trading strategy, get familiar with the risk in various forms and aspects with a special focus on ordering and transactional risk, and also mention market data processing in a very general sense.

In this chapter, we will discuss the following topics:

Trading strategies – it’s all about how you manage riskAutomated trading – operational risk and regulatory limitationsMaking actual trading decisions – trading logic and credit riskOrdering – transactional riskThe general architecture of a trading application

Trading strategies – it’s all about how you manage risk

Let’s start from the beginning by giving definitions. I know it sounds like an academic textbook, but I promise that it’s going to get more and more interesting very quickly.

According to the definition provided by Cambridge Dictionary, trade is "the activity of buying and selling, or exchanging, goods and/or services between people or countries.” This means that if I exchange my money for meat or vegetables, then I am at a grocery market. If I exchange my money for obligations to buy or sell live cattle or wheat in the future, then I am in a derivatives market. And if I exchange my money for another type of money, I am in the foreign exchange, forex or FX market.

What I buy and sell in a particular market is called an asset. If I buy or sell something that can be delivered (wheat, gold, stocks, or even money) it’s called the underlying. If I buy or sell an obligation or the right to buy or sell the underlying at a certain price in the future, then it’s called a derivative.

Underlying and derivatives are traded at trading venues – previously, physical locations such as exchanges, but today, more electronic networks where traders, liquidity providers, market makers, and other market participants match their orders.

If I went to a grocery market to sell potatoes, then I am on the sell side. If I came to buy potatoes, I am on the buy side. In financial markets, buy-side market participants are mostly referred to as price takers (because they can only take prices offered by the sell side), and sell-side market participants are referred to as price givers.

Not all trading venues welcome everyone as a direct participant. Most of them work via a network of brokers – entities that accept orders from their clients and route them to one or more trading venues, possibly internally netting client positions and sometimes even acting as the counterparty for their clients.

In Chapter 3, FX Market Overview from a Developer’s Standpoint, we will consider all these entities in more detail to see why we should pay particular attention to their behavior and how to actually use them to make our trading strategies more robust.

Let’s proceed with a very quick look into the essence of a trading strategy and see the main risks that are natural to the systematic or algo trading business.

Trading strategies – how we make money in financial markets

A trading strategy is a set of rules that tells me when to buy and when to sell the asset if I am on the buy side, or when and how much to offer to the market if I am on the sell side. But in any case, the ultimate goal of a trading strategy is to make money.

If I am on the buy side, I can make money only in the following two ways:

I buy low and I sell high, or vice versa (earning on the price difference)I buy and receive dividends

The former is called active trading, and the latter is normally called investing, or getting passive income. In this book, we will consider only active trading.

There are two major classes of active trading strategies on the buy side. The first one is called directional trading and in essence, it’s when we buy, wait, and sell. If we manage to sell at a price greater than the price at which we bought, we make money.

The second class of buy-side trading strategies is arbitrage. Such a strategy identifies moments when the same asset is priced differently at different trading venues (so-called real or pure arbitrage), or when there is a tradable difference between the price of the underlying and a derivative, different derivatives, or even more complex setups that consist of multiple instruments (so-called statistical arbitrage or stat arb).

Sounds a bit overwhelming? Don’t worry, we will now consider each case in more detail.

Trading app – what can be simpler?

At this point, if you have sufficient experience in application development and a reasonable knowledge of markets, you might exclaim the following:

“Why do we need all this? It is so easy to build a trading application: you only need to get market data, calculate entry and exit orders, and send them to the broker!”

So, the suggested general architecture of a trading application from that standpoint may look as simple as this:

Figure 1.1 – Initial architecture of a trading application

However, as we will see very soon, this is an overly simplified view and it lacks at least one key feature that makes developing trading strategies different from developing other applications: it doesn’t include risk management.

So, what about that risk?

Before discussing ways to make money in trading, let’s first pay closer attention to avoiding losses – at least because according to all statistics, more than 70% of active traders, unfortunately, lose money.

Before we proceed, let me note that the following classification of risks does not fully correspond to official legal and academic classifications. This informal classification is used here for the sake of simplicity, to sort out rather complex matters quickly and with an acceptable degree of comfort.

All risks in active trading can be generally put into three major categories, as follows:

Operational risks are those associated with how you place trades and depend on your own business procedures as well as third parties, such as brokers, trading venues, and regulators.Systemic risks are those pertaining to the very market itself and the logic of a trading strategy.Transactional risks are those that make the execution of orders different from expectations; this is the main reason why many strategies that work on paper are unable to make money in reality.

Now that we are a bit more familiar with the idea of a trading strategy and know that the main problem any systematic trader should solve is risk mitigation, let’s dive a little deeper into risks specific to the algo trading business.

Automated trading – operational risk and regulatory limitations

Operational risk is the risk of direct or indirect loss resulting from inadequate or failed internal procedures, people, and systems, or from external events (Bank for International Settlements, Basel Committee on Banking Supervision, Operational Risk Supporting Documentation to the New Basel Capital Accord (Basel: BIS, 2002), p. 2, https://www.bis.org/publ/bcbsca07.pdf).

Since, in this book, we will talk mostly about developing trading algos with Python and not about running a trading business, the main operational risk in this context could be that you don’t follow your own strategy or intervene in the algorithmic trading process discretionarily.

Another risk that may be considered operational (although it is normally considered as money management) is the improper use of leverage. In essence, leverage is a credit line provided by the broker that allows you to buy more than you have in your account. If the leverage is too high, you are at risk of being unable to enter the market, or in certain cases, even worse – liquidating your positions that are rapidly losing money.

Broker risk can also be attributed to operational risks as the broker is the very entity that provides you with access to the market, gives you a credit line to open positions, and does the clearing and settlement. Some brokers also act as market makers for their clients, netting their positions internally and acting as the counterparty for their own clients, which may lead to a conflict of interest, and even worse – loss of money if the broker didn’t have sufficient capital to perform these operations.

Last, but not least, we should note that algorithmic and/or automated trading may be fully or partially prohibited in certain jurisdictions. So always check with the respective market regulators to make sure you can run your algo trading at all.

Key takeaways

Always perform a background check of all counterparties, especially your broker. Be careful with leverage and check local regulatory documents on algo trading.

Enough on operational risk – at least for a quick start – and let’s move on to another kind of risk that is common for any trading activity but becomes particularly problematic for algo trading: the risk of basing trading decisions on incorrect market data.

Retrieving market data – quality and consistency as keys to success

Market data is often considered something that cannot contribute to the overall risk in systematic trading. However, this is a massive mistake. There are two key risks associated with market data:

Issues with receiving dataIssues with received data

In the next two subsections, we will dive deeper into the preceding risks.

Receiving data – when size does matter

There are two forms in which we get market data: real-time or historical. In both cases, we obtain it from a data vendor, a broker, or directly from an exchange. The difference is that real-time data is used for actual trading (as it reflects what is going on in the market right now) and historical data is used only for research and development, to rebuild hypothetical trades and estimate the theoretical performance of a trading algorithm.

The issues with receiving data are mostly related to real-time data.

Let’s now add some more definitions as we need to acquire some common terminology to move on with market data and ordering.

A request to buy an asset at a certain price is called a bid. It’s like you go to a market and shout, “I want to buy this asset at this price. Is anyone willing to sell it to me?”

A request to sell an asset at a certain price is called an ask or offer. It means that you are ready to sell it to anyone willing to accept your price.

In financial markets, both requests are realized by buy side traders with a limit order (see Chapter 10, Types of Orders and Their Simulation in Python, for a detailed discussion on types of orders).

When another counterparty agrees to place a trade at the order price, a new trade is registered and its information is included in the data stream and distributed across data vendors, brokers, and other recipients. Such a record is called a tick. In other words, a tick is a minimal piece of information in the market data and normally consists of the following fields:

datetimepricetraded volumecounterparty1counterparty2

The last two fields contain information about actual counterparties and are normally not disclosed or distributed to protect the market participants. Traded volume means the amount of the asset that was traded (number of contracts, or just the amount of money if we are talking about forex).

The main problem with receiving market data in its raw form is that it’s simply overwhelmingly huge. There are so many market participants and so many trading venues that streaming all transactions for just one asset (which is also called a “financial instrument”) may easily reach megabytes per second – receiving it is already a challenge by itself (don’t worry, we are not going to work with data feeds of this sort in this book). Next, even if we are able to receive a data stream with such a throughput, we need to store and handle this data somehow, and thus a very fast database is required. And finally, we need to be able to process this amount of data at an adequate speed, so we need blazingly fast computers.

But there is good news. Despite some strategies (mostly arbitrage and high-frequency trading) do require raw market data in the format just described (also frequently referred to as time and sales data) to identify trading opportunities, most directional trading algorithms are far less sensitive to lack of information about each and every trade. So, data vendors provide data in a compressed format. This becomes possible because most of the raw market data contains sequences of ticks with identical prices, and removing them won’t distort the price movements. This happens because there may be many market participants placing trades at the same price at almost the same time, so by excluding these sequences, we lose information about each transaction but retain information about any change in price. Such a market data stream is often referred to as filtered or cleaned. Besides that, some trades are made at bids, others at asks, and while both bids and asks remain the same, these trades form sequences of trades where prices seem to be different. However, in reality, they are always at the distance of the difference between bids and asks. This difference doesn’t mean that the market price changes. Such a phenomenon is called a bounce and is normally also excluded from cleaned data.

Some vendors and brokers go even further and send snapshots of the market data instead of a filtered data stream. A snapshot is sent at regular time intervals, for example, 100 ms or 1 s, and contains only the following information:

DateTimePrice at the beginning of the interval (also known as open, or just O)Maximum price during the interval (also known as high, or H)Minimum price during the interval (also known as low, or L)Price at the end of the interval (also known as close, or C)Traded volume

Therefore, instead of thousands of ticks, we receive only one tick with seven data fields. This approach dramatically reduces the throughput but is obviously somewhat destructive to the data, and snapshot data may not be suitable for some strategies.

Key takeaway

Be careful with choosing the source of data, especially for live trading, and always make sure it contains sufficient information for your strategy.

Received data – looking at it from a critical angle

After we have successfully received the data, we should make sure it makes sense. Often, data, especially tick data, contains erroneous prices. These prices may be received due to a number of reasons, which we will discuss in detail in Chapter 5, Retrieving and Handling Market Data with Python.

Erroneous, otherwise known as non-market, prices may cause trouble for systematic traders because a single wrong quote may trigger an algorithm to buy or sell something, and such a trade should not have happened according to the strategy logic.

Sometimes, these incorrect quotes can be seen if plotted on a chart. The human eye intuitively expects data points to be within a certain reasonable range and easily catches the outliers, as can be seen in the following chart:

Figure 1.2 – Non-market prices seen on a tick chart

In case we receive snapshots or other compressed data, there could be missing intervals when we receive no quotes. It can happen because of the following:

The market is closed (scheduled or due to an emergency)The data server is downThe connection is broken

Key takeaway

A robust trading app should have a module capable of checking data consistency and connection persistence.

Alright, we are now aware of the operational risks and know how harmful incorrectly handling market data could be. Anything else? Of course, here comes the main risk: systemic.

Making actual trading decisions – trading logic and credit risk

In directional trading, systemic risks are mostly realized when you or your trading strategy supposed the price would go in one direction, but in reality, it went in the opposite direction. Don’t worry, this situation is absolutely normal for systematic trading, and no one can guarantee that any strategy will generate a win rate of 100%.

There are two major types of trading strategies: data-driven and event-driven. Data-driven strategies analyze the price time series (which we discussed in the Retrieving market data – quality and consistency as keys to success section) in order to find a certain pattern or sequence that then triggers an order. Event-driven strategies wait for a certain event to happen – for example, a new tick at a certain price and with a certain volume, or a release of political news or economic indicators. In both cases, a trading app should have rules to not only open positions but also to liquidate them – again, based on price data or events (or both).

In general, if a strategy generates some wins and some losses, there are only two ways for it to make money:

Generate more wins than lossesHave the average win greater than the average loss

If your trading algorithm does not have a routine that handles situations when the market goes against the position, then most likely, the average loss across a statistically significant number of trades will be greater than the average win, and it will become really problematic to make money with such a strategy.

Don’t forget that forex is traded using leverage, which means the ability to trade much greater amounts of money than you actually have in your account. In the case that your trading algorithm has poor risk management logic and incorrect position sizing, an open position may quickly drain your account to zero if the price goes the opposite way, and even worse – some brokers will even let you go negative and you will be left with debt instead of profit.

Key takeaway

Systemic risk management and position sizing algorithms are crucial parts of an algo trading application.

Systemic risk is very important, but there’s good news for systematic traders: it is possible to mitigate this risk by carefully testing the strategy before going to production and adjusting it so the systemic risk is minimized. But there is one more risk that is hard to mitigate during the research and development phase: transactional risk.

Ordering – transactional risk

Transactional risks are the real problem in the first place for arbitrage, but they also affect directional strategies. In simple terms, this is a risk of the following:

Entering or exiting the market at a wrong priceEntering or exiting the market at a wrong timeEntering or exiting the market with a wrong trading sizeNot entering or exiting the market at all

All four situations are more than possible in all markets and are even quite frequent during periods of insufficient liquidity (see Chapter 3, FX Market Overview from a Developer’s Standpoint, for a more detailed discussion of liquidity issues).

Key takeaway

Transactional risks are managed by a set of algorithms that are also an essential part of any trading application.

Well, it’s been quite a trip across the various risks, and we now understand that the initial idea of a trading application with a simple and straightforward linear logic definitely won’t work in real life. Now, we can suggest something (unfortunately) more complex, but (fortunately) more realistic.

The general architecture of a trading application

Now, we can improve our initial diagram (see Figure 1.1) representing the architecture of a trading application. Although it is still very general and high level, it now radically differs from what we suggested in the beginning:

Figure 1.3 – More complete general architecture of a trading app

Here, we can see that the real trading application features a number of blocks or modules, and some of them receive feedback from others. After we have received market data, we should clean it up and add storage and retrieval facilities, as pieces of data may be reused in the following code. Then, we generate trading signals according to the strategy logic. But before sending these signals to the market in the form of orders, we should do some checks to make sure that the order size is comfortable for both the strategy and the market. After that, we actually trade – send orders to the market controlling their execution. And finally, we track the open positions and manage exposure according to the running profit or loss.

Summary

In this chapter, we familiarized ourselves with the core terminology and key concepts of FX markets, learned about counterparties and transactions, understood the intrinsic issues with market data, reviewed various risks, and drafted the first mockup of a trading application’s architecture. We now know that a robust trading app is more about risk management, checking various situations that may happen in real life before and after the trade, and making corrections on the fly. This is what makes the development of trading apps different.

In the next chapter, let us see how using Python in algo trading helps improve the research and development of trading applications.