Evidence-Based Technical Analysis E-Book

David Aronson

• Herausgeber: John Wiley & Sons
• Kategorie: Fachliteratur
• Serie: Wiley Trading Series
• Sprache: Englisch

Seitenzahl: 863

Veröffentlichungsjahr: 2011

Beschreibung

Evidence-Based Technical Analysis examines how you can apply the scientific method, and recently developed statistical tests, to determine the true effectiveness of technical trading signals. Throughout the book, expert David Aronson provides you with comprehensive coverage of this new methodology, which is specifically designed for evaluating the performance of rules/signals that are discovered by data mining.

Leseprobe

Contents

Acknowledgments

About the Author

Introduction

Part I: Methodological, Psychological, Philosophical, and Statistical Foundations

Chapter 1: Objective Rules and Their Evaluation

The Great Divide: Objective Versus Subjective Technical Analysis

TA Rules

Traditional Rules and Inverse Rules

The Use of Benchmarks in Rule Evaluation

Other Details: The Look-Ahead Bias and Trading Costs

Chapter 2: The Illusory Validity of Subjective Technical Analysis

Subjective TA is Not Legitimate Knowledge

A Personal Anecdote: First A True TA Believer, Then A Skeptic

The Mind: A Natural Pattern Finder

The Epidemic of Weird Beliefs

Cognitive Psychology: Heuristics, Biases, and Illusions

Human Information Processing Limitations

Too Dang Certain: The Overconfidence BIAS

Second-Hand Information BIAS: The Power of A Good Story

Confirmation BIAS: How Existing Beliefs Filter Experience and Survive Contradicting Evidence

Illusory Correlations

Misplaced Faith in Chart Analysis

The Intuitive Judgment and The Role of Heuristics

The Representativeness Heuristic and The Illusion Trends and Patterns in Charts: Real and Fake

The Antidote To Illusory Knowledge: The Scientific Method

Chapter 3: The Scientific Method and Technical Analysis

The Most Important Knowledge of All: A Method to Get More

The Legacy of Greek Science: A Mixed Blessing

The Birth of The Scientific Revolution

Faith in Objective Reality and Objective Observations

The Nature of Scientific Knowledge

The Role of Logic In Science

The Philosophy of Science

The End Result: The Hypothetico-Deductive Method

Rigorous and Critical Analysis of Observed Results

Summary of Key Aspects of The Scientific Method

If TA Were to Adopt The Scientific Method

Objectification of Subjective TA: An Example

Subsets of TA

Chapter 4: Statistical Analysis

A Preview of Statistical Reasoning

The Need for Rigorous Statistical Analysis

An Example of Sampling and Statistical Inference

Probability Experiments and Random Variables

Statistical Theory

Descriptive Statistics

Probability

Probability Distributions of Random Variables

Relationship Between Probability and Fractional Area of The Probability Distribution

The Sampling Distribution: The Most Important Concept in Statistical Inference

Deriving The Sampling Distribution: The Classical Approach

Deriving The Sampling Distribution With The Computer-Intensive Approach

Preview of Next Chapter

Chapter 5: Hypothesis Tests and Confidence Intervals

Two Types of Statistical Inference

Hypothesis Tests Versus Informal Inference

Rationale of The Hypothesis Test

Hypothesis Testing: The Mechanics

Computer-Intensive Methods for Generating The Sampling Distribution

Estimation

Chapter 6: Data-Mining Bias: The Fool’s Gold of Objective TA

Falling into The PIT: Tales of The Data-Mining BIAS

The Problem of Erroneous Knowledge in Objective Technical Analysis

Data Mining

Objective TA Research

Data Mining and Statistical Inference

Data-Mining BIAS: An Effect With Two Causes

Experimental Investigation of The Data-Mining BIAS

Solutions: Dealing With The Data-Mining BIAS

Chapter 7: Theories of Nonrandom Price Motion

The Importance of Theory

Scientific Theories

What is Wrong With Popular TA Theory?

The Enemy’s Position: Efficient Markets and Random Walks

Challenging EMH

Behavioral Finance: A Theory of Nonrandom Price Motion

Nonra NDOM Price Motion in The Context of Efficient Markets

Conclusion

Part II: Case Study: Signal Rules for the S&P 500 Index

Chapter 8: Case Study of Rule Data Mining for the S&P 500

Data Mining BIAS and Rule Evaluation

Avoidance of Data Snooping BIAS

Analyzed Data Series

Technical Analysis Themes

Performance Statistic: Average Return

No Complex Rules Were Evaluated

The Case Study Defined in Statistical Terms

Rules: Transforming Data Series Into Market Positions

Time-Series Operators

Input Series To Rules: Raw Time Series and Indicators

Table of 40 Input Series Used in Case Study

The Rules

Chapter 9: Case Study Results and the Future of TA

Presentation of Results

Critique of Case Study

Possible Case Study Extensions

The Future of Technical Analysis

Appendix: Proof That Detrending Is Equivalent to Benchmarking Based on Position Bias

Index

Founded in 1807, John Wiley & Sons is the oldest independent publishing company in the United States. With offices in North America, Europe, Australia and Asia, Wiley is globally committed to developing and marketing print and electronic products and services for our customers’ professional and personal knowledge and understanding.

The Wiley Trading series features books by traders who have survived the market’s ever changing temperament and have prospered—some by reinventing systems, others by getting back to basics. Whether a novice trader, professional or somewhere in-between, these books will provide the advice and strategies needed to prosper today and well into the future.

For a list of available titles, please visit our Web site at www.WileyFinance.com.

Copyright © 2007 by David R. Aronson. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax [phone], or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, [phone], fax [phone], or online at http://www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at [phone], outside the United States at [phone] or fax [phone].

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. For more information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data:

Aronson, David R., 1945–

Evidence-based technical analysis : applying the scientific method and statistical inference to trading signals / David R. Aronson.

p. cm.—(Wiley trading series)

Includes bibliographical references and index.

ISBN-13: 978-0-470-00874-4 (cloth)

ISBN-10: 0-470-00874-1 (cloth)

1. Investment analysis. I. Title. II. Series.

HG4529.A77 2007

332.63'2042—dc22

2006014664

To Jack and Belma

Acknowledgments

Though a book is attributed to its author(s), it truly reflects the efforts of many more people. I wish to acknowledge those individuals without whom this book would have been impossible or a much lesser work.

I am most indebted to Dr. Timothy Masters, whom I have had the pleasure of knowing for over 10 years. His patient and intelligent guidance kept me on a solid statistical footing. Tim not only gave me important feedback on technical issues but was responsible for coding and running the ATR rule experiments and the statistical routines used to test the over 6,400 rules examined. Tim also innovated the Monte Carlo permutation method as an alterative to the patented method of White, called Reality-Check, for testing the statistical significance of rules discovered by data mining. Tim has graciously decided to put the method in the public domain and has allowed it to be published for the first time here.

Also crucial were the programming talents of Stuart Okorofsky and the database creation by Dr. John Wolberg. I am indebted Dr. Halbert White, inventor of Reality-Check and for the help of Professor David Jensen, director of the Knowledge Discovery Lab at the University of Massachusetts–Amherst.

I also wish to express my appreciation to the following people for reviewing and commenting on various chapters. Their feedback was essential: Charles Neumann, Lance Rembar, Dr. Samuel Aronson, Dennis Katz, Hayes Martin, George Butler, Dr. John Wolberg, Jay Bono, Dr. Andre Shlefier, Dr. John Nofsinger, Doyle Delaney, Ken Byerly, James Kunstler, and Kenny Rome.

Special thanks to the helpful folks at John Wiley & Sons: Kevin Commins, for seeing the value of a critical appraisal of technical analysis, and Emilie Herman, for her steady hand in editing the book. Thanks as well to Michael Lisk and Laura Walsh.

About the Author

David Aronson is an adjunct professor of finance at Baruch College’s Zicklin School of Business in New York, where he teaches a graduate-level course in technical analysis to MBA and financial-engineering students, and vice-president of Hood River Research Inc., a firm that develops signal filters and predictive models. He was formerly a proprietary trader and technical analyst at Spear, Leeds and Kellogg and president of Raden Research Group Inc., a consulting firm that developed the data-mining software PRISM and filters and systems for various trading firms. Prior to that, he founded AdvoCom Corporation, which managed client funds in portfolios of futures trading advisors using portfolio optimization. He received a BA in philosophy from Lafayette College in 1967 and served in the Peace Corps in El Salvador.

Introduction

Technical analysis (TA) is the study of recurring patterns in financial market data with the intent of forecasting future price movements.1 It is comprised of numerous analysis methods, patterns, signals, indicators, and trading strategies, each with its own cheerleaders claiming that their approach works.

Much of popular or traditional TA stands where medicine stood before it evolved from a faith-based folk art into a practice based on science. Its claims are supported by colorful narratives and carefully chosen (cherry picked) anecdotes rather than objective statistical evidence.

This book’s central contention is that TA must evolve into a rigorous observational science if it is to deliver on its claims and remain relevant. The scientific method is the only rational way to extract useful knowledge from market data and the only rational approach for determining which TA methods have predictive power. I call this evidence-based technical analysis (EBTA). Grounded in objective observation and statistical inference (i.e., the scientific method), EBTA charts a course between the magical thinking and gullibility of a true believer and the relentless doubt of a random walker.

Approaching TA, or any discipline for that matter, in a scientific manner is not easy. Scientific conclusions frequently conflict with what seems intuitively obvious. To early humans it seemed obvious that the sun circled the earth. It took science to demonstrate that this intuition was wrong. An informal, intuitive approach to knowledge acquisition is especially likely to result in erroneous beliefs when phenomena are complex or highly random, two prominent features of financial market behavior. Although the scientific method is not guaranteed to extract gold from the mountains of market data, an unscientific approach is almost certain to produce fool’s gold.

This book’s second contention is that much of the wisdom comprising the popular version of TA does not qualify as legitimate knowledge.

KEY DEFINITIONS: PROPOSITIONS AND CLAIMS, BELIEF AND KNOWLEDGE

I have already used the terms knowledge and belief but have not rigorously defined them. These and several other key terms will be used repeatedly in this book, so some formal definitions are needed.

The fundamental building block of knowledge is a declarative statement, also known as a claim or a proposition. A declarative statement is one of four types of utterances that also include exclamations, questions, and commands. Declarative statements are distinguished from the others in that they have truth value. That is to say, they can be characterized as either true or false or probably true or probably false.

The statement “Oranges are on sale at the supermarket for five cents a dozen” is declarative. It makes a claim about a state of affairs existing at the local market. It may be true or false. In contrast, the exclamatory statement “Holy cow, what a deal,” the command “Go buy me a dozen,” or the question “What is an orange?” cannot be called true or false.

Our inquiry into TA will be concerned with declarative statements, such as, “Rule X has predictive power.” Our goal is to determine which of these declarative statements warrant our belief.

What does it mean to say, “I believe X.”? “With regard to states of affairs in general (i.e., ‘matters of fact’ or ‘what will happen’) believing X amounts to expecting to experience X if and when we are in a position to do so.”2 Therefore, if I believe the claim that oranges are on sale for five cents a dozen, it means that I expect to be able to buy oranges for five cents a dozen if I go to the store. However, the command to buy some oranges or the exclamation that I am happy about the opportunity, set up no such expectation.

What does all this all means for us? For any statement to even be considered as a candidate for belief, it must “assert some state of affairs that can be expected.3 Such statements are said to have cognitive content—they convey something that can be known. “If the statement contains nothing to know then there is nothing there to be believe.”4

Although all declarative statements presumably have cognitive content, not all actually do. This is not a problem if the lack of cognitive content is obvious, for example, the declaration “The square root of Tuesday is a prime number.”5 This utterance is, on its face, nonsense. There are other declarative statements, however, whose lack of cognitive content is not so obvious. This can be a problem, because such statements can fool us into thinking that a claim has been made that sets up an expectation, when, in fact, no claim has really been put forward. These pseudo-declarative-statements are essentially meaningless claims or empty propositions.

Although meaningless claims are not valid candidates for belief, this does not stop many people from believing in them. The vague predictions made in the daily astrology column or the nebulous promises made by promoters of bogus health cures are examples of meaningless claims. Those who believe these empty propositions simply do not realize that what they have been told has no cognitive content.

A way to tell if a statement has cognitive content and is, thus, a valid candidate for belief is the discernible-difference test6 described by Hall. “Utterances with cognitive content make claims that are either true or false; and whether they are true or false makes a difference that can be discerned. That is why these utterances offer something to believe and why there is no point in trying to believe an utterance that makes no such offer”7 In other words, a proposition that passes the discernible-difference test sets up an expectation such that the state of affairs, if the statement were true, is recognizably different from the state of affairs, if the statement were false.

The discernible-difference criterion can be applied to statements purporting to be predictions. A prediction is a claim to know something about the future. If a prediction has cognitive content, it will be clearly discernible in the outcome if the prediction was accurate or not. Many, if not most, of the forecasts issued by practitioners of popular TA are devoid of cognitive content on these grounds. In other words, the predictions are typically too vague to ever determine if they were wrong.

The truth or falsity of the claim oranges are on sale for five cents a dozen will make a discernible difference when I get to the market. It is this discernible difference that allows the claim to be tested. As will be described in Chapter 3, testing a claim on the basis of a discernible difference is central to the scientific method.

Hall, in his book Practically Profound, explains why he finds Freudian psychoanalysis to be meaningless when examined in light of the discernible-difference test.

“Certain Freudian claims about human sexual development are compatible with all possible states of affairs. There is no way to confirm or disconfirm either ‘penis envy’ or ‘castration complex’ because there is no distinguishable difference between evidence affirming and evidence denying these interpretations of behavior. Exactly opposite behaviors are equally predictable, depending on whether the alleged psychosexual stress is overt or repressed.” The requirement of “cognitive content rules out all utterances that are so loose, poorly formed or obsessively held (e.g., conspiracy theories) that there is no recognizable difference between what would be the case if they were so, and what would be the case if they were not.”8 In a like vein, the Intelligent Design Theory carries no cognitive freight in the sense that no matter what life form is observed it is consistent with the notion that it manifests an underlying form specified by some intelligent designer.9

What then is knowledge? Knowledge can be defined as justified true belief. Hence, in order for a declarative statement to qualify as knowledge, not only must it be a candidate for belief, because it has cognitive content, but it must meet two other conditions as well. First, it must be true (or probably true). Second, the statement must be believed with justification. A belief is justified when it is based on sound inferences from solid evidence.

Prehistoric humans held the false belief that the sun moved across the sky because the sun orbited the earth. Clearly they were not in possession of knowledge, but suppose that there was a prehistoric person who believed correctly that the sun moved across the sky because of the earth’s rotation. Although this belief was true, this individual could not be described as possessing knowledge. Even though they believed what astronomers ultimately proved to be true, there was no evidence yet to justify that belief. Without justification, a true belief does not attain the status of knowledge. These concepts are illustrated in Figure I.1.

FIGURE I.1 Knowledge: justified true belief.

From this it follows that erroneous beliefs or false knowledge fail to meet one or more of the necessary conditions of knowledge. Thus, an erroneous belief can arise either because it concerns a meaningless claim or because it concerns a claim that, though meaningful, is not justified by valid inferences from solid evidence.

Still, even when we have done everything right, by drawing the best possible inference from sound evidence, we can still wind up adopting erroneous beliefs. In other words, we can be justified in believing a falsehood, and honestly claim to know something, if it appears to be true according to logically sound inferences from the preponderance of available evidence. “We are entitled to say ‘I know’ when the target of that claim is supported beyond reasonable doubt in the network of well-tested evidence. But that is not enough to guarantee that we do know.”10

Falsehoods are an unavoidable fact of life when we attempt to know things about the world based on observed evidence. Thus, knowledge based on the scientific method is inherently uncertain, and provisional, though less uncertain than knowledge acquired by less formal methods. However, over time, scientific knowledge improves, as it comes to describe reality in a progressively more accurate manner. It is a continual work in progress. The goal of EBTA is a body of knowledge about market behavior that is as good as can be had, given the limits of evidence gathering and the powers of inference.

ERRONEOUS TA KNOWLEDGE: THE COST OF UNDISCIPLINED ANALYSIS

To understand why the knowledge produced by the popular version of TA is untrustworthy, we must consider two distinct forms of TA: subjective and objective. Both approaches can lead to erroneous beliefs, but they do so in distinct ways.

Objective TA methods are well defined repeatable procedures that issue unambiguous signals. This allows them to be implemented as computerized algorithms and back-tested on historical data. Results produced by a back test can be evaluated in a rigorous quantitative manner.

Subjective TA methods are not well-defined analysis procedures. Because of their vagueness, an analyst’s private interpretations are required. This thwarts computerization, back testing, and objective performance evaluation. In other words, it is impossible to either confirm or deny a subjective method’s efficacy. For this reason they are insulated from evidentiary challenge.

From the standpoint of EBTA, subjective methods are the most problematic. They are essentially meaningless claims that give the illusion of conveying cognitive content. Because the methods do not specify how they are to be applied, different analysts applying it to the same set of market data can reach different conclusions. This makes it impossible to determine if the method provides useful predictions. Classical chart pattern analysis,11 hand-drawn trend lines, Elliott Wave Principle,12 Gann patterns, Magic T’s and numerous other subjective methods fall into this category.13 Subjective TA is religion—it is based on faith. No amount of cherry-picked examples showing where the method succeeded can cure this deficiency.

Despite their lack of cognitive content and the impossibility of ever being supported by sound evidence, there is no shortage of fervent believers in various subjective methods. Chapter 2 explains how flaws in human thinking can produce strong beliefs in the absence of evidence or even in the face of contradictory evidence.

Objective TA can also spawn erroneous beliefs but they come about differently. They are traceable to faulty inferences from objective evidence. The mere fact that an objective method has been profitable in a back test is not sufficient grounds for concluding that it has merit. Past performance can fool us. Historical success is a necessary but not a sufficient condition for concluding that a method has predictive power and, therefore, is likely to be profitable in the future. Favorable past performance can occur by luck or because of an upward bias produced by one form of back testing called data mining. Determining when back-test profits are attributable to a good method rather than good luck is a question that can only be answered by rigorous statistical inference. This is discussed in Chapters 4 and 5. Chapter 6 considers the problem of data-mining bias. Although I will assert that data mining, when done correctly, is the modern technician’s best method for knowledge discovery, specialized statistical tests must be applied to the results obtained with data mining.

HOW EBTA IS DIFFERENT

What sets EBTA apart from the popular form of TA? First, it is restricted to meaningful claims—objective methods that can be tested on historical data. Second, it utilizes advanced forms of statistical inference to determine if a profitable back test is indicative of an effective method. Thus, the prime focus of EBTA is determining which objective methods are worthy of actual use.

EBTA rejects all forms of subjective TA. Subjective TA is not even wrong. It is worse than wrong. Statements that can be qualified as wrong (untrue) at least convey cognitive content that can be tested. The propositions of subjective TA offer no such thing. Though, at first blush, they seem to convey knowledge, when they are examined critically, it becomes clear they are empty claims.

Promoters of New Age health cures excel at empty claims. They tell you that wearing their magic copper bracelet will make you will feel better and put more bounce in your step. They suggest your golf game will improve and maybe even your love life. However, the claim’s lack of specificity makes it impossible to nail down exactly what is being promised or how it can be tested. Such claims can never be confirmed or contradicted with objective evidence. On these same grounds, it can be said that the propositions of subjective TA are empty and thus insulated from empirical challenge. They must be taken on faith.

In contrast, a meaningful claim is testable because it makes measurable promises. It states specifically how much your golf game will improve or how bouncy your steps will be. This specificity opens the claim to being contradicted with empirical evidence.

From the perspective of EBTA, proponents of subjective methods are faced with a choice: They can reformulate the method to be objective, as one practitioner of the Elliott Wave Principle has done,14 thus exposing it to empirical refutation, or they must admit the method must be accepted on faith. Perhaps Gann lines actually provide useful information. In their present form, we are denied this knowledge.

With respect to objective TA, EBTA does not take profitable back tests at face value. Instead, they are subjected to rigorous statistical evaluation to determine if profits were due to luck or biased research. As will be pointed out in Chapter 6, in many instances, profitable back tests may be a data miner fool’s gold. This may explain why many objective TA methods that perform well in a back testing perform worse when applied to new data. Evidence-based technical analysis uses computer-intensive statistical methods that minimize problems stemming from the data-mining bias.

The evolution of TA to EBTA also has ethical implications. It is the ethical and legal responsibility of all analysts, whatever form of analysis they practice, to make recommendations that have a reasonable basis and not to make unwarranted claims.15 The only reasonable basis for asserting an analysis method has value is objective evidence. Subjective TA methods cannot meet this standard. Objective TA, conducted in accordance with the standards of EBTA can.

EBTA RESULTS FROM ACADEMIA

Evidence-based technical analysis is not a new idea. Over the past two decades, numerous articles in respected academic journals16 have approached TA in the rigorous manner advocated by this book.17 The evidence is not uniform. Some studies show TA does not work, but some show that it does. Because each study is confined to a particular aspect of TA and a specific body of data, it is possible for studies to reach different conclusions. This is often the case in science.

The following are a few of the findings from academic TA. It shows that, when approached in a rigorous and intellectually honest manner, TA is a worthwhile area of study.

Expert chartists are unable to distinguish actual price charts of stocks from charts produced by a random process.18There is empirical evidence of trends in commodities19 and foreign exchange markets that can be exploited with the simple objective trend indicators. In addition, the profits earned by trend-following speculators may be justified by economic theory20 because their activities provide commercial hedgers with a valuable economic service, the transference of price risk from hedger to speculator.Simple technical rules used individually and in combinations can yield statistically and economically significant profits when applied to stock market averages composed of relatively young companies (Russell 2000 and NASDAQ Composite).21Neural networks have been able to combine buy/sell signals of simple moving-average rules into nonlinear models that displayed good predictive performance on the Dow Jones Average over the period 1897 to 1988.22Trends in industry groups and sectors persist long enough after detection by simple momentum indicators to earn excess returns.23Stocks that have displayed prior relative strength and relative weakness continue to display above-average and below-average performance over horizons of 3 to 12 months.24United States stocks, selling near their 52-week highs, outperform other stocks. An indicator defined as the differential between a stock’s current price and its 52-week high is a useful predictor of future relative performance.25 The indicator is an even more potent predictor for Australian stocks.26The head-and-shoulders chart pattern has limited forecasting power when tested in an objective fashion in currencies. Better results can be had with simple filter rules. The head-and-shoulders pattern, when tested objectively on stocks, does not provide useful information.27 Traders who act on such signals would be equally served by following a random signal.Trading volume statistics for stocks contain useful predictive information28 and improve the profitability of signals based on large price changes following a public announcement.29Computer-intensive data-modeling neural networks, genetic algorithms, and other statistical learning and artificial-intelligence methods have found profitable patterns in technical indicators.30

WHO AM I TO CRITICIZE TA?

My interest in TA began in 1960 at the age of 15. During my high-school and college years I followed a large stable of stocks using the Chartcraft point and figure method. I have used TA professionally since 1973, first as a stock broker, then as managing partner of a small software company, Raden Research Group Inc.—an early adopter of machine learning and data mining in financial market applications—and finally as a proprietary equities trader for Spear, Leeds & Kellogg.31 In 1988, I earned the Chartered Market Technician designation from the Market Technicians Association. My personal TA library has over 300 books. I have published approximately a dozen articles and have spoken numerous times on the subject. Currently I teach a graduate-level course in TA at the Zicklin School of Business, Baruch College, City University of New York. I freely admit my previous writings and research do not meet EBTA standards, in particular with regard to statistical significance and the data-mining bias.

My long-standing faith in TA began to erode in response to a very mediocre performance over a five-year period trading capital for Spear, Leeds and Kellogg. How could what I believed in so fervently not work? Was it me or something to do with TA in general? My academic training in philosophy provided fertile grounds for my growing doubts. My concerns crystallized into full fledged skepticism as a result of reading two books: How We Know What Isn’t So by Thomas Gilovich and Why People Believe Weird Things, by Michael Shermer. My conclusion: Technical analysts, including myself, know a lot of stuff that isn’t so, and believe a lot of weird things.

TECHNICAL ANALYSIS: ART, SCIENCE, OR SUPERSTITION?

There is a debate in the TA community: Is it an art or a science? The question has been framed incorrectly. It is more properly stated as: Should TA be based on superstition or science? Framed this way the debate evaporates.

Some will say TA involves too much nuance and interpretation to render its knowledge in the form of scientifically testable claims. To this I retort: TA that is not testable may sound like knowledge, but it is not. It is superstition that belongs in the realm of astrology, numerology, and other nonscientific practices.

Creativity and inspiration play a crucial role in science. They will be important in EBTA as well. All scientific inquiries start with a hypothesis, a new idea or a new insight inspired by a mysterious mixture of prior knowledge, experience and a leap of intuition. Yet, good science balances creativity with analytical rigor. The freedom to propose new ideas must be married to an unyielding discipline that eliminates ideas that prove worthless in the crucible of objective testing. Without this anchor to reality, people fall in love with their ideas, and magical thinking replaces critical thought.

It is unlikely that TA will ever discover rules that predict with the precision of the laws of physics. The inherent complexity and randomness of financial markets and the impossibility of controlled experimentation preclude such findings. However, predictive accuracy is not the defining requirement of science. Rather, it is defined by an uncompromising openness to recognizing and eliminating wrong ideas.

I have four hopes for this book: First, that it will stimulate a dialogue amongst technical analysts that will ultimately put our field on a firmer intellectual foundation; second, that it will encourage further research along the lines advocated herein; third, that it will encourage consumers of TA to demand more “beef” from those who sell products and services based upon TA; and fourth, that it will encourage TA practitioners, professional and otherwise, to understand their crucial role in a human-machine partnership that has the potential to accelerate the growth of legitimate TA knowledge.

No doubt some fellow practitioners of TA will be irritated by these ideas. This can be a good thing. An oyster irritated by a grain of sand sometimes yields a pearl. I invite my colleagues to expend their energies adding to legitimate knowledge rather than defending the indefensible.

This book is organized in two sections. Part One establishes the methodological, philosophical, psychological, and statistical foundations of EBTA. Part Two demonstrates one approach to EBTA: testing of 6,402 binary buy/sell rules on the S&P 500 on 25 years of historical data. The rules are evaluated for statistical significance using tests designed to cope with the problem of data-mining bias.

1. Data typically considered by TA includes prices of financial instruments; trading volume; open interest, in the case of options and futures; as well as other measures that reflect the attitudes and behavior of market participants.

2. J. Hall, Practically Profound: Putting Philosophy to Work in Everyday Life (Lanham, MD: Rowman & Littlefield Publishers, 2005).

3. Ibid., 4.

4. Ibid., 4.

5. Ibid., 5.

6. Ibid., 5.

7. Ibid., 5.

8. Ibid., 6.

9. Ibid., 5.

10. Ibid., 81.

11. R.D. Edwards and J. Magee, Technical Analysis of Stock Trends, 4th ed. (Springfield, MA: John Magee, 1958).

12. For a complete description of Elliott wave theory see R.R. Prechter and A.J. Frost, Elliott Wave Principle (New York: New Classics Library, 1998).

13. Any version of these methods that has been made objective to the point where it is back testable would negate this criticism.

14. The professional association of technical analysts, the Market Technicians Association (MTA), requires compliance with the National Association of Securities Dealers and the New York Stock Exchange. These self-regulating bodies require “that research reports have a reasonable basis and no unwarranted claims.” Going even further, the MTA requires of its members that they “shall not publish or make statements concerning the technical position of a security, a market or any of its components or aspects unless such statements are reasonable and consistent in light of the available evidence and the accumulated knowledge in the field of technical analysis.”

15. Some peer-reviewed academic journals include Journal of Finance, Financial Management Journal, Journal of Financial Economics, Journal of Financial and Quantitative Analysis, and Review of Financial Studies.

16. Outside of academia, there has been a move to greater emphasis on objective methods of TA, but often the results are not evaluated in a statistically rigorous manner.

17. F.D. Arditti, “Can Analysts Distinguish Between Real and Randomly Generated Stock Prices?,” Financial Analysts Journal 34, no. 6 (November/December 1978), 70.

18. J.J. Siegel, Stocks for the Long Run, 2nd ed. (New York: McGraw-Hill, 1998), 243.

19. G.R. Jensen, R.R. Johnson, and J.M. Mercer, “Tactical Asset Allocation and Commodity Futures: Ways to Improve Performance,” Journal of Portfolio Management 28, no. 4 (Summer 2002).

20. C.R. Lightner, “A Rationale for Managed Futures,” Technical Analysis of Stocks & Commodities (2003). Note that this publication is not a peer-reviewed journal but the article appeared to be well supported and its findings were consistent with the peer-reviewed article cited in the prior note.

21. P.-H. Hsu and C.-M. Kuan, “Reexamining the Profitability of Technical Analysis with Data Snooping Checks,” Journal of Financial Economics 3, no. 4 (2005), 606–628.

22. R. Gency, “The Predictability of Security Returns with Simple Technical Trading Rules,” Journal of Empirical Finance 5 (1998), 347–349.

23. N. Jegadeesh, “Evidence of Predictable Behavior of Security Returns,” Journal of Finance 45 (1990), 881–898.

24. N. Jegadeesh and S. Titman, “Returns to Buying Winners and Selling Losers: Implications for Stock Market Efficiency,” Journal of Finance 48 (1993), 65–91.

25. T.J. George and C.-Y. Hwang, “The 52-Week High and Momentum Investing,” Journal of Finance 59, no. 5 (October 2004), 2145–2184.

26. B.R. Marshall and R. Hodges, “Is the 52-Week High Momentum Strategy Profitable Outside the U.S.?” awaiting publication in Applied Financial Economics.

27. C.L. Osler, “Identifying Noise Traders: The Head and Shoulders Pattern in U.S. Equities,” Staff Reports, Federal Reserve Bank of New York 42 (July 1998), 39 pages.

28. L. Blume and D. Easley, “Market Statistics and Technical Analysis: The Role of Volume,” Journal of Finance 49, no. 1 (March 1994), 153–182.

29. V. Singal, Beyond the Random Walk: A Guide of Stock Market Anomalies and Low-Risk Investing (New York: Oxford University Press, 2004). These results are discussed in Chapter 4, “Short Term Price Drift.” The chapter also contains an excellent list of references of other research relating to this topic.

30. A.M. Safer, “A Comparison of Two Data Mining Techniques to Predict Abnormal Stock Market Returns,” Intelligent Data Analysis 7, no. 1 (2003), 3–14; G. Armano, A. Murru, and F. Roli, “Stock Market Prediction by a Mixture of Genetic-Neural Experts,” International Journal of Pattern Recognition & Artificial Intelligence 16, no. 5 (August 2002), 501–528; G. Armano, M. Marchesi, and A. Murru, “A Hybrid Genetic-Neural Architecture for Stock Indexes Forecasting,” Information Sciences 170, no. 1 (February 2005), 3–33; T. Chenoweth, Z.O. Sauchi, and S. Lee, “Embedding Technical Analysis into Neural Network Based Trading Systems,” Applied Artificial Intelligence 10, no. 6 (December 1996), 523–542; S. Thawornwong, D. Enke, and C. Dagli, “Neural Networks as a Decision Maker for Stock Trading: A Technical Analysis Approach,” International Journal of Smart Engineering System Design 5, no. 4 (October/December 2003), 313–325; A.M. Safer, “The Application of Neural-Networks to Predict Abnormal Stock Returns Using Insider Trading Data,” Applied Stochastic Models in Business & Industry 18, no. 4 (October 2002), 380–390; J. Yao, C.L. Tan, and H.-L. Pho, “Neural Networks for Technical Analysis: A Study on KLCI,” International Journal of Theoretical & Applied Finance 2, no. 2 (April 1999), 221–242; J. Korczak and P. Rogers, “Stock Timing Using Genetic Algorithms,” Applied Stochastic Models in Business & Industry 18, no. 2 (April 2002), 121–135; Z. Xu-Shen and M. Dong, “Can Fuzzy Logic Make Technical Analysis 20/20?,” Financial Analysts Journal 60, no. 4 (July/August 2004), 54–75; J.M. Gorriz, C.G. Puntonet, M. Salmeron, and J.J. De la Rosa, “A New Model for Time-Series Forecasting Using Radial Basis Functions and Exogenous Data,” Neural Computing & Applications 13, no. 2 (2004), 100–111.

31. This firm was acquired by Goldman Sachs in September 2000.

PART I

Methodological, Psychological, Philosophical, and Statistical Foundations

CHAPTER 1

Objective Rules and Their Evaluation

This chapter introduces the notion of objective binary signaling rules and a methodology for their rigorous evaluation. It defines an evaluation benchmark based on the profitability of a noninformative signal. It also establishes the need to detrend market data so that the performances of rules with different long/short position biases can be compared.

THE GREAT DIVIDE: OBJECTIVE VERSUS SUBJECTIVE TECHNICAL ANALYSIS

Technical analysis (TA) divides into two broad categories: objective and subjective. Subjective TA is comprised of analysis methods and patterns that are not precisely defined. As a consequence, a conclusion derived from a subjective method reflects the private interpretations of the analyst applying the method. This creates the possibility that two analysts applying the same method to the same set of market data may arrive at entirely different conclusions. Therefore, subjective methods are untestable, and claims that they are effective are exempt from empirical challenge. This is fertile ground for myths to flourish.

In contrast, objective methods are clearly defined. When an objective analysis method is applied to market data, its signals or predictions are unambiguous. This makes it possible to simulate the method on historical data and determine its precise level of performance. This is called back testing. The back testing of an objective method is, therefore, a repeatable experiment which allows claims of profitability to be tested and possibly refuted with statistical evidence. This makes it possible to find out which objective methods are effective and which are not.

The acid test for distinguishing an objective from a subjective method is the programmability criterion: A method is objective if and only if it can be implemented as a computer program that produces unambiguous market positions (long,1short,2or neutral3). All methods that cannot be reduced to such a program are, by default, subjective.

TA RULES

Objective TA methods are also referred to as mechanical trading rules or trading systems. In this book, all objective TA methods are referred to simply as rules.

A rule is a function that transforms one or more items of information, referred to as the rule’s input, into the rule’s output, which is a recommended market position (e.g., long, short, neutral). Input(s) consists of one or more financial market time series. The rule is defined by one or more mathematical and logical operators that convert the input time series into a new time series that consists of the sequence of recommended market position (long, short, out-of-the-market). The output is typically represented by a signed number (e.g., +1 or −1). This book adopts the convention of assigning positive values to indicate long positions and negative values to indicate shorts position. The process by which a rule transforms one or more input series into an output series is illustrated in .