40,99 €
Mehr erfahren.
- Herausgeber: John Wiley & Sons
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Experiments are the most effective way to learn about the world. By cleverly interfering with something to see how it reacts we are able to find out how it works. In contrast to passive observation, experimenting provides us with data relevant to our research and thus less time and effort is spent separating relevant from irrelevant information.
The art of experimentation is often learnt by doing, so an intuitive understanding of the experimental method usually evolves gradually through years of trial and error. This book speeds up the journey for the reader to becoming a proficient experimenter.
Organized in two parts, this unique text begins by providing a general introduction to the scientific approach to experimentation. It then describes the processes and tools required, including the relevant statistical and experimental methods. Towards the end of the book a methodology is presented, which leads the reader through the three phases of an experiment: ‘Planning’, ‘Data Collection’, and ‘Analysis and Synthesis’.
Experiment!
- Provides an excellent introduction to the methodology and implementation of experimentation in the natural, engineering and medical sciences
- Puts practical tools into scientific context
- Features a number of selected actual experiments to explore what are the key characteristics of good experiments
- Includes examples and exercises in every chapter
This book focuses on general research skills, such as adopting a scientific mindset, learning how to plan meaningful experiments and understanding the fundamentals of collecting and interpreting data. It is directed to anyone engaged in experiments, especially Ph.D. and masters students just starting to create and develop their own experiments.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 656
Veröffentlichungsjahr: 2012
Ähnliche
Contents
Cover
Title Page
Copyright
Dedication
Preface
Part One: Understanding the World
Chapter 1: You, the Discoverer
1.1 Venturing into the Unknown
1.2 Embarking on a Ph.D.
1.3 The Art of Discovery
1.4 About this Book
1.5 How to Use this Book
Further Reading
Chapter 2: What is Science?
2.1 Characteristics of the Scientific Approach
2.2 The Inductive Method
2.3 The Hypothetico-Deductive Method
2.4 Consequences of Falsification
2.5 The Role of Confirmation
2.6 Perception is Personal
2.7 The Scientific Community
2.8 Summary
Further Reading
Chapter 3: Science's Childhood
3.1 Infancy
3.2 Ionian Dawn
3.3 Divine Mathematics
3.4 Adolescence – Revolution!
3.5 The Children of the Revolution
3.6 Summary
Further Reading
Chapter 4: Science Inclined to Experiment
4.1 Galileo's Important Experiment
4.2 Experiment or Hoax?
4.3 Reconstructing the Experiment
4.4 Getting the Swing of Things
4.5 The Message from the Plane
4.6 Summary
Chapter 5: Scientists, Engineers and Other Poets
5.1 Research and Development
5.2 Characteristics of Research
5.3 Building Theories
5.4 The Relationship between Theory and Reality
5.5 Creativity
5.6 Summary
Further Reading
Part Two: Interfering with theWorld
Chapter 6: Experiment!
6.1 What is an Experiment?
6.2 Questions, Answers and Experiments
6.3 A Gallery of Experiments
6.4 Reflections on the Exhibition
6.5 Summary
Further Reading
Chapter 7: Basic Statistics
7.1 The Role of Statistics in Data Analysis
7.2 Populations and Samples
7.3 Descriptive Statistics
7.4 Probability Distribution
7.5 The Central Limit Effect
7.6 Normal Probability Plots
7.7 Confidence Intervals
7.8 The t-Distribution
7.9 Summary
Further Reading
Chapter 8: Statistics for Experiments
8.1 A Teatime Experiment
8.2 The Importance of Randomization
8.3 One-Sided and Two-Sided Tests
8.4 The t-Test for One Sample
8.5 The Power of a Test
8.6 Comparing Two Samples
8.7 Analysis of Variance (ANOVA)
8.8 A Measurement System Analysis
8.9 Other Useful Hypothesis Tests
8.10 Interpreting p-Values
8.11 Correlation
8.12 Regression Modeling
8.13 Summary
Further Reading
Chapter 9: Experimental Design
9.1 Statistics and the Scientific Method
9.2 Designs with One Categorical Factor
9.3 Several Categorical Factors: the Full Factorial Design
9.4 Are Interactions Important?
9.5 Factor Screening: Fractional Factorial Designs
9.6 Determining the Confounding Pattern
9.7 Design Resolution
9.8 Working with Screening Designs
9.9 Continuous Factors: Regression and Response Surface Methods
9.10 Summary
Further Reading
Chapter 10: Phase I: Planning
10.1 The Three Phases of Research
10.2 Experiment 1: Visual Orientation in a Beetle
10.3 Experiment 2: Lift-Off Length in a Diesel Engine
10.4 Finding Out What is Not Known
10.5 Determining the Scope
10.6 Tools for Generating Hypotheses
10.7 Thought Experiments
10.8 Planning Checklist
10.9 Summary
Chapter 11: Phase II: Data Collection
11.1 Generating Understanding from Data
11.2 Measurement Uncertainty
11.3 Developing a Measurement System
11.4 Measurement System Analysis
11.5 The Data Collection Plan
11.6 Summary
Further Reading
Chapter 12: Phase III: Analysis and Synthesis
12.1 Turning Data into Information
12.2 Graphical Analysis
12.3 Mathematical Analysis
12.4 Writing a Scientific Paper
12.5 Writing a Ph.D. Thesis
12.6 Farewell
12.7 Summary
Further Reading
Appendix
Standard Normal Probabilities
Probability Points for the t-Distribution
Index
This edition first published 2012
© 2012 John Wiley & Sons, Ltd
Registered office
John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom
For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our web site at www.wiley.com.
The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988.
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher.
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.
Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought.
The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of fitness for a particular purpose. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for every situation. In view of ongoing research, equipment modifications, changes in governmental regulations, and the constant flow of information relating to the use of experimental reagents, equipment, and devices, the reader is urged to review and evaluate the information provided in the package insert or instructions for each chemical, piece of equipment, reagent, or device for, among other things, any changes in the instructions or indication of usage and for added warnings and precautions. The fact that an organization or web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or Website may provide or recommendations it may make. Further, readers should be aware that Internet web sites listed in this work may have changed or disappeared between when this work was written and when it is read. No warranty may be created or extended by any promotional statements for this work. Neither the publisher nor the author shall be liable for any damages arising herefrom.
Library of Congress Cataloging-in-Publication Data
Andersson, Öivind, 1970– Experiment! : planning, implementing, and interpreting / Öivind Andersson. p. cm. ISBN 978-0-470-68826-7 (cloth) – ISBN 978-0-470-68825-0 (pbk.) 1. Research. 2. Experimental design 3. Science–Methodology. I. Title. Q180.A1A625 2012 001.4′34–dc23 2012008855
A catalogue record for this book is available from the British Library.
Print ISBN: Cloth: 9780470688267 Paper: 9780470688250
Instructors can access PowerPoint files of the illustrations presented within this text, for teaching, at: http://booksupport.wiley.com
For Carl-Johan,
an avid experimenter
Preface
Today, externally funded research is the rule rather than the exception. The problem statements and hypotheses of a research project have already been formulated by a senior researcher when filing the funding application. When the research student enters the picture a project plan has already been made. All that remains is to produce results in the laboratory. This situation enhances the impression that many students have, that experiments are mainly about taking measurements – a belief arising during traditional, structured laboratory classes that are based on demonstrations rather than real experiments.
Experienced experimenters know that the main efforts in a successful experiment lie in meticulous planning and analysis, and that both of these must be firmly rooted in a scientific approach. Entering the research process at such a late stage, students may be delayed in realizing this crucial point. The art of experimentation is often learned by doing, so an intuitive understanding of the experimental method usually evolves gradually during years of trial and error. The aim of this book is to speed up the journey.
Research education is often practically oriented, focusing on techniques and concepts that are used in specific research areas. In order to become an independent researcher it is important to also acquire more general research competencies. Research students must adopt a scientific mindset, learn how to plan meaningful experiments, and understand the fundamentals of collecting and interpreting data. This book focuses on these general research skills. It is directed to anyone engaged in experiments, and especially Ph.D. and master’s students just starting to develop their own experiments.
There are several books that discuss the scientific method from a philosophical standpoint. There are also good books covering research methods from a statistical point of view. Why do we need another book on scientific method? The reason is that scientists need to employ several types of research competency in parallel, and it is unfortunate that these tend to be taught separately. The philosophy of science is frequently introduced using historical examples in courses where more time is spent criticizing ideas than explaining how to apply them. It is too common to teach statistics as a pure study of numbers without explaining how numbers may be turned into meaningful scientific knowledge. The lack of a context relevant to working scientists makes it difficult for research students to incorporate such knowledge into their other research competencies. This book aims to merge these aspects of research into a practically workable package.
The book is organized in two parts. The first part gives a general introduction to the scientific approach while the second describes general methods and tools with a focus on experiments. Towards the end of the book a methodology is presented, which leads the reader through the three phases of an experiment: “Planning”, “Data Collection”, and “Analysis and Synthesis”. The first phase puts the discussion about scientific and experimental method into a practical context, treating how research problems may be identified and how to approach them by conceiving meaningful experiments. The following two phases continue to build on these ideas, but incorporate statistical techniques into the process. In the data collection phase, a measurement system is devised, analyzed and improved to obtain data of sufficient quality. In the last phase, the raw data are turned into meaningful information, analyzed with graphical and mathematical tools, and connected back to the research question.
It is my experience that this methodology helps students connect the somewhat abstract concepts from statistical theory and the philosophy of science to their scientific praxis. I am not an advocate of cookbook recipes for research and the methodology is not to be viewed that way. The idea is rather that readers, by applying these simple tools to their own research problems, will reflect on and learn to understand key elements of the experimental method.
Many people have kindly helped me in the preparation of this book. My friend and colleague Rolf Egnell deserves a very special thank you for taking the time to read and comment on the manuscript in its entirety. He has had the admirable ability to look beyond the depths of his knowledge and read the text with the eyes of a layman. His valuable comments have significantly improved the readability of this text. I am also grateful to Tim Davis for his comments on the statistics material, and to Staffan Ulfstrand for providing feedback on parts where evolutionary biology is discussed. Advice and help of various kinds were willingly given by Malte Andersson, Marie Dacke, Leif Lönnblad, Carl-Erik Magnusson, Clément Chartier, Johan Zetterberg, and Tony Greenfield. I also want to thank Sarah Tilley at Wiley in Chichester, UK, for her support and encouraging remarks along the way. I gratefully acknowledge these people while stressing that I am, of course, solely responsible for any mistakes or other authorial shortcoming still present in the text.
When writing this, others come to mind that have helped me over the years. Many have had a hand in teaching me to see the world around me and to think independently, but space prevents me from mentioning them all. I especially owe thanks to the teachers and staff at the Departments of Physics and Theoretical Physics in Lund, who always presented scientific ideas with a perfect blend of playfulness and intellectual sharpness. I am also grateful to my mother for her unconditional support in all life’s endeavors. As a child she instilled in me that I succeeded with everything I tried my hand at, even though I often did not.
Above all, my deepest and most heartfelt gratitude goes to Gunnel, the love of my life, and Carl-Johan, our son, for their sympathy, patience and laughter. You are the twin star around which my world blissfully revolves.
Part One
Understanding the World
1
You, the Discoverer
All truths are easy to understand once they are discovered; the point is to discover them.
—Galileo Galilei
First of all, I would like to congratulate you. The reason that you are reading this book must be that you are learning about scientific method. This should open up possibilities for a variety of interesting professional tasks in the future. Scientific method can be said to be an approach for breaking a complex problem down into its essential components, investigating these components through relevant data and critical thinking, and finally solving the main problem by putting the components back together. Being familiar with scientific method makes it easier to handle complex problems, wherever we encounter them. This is of course useful in various branches of scientific research, where the method was developed, but complex problems occur in many situations. It does not harm policy makers, managers, or technical experts, for example, to be skilled critical thinkers, creative problem solvers, and to be able to draw relevant conclusions from data.
As conveyed by its title, this book is written for experimenters. Experiments are made in a wide variety of research and development tasks, to test ideas and to find out how things work. The book grew out of a need to teach research students general research skills. This means that it is mainly directed towards Ph.D. students, but it is likely also that others working with research and development will find it useful. Having worked with technical development myself, I know that planning, conducting and interpreting experiments are crucial for engineers too. I hope that no one will feel excluded by the way the text addresses research students because most of the contents are very generally applicable. Most readers will probably be found in various parts of the natural, engineering and medical sciences. The reason why I think that the book is relevant across so many disciplines is that the experimental method is such a general approach to finding things out.
Experiments are our most efficient method of discovery. They are based on interfering with something to see how it reacts and find out how it works. The alternative to this is passive observation – standing back, watching and waiting for something interesting to happen. Passive observers obtain large amounts of data, but it may be difficult for them to isolate the information that is relevant to a particular research problem. Experimentation, on the other hand, provides us with data that are tailored to our problem. This means that experimenters have to spend less effort sorting out irrelevant information.
1.1 Venturing into the Unknown
Research studies are a voyage into the unknown in at least two respects. Firstly, as a Ph.D. student you will often find yourself at the borders of our current knowledge, spying out into the uncharted territories beyond. Secondly, research studies involve learning in ways that are unfamiliar compared to the previous stages of your education. As will be explained in later sections, this is because research studies are focused on acquiring skills in addition to knowledge.
Science is probably the most remarkable enterprise that humankind has ever undertaken. During the couple of hundred thousand years that our species has existed, almost all the knowledge we have about the world we live in has been established during the last couple of hundred years – a mere thousandth of our history. We have been intelligent and curious for eons but only learned to understand Nature once science was invented. This says something of how powerful this idea is. Besides the knowledge it has brought with it, science has also fundamentally changed our living conditions. Scientific progress has found applications in technology and medicine, and these have opened up new avenues of applied research. Science is so integrated in the progress of society that, today, it is all but impossible to imagine an existence without it. This is probably one reason that science is held in such high esteem and that scientists have such high credibility among people.
I would like to clarify that the word science, in this book, refers to the natural sciences in a broad sense. This is not to say that other branches of research are of lesser value, but the methods of science differ between disciplines and the definition has simply been chosen to fit the audience. Many methods of natural science are shared between the natural, engineering and medical sciences. The word “science” itself has Latin roots and originally means knowledge, but it is generally used in a wider sense to include both scientific research and the knowledge that results from it.
With a bit of poetic license you could say that science is a state of mind. Of course, it consists of various facts and methods, but it is also an attitude to the world around us. Scientists act from the presumption that the world is understandable. They try to look beyond the apparent disorder and search for patterns. From these they identify the underlying, general rules that govern the world. This attitude is the basis of scientific discovery.
Do not be fooled into believing that there are no more discoveries to be made. Even relatively simple questions may direct us to fascinating insights about the world. Have you ever wondered why the sky is blue or why the grass is green? The blue sky is now well understood, thanks to the nineteenth century discovery of a phenomenon called Rayleigh scattering. We will return to it in Chapter 10, in connection with a nice experiment about how an African dung beetle finds its way at twilight. The color of the grass, on the other hand, is not so well understood. There is, of course, the obvious reason that grass contains chlorophyll, which absorbs sunlight for the photosynthesis. But it seems odd that chlorophyll should be green because this means that it reflects light in the green wavelength region, where the sun's radiation contains most energy. A black plant, absorbing all visible wavelengths, would extract more energy from the sunlight than a green one. Why do plants reject the sun's energy where it is most abundant? When asking biologists about this, they usually embark on long, interesting discussions about photosynthesis in various organisms before wrapping it up with “So, it's really a good question!” The fact seems to be that we simply do not know.
When you start doing research you will find that there are plenty of unanswered questions in your field as well. Towards the end of this book we will discuss how you may go about finding, approaching and answering them. There are probably more unanswered questions in research today than ever before. As our knowledge grows, the boundaries of knowledge grow too. This means that many discoveries remain to be made, and some of them are waiting for you.
1.2 Embarking on a Ph.D.
To take a systematic approach to discovery you need to acquire new skills as well as new knowledge. Undertaking a Ph.D. will, hopefully, be one of the most developing periods in your life. In all fairness, it will be challenging too. You are likely to work alone a good part of the time. Another challenge is that you will be learning in a completely new way. It may be a shock to discover that high grades from previous studies do not necessarily help you now. The educational system often tends to assess our ability to learn facts and techniques rather than our actual problem solving skills, and research studies require more than having a good head for studying. When you embark on a Ph.D. you go from being a consumer of knowledge to a producer of knowledge; this requires a new set of talents. You do not obtain a Ph.D. for what you know but for what you can do. In other words, research education is largely about acquiring skills and this is why it takes time.
Undertaking a Ph.D. involves a more holistic approach to learning than your previous studies did. Schoolteachers often tend to teach their subjects separately. In high school, for example, it took my fellow students and I a good while to understand that calculus could be used to solve physics problems because it was taught without any hints to applications in other subjects. Little did we know that calculus had been developed by physicists in order to solve physics problems! This tradition of teaching often continues at university. The title of my undergraduate statistics book, for instance, announced that it was written for scientists and engineers. Despite this it did not contain a single example from science or engineering. I often wonder how students are expected to be able to apply subjects that are taught devoid of a relevant context. It is like teaching carpentry by handing out toolboxes and waiting for the students to discover how to use them.
From this point on, things will be different. During your research studies, you will acquire skills by engaging in real research tasks under the supervision of a professional researcher. This is similar to how carpentry is actually taught: if you are to learn how to build a house, you build a house to learn it. Although supervision and textbooks are required, skills can only be learned through hands-on experience. As a research student you will learn about research methods, measurement techniques, and statistical analyses by applying them to real research problems.
1.3 The Art of Discovery
Methods and techniques are all very well but good research involves more than that. As previously stated, science is also an attitude to the world. Professional researchers need to have a scientific mindset. When graduating you should be able to identify and formulate new research questions, develop your own approaches to answering them, and independently conduct research that meets the accepted quality standards of your field. In other words, you should master the scientific method to the point that you are able to make your own contribution to science.
To do this, you need a competency that is made up of several components. One of them is knowledge. In order to identify interesting research questions, you must have broad knowledge within your field as well as deep knowledge within the limited part where you are conducting your research. Another component is skills, meaning that you need to master a number of methods and techniques. Some of these will be quite general while others will be specific to your field, such as biochemical laboratory techniques or thermodynamic analyses in engineering. The final component is the one we started this chapter with: the scientific mindset.
You are expected to acquire these three components in different ways. The knowledge part mostly comes by reading books and scientific papers. Skills are often acquired by “on-the-job” training, in a laboratory or in the field. We frequently expect the mindset to be transferred, as it were, by molecular diffusion or some similar process. The hope is that you will crack the code just by being around researchers in a professional setting. Remarkably, this actually seems to work in many cases, but one purpose of this book is to make the process easier.
Figure 1.1 aims to illustrate that researchers use these three competencies in parallel. It also shows that they can be divided into competencies that are specific to a certain field of research and more general ones. Understanding the scientific approach is a general competency that should be shared by all researchers. The same is true of skills in applying general statistical methods, understanding how to plan and conduct experiments, knowing how to interpret data and so on. It is too common that the research education focuses on specific competencies, like subject matter knowledge and specific laboratory techniques, while neglecting the general aspects. This is probably because they are more difficult to define than the specific ones. Another reason is that the research system is designed in a way that does not naturally engage these competencies in Ph.D. students. Senior researchers tend to plan projects and students tend to work in the laboratory producing data. The aim of this book is to fill this gap in the research education.
Figure 1.1 Schematic representation of the three components of research competency. It is common that research education focuses on competencies that are specific to a certain field, while neglecting general research competencies.
1.4 About this Book
This book grew out of a need to teach general research competencies. As a Ph.D. student you need to understand how research differs from other activities. If you are working in an experimental field, you need to understand how experimentation differs from other forms of scientific investigation. It is also important to understand the general aspects of collecting and analyzing data. Finally, it is central to understand the scientific process: how to identify research problems, how to approach them, and how to communicate the results in the scientific community. There are good books that cover at least some of these aspects but it would be useful to introduce them in parallel, I thought, between the same pair of covers. It is often difficult for students to fit the pieces together after taking separate courses in the philosophy of science and statistics, for example, especially when the teachers are not experimental researchers themselves. Of course, a single book cannot provide a comprehensive treatment – especially not one as brief as this – but it can serve as a starting point.
The book is organized in two parts. The first part explains different aspects of scientific thinking and the second addresses general methods and techniques that are important to experimenters. The second part often refers to the first, but they can in principle be read separately. It may even be a good idea to skip back and forth between them. Whatever works for you will be fine.
The first part is called “Understanding the World”. It contains five chapters, of which you are halfway through the first. The next chapter introduces some central concepts from the philosophy of natural science. It explains that different approaches to scientific exploration result in different types of knowledge, and discusses the connection between observation and theory in science. Chapter 3 very briefly describes how science was invented and how it evolved into its modern form. There is also a discussion about how science has impacted on our civilization. After this, Chapter 4 takes a first look at experiments. It uses Galileo's pioneering experiment on the inclined plane to discuss some interesting aspects of the experimental method. The final chapter of the first part aims to demarcate between research and development, as this is an important distinction for students in applied research fields. There is a discussion about what we mean by theories and how they are developed. It also touches upon creativity, an important aspect of research that is often neglected. Discussing science and experimentation from different perspectives, this part of the book aims to give a better understanding of their essential characteristics.
The second part is called “Interfering With the World” as this is the very basis of the experimental method. It begins with a discussion about what distinguishes experimentation from passive observation. An important message is that observational studies only reveal correlations, whereas experiments can provide evidence for causation. It also explains under which conditions experiments may provide explanatory knowledge. After this, Chapters 7 and 8 cover statistical concepts and techniques that are useful in the collection and analysis of data. Together with Chapter 6 they form a basis for experimental design, which is introduced in Chapter 9. It combines experimental method with statistical techniques to make data collection a more economical and precise process.
The final three chapters describe an experimental research methodology. The methodology draws upon previous parts of the book and is built around a few simple tools. The tools aim to familiarize the reader with ways of thinking that are useful throughout the various stages of a research project. As illustrated in Figure 1.2, this book divides the research process into three phases. During the planning phase, a research question is formulated and an approach is developed to answer it. This is followed by the data collection phase, where a measurement system is devised, analyzed and improved to ensure that the data quality is sufficient for the purpose. After this, a data collection plan is formulated and the actual data are collected. In the third phase, focusing on analysis and synthesis, the data are processed and investigated using graphical and mathematical tools. The aim is to formulate conclusions supported by the data. The synthesis consists of relating these findings to the greater body of scientific knowledge and defining your contribution. The research process is depicted as circular since one experiment often uncovers new aspects of a problem and becomes the starting point of a new investigation.
Figure 1.2 The three phases of research.
This research process may not completely cover all branches of experimental research. It may emphasize parts that are less important in some cases, and may even lack features that may be important in others. It is important to note that the methodology is meant to be a pedagogical tool rather than a strict description of how research is done. The purpose is not to provide a cookbook, but to help you organize and elaborate your thinking about how experiments are planned, implemented and interpreted. To put the methodology into a practical context and make the discussion more concrete, we are going to follow two real world experiments through the three phases.
1.5 How to Use this Book
You will find exercises scattered throughout the chapters. The idea is that these should be completed when encountered to ensure that you have understood central concepts and ideas before continuing. There are more exercises in the second part of the book as it covers more technical topics than the first.
I have tried to write in a style that makes it possible to follow the book on one's own. This is also the reason why it is relatively light on mathematics. The amount of mathematics taught at the undergraduate level varies greatly between different fields, so some readers will be less familiar with mathematical concepts than others. Since it is not an end in itself to have to struggle with these concepts, I have tried to be inclusive rather than exclusive in the way that these are presented. This probably means that students in the engineering and physical sciences will be slightly bored by some passages. If you feel that they are not helpful, you should of course feel free to skip over them.
Although most chapters should be quite straightforward to read, some parts are necessarily more technical than others. When reading Chapters 7 through 9 you will probably need to switch to a lower gear and possibly re-read some passages to get through. Do not lose heart over this. It is not your fault that some parts of the terrain are rougher than others.
Many of the computational exercises in this book refer to worksheet functions in Microsoft Excel®. The reason is that this software is very widely available. You can of course complete these exercises just as well in other programs. Some readers will have access to MathWorks MATLAB®, for example, which provides a richer selection of functions and possibilities to write more complex programs.
If you are a Ph.D. student you will probably read this book at the beginning of your studies. It is my hope that it will provide useful ideas and methods that, in time, will be transferred into real skills. But this will not happen unless you apply what you learn to your own research. Reading conveys a theoretical picture but it is only through application that you challenge your understanding and reflect on what the theory really means. For this reason, I hope that this book will become something of a companion during your research studies – a book to return to from time to time when you develop your research ideas.
For those who are using this book to teach a course, a few practical recommendations may be useful. I have found that it helps to make the lectures as varied as possible. It is easier to maintain the audience's attention if you mix statistics lectures with material about scientific thinking, and mix exercises into the lectures. Since the aim of this material is to develop skills rather than knowledge, it is useful to teach and assess the learning outcomes in a practical context. My own approach has been to introduce much of the theoretical material in the first half of a course and devote the second half to simple “research” projects. During these, the class is divided into teams that choose simple research problems where they can apply the methodology and techniques. They have chosen to work with problems like brewing coffee, the rolling resistance of Lego vehicles, the lifetime of soap bubbles, or even strategies for minimizing the time needed to get out of bed in the morning. They begin by identifying interesting aspects of their problems. Is it, for example, possible to control the bitterness of coffee independently of the flavor strength? Or how do stress, comfort and other factors affect how quickly you get up in the morning? The purpose of these discussions is to generate hypotheses that can be tested in experiments. Apart from developing suitable measurement systems and test procedures for such problems, the main challenge is often to develop experiments that allow the teams to explain the outcomes, rather than just describe them. I have found that it is important to keep the problems simple enough to let the teams focus on the methodology, rather than to spend undue time getting sophisticated equipment to work. After the analysis they summarize their findings in a “scientific paper”, which is peer reviewed by another team before being presented to the others in the form of a conference paper.
Although these problems have little in common with real research problems, they allow the students to apply general research skills to solve real, unstructured problems. They also open up for creative discussions around all the various aspects of solving a research problem. According to the course evaluations, most students perceive these projects as very valuable for turning the theoretical knowledge into practical research skills.
But now it is time to be on our way. There are many stops on our journey and the first is the fundamental question: “What is science?”
Further Reading
Phillips and Pugh [1] provide an interesting and valuable overview of the Ph.D. process.
References
1. Phillips, E.M. and Pugh, D.S. (1994) How to Get A PhD: A Handbook For Students and Their Supervisors, 2nd edn, Open University Press, Buckingham.
2
What is Science?
The key to the approach is to keep firmly in mind that the classic position of a researcher is not that of one who knows the right answers but of one who is struggling to find out what the right questions might be!
—E.M. Phillips and D.S. Pugh
The purpose of this chapter is to discuss what science is, and what it is not. This is more of a tall order than it first seems, since science spans so many disciplines. Still, if we are going to learn how to develop science, it is important that we at least reflect on what that means.
Scientists at the beginning of their careers are introduced to the methods of science in different ways. Most learn the tools of the trade from experienced researchers but some complement this training with a course in the philosophy of natural science. Philosophy and science are different subject areas, so philosophers tend to see science from a different perspective than scientists do. You could say that they are more interested in the nature of knowledge than in knowledge of nature. Both perspectives are important but it can sometimes be difficult for science students to merge them in a constructive way. This chapter, along with other sections of this book, is an attempt to bridge the gap between some basic ideas in the philosophy of science and everyday scientific praxis.
Since philosophers and scientists approach science differently, I find it necessary to discuss the methods of science from different standpoints. To avoid confusion, therefore, I will try to indicate when I am speaking from the point of view of a philosopher and when from that of a scientist. I will start by introducing and discussing some basic concepts in the philosophy of natural science. It is important to point out that I neither aim to, nor can, give a deeper account of this subject. This is best left to real philosophers; further reading is suggested at the end of the chapter for those who wish to plunge deeper into it. Later in this chapter I will criticize some of these ideas from the more practical standpoint of a scientist. Although the philosophy of science is important for understanding the nature of scientific knowledge it can be perceived to be detached from the diverse reality of research. I hope that this approach will provide a balanced view in the end.
2.1 Characteristics of the Scientific Approach
Imagine that you are driving a rental car in a foreign country. You have never driven the model of car before and, despite the car being brand new, you find that it does not seem to work properly. Sometimes when you turn the ignition key the engine just will not start. Although you are not a specialist you do have a basic understanding of how an engine works and, starting from there, you begin to investigate the problem.
Based on your limited knowledge of engines you make a list of potential causes of the problem. Comparing the symptoms you would expect from these causes with your experience of the problem, you find yourself forced to discard one point after another on the list until there are no potential causes left. The next time the engine fails to start you are faced with the fact that you are completely clueless about what to do. In an act of desperation you decide to walk around the car before turning the ignition key again and, to your immense surprise, the engine now starts without a problem. Encouraged, you begin to experiment with this new method and find that walking in a clockwise direction around the car does not work. After a walk in a counter-clockwise direction, however, the engine always starts perfectly. So, in the course of your systematic investigation of the problem you have made a discovery, and a highly unexpected discovery at that!
Later, when returning the car to the rental car office, you complain about the problem. The woman behind the desk asks you if you remembered to push down the brake pedal when turning the key. She explains that this is a safety feature in some cars with automatic gearboxes. To decrease the risk of the car moving during starting, the brake must be pushed down. Thinking back you realize that you must unconsciously have put your foot on the brake pedal when entering the car after the counter-clockwise walk but not after the clockwise walk. That could explain why your method worked.
This may sound like a contrived example but I assure you that it is a true story from life, once told to me by a friend. (Being an engineer he was not particularly proud of trying to walk counter-clockwise around the car, but he admitted to being desperate when doing it.) We will return to this example later. It is useful here because the two methods presented in it could be said to represent two types of knowledge of a problem. Surely, many of us would agree that walking counter-clockwise around the car seems like a less “scientific” method than pushing on the brake. A possible justification is that the latter method is based on a deeper understanding of how cars work. On the other hand, the former method was discovered through a more or less structured investigation of the problem. Isn't that how scientists work? At any rate, we have seen that the fact that a method seems unscientific does not necessarily mean that it does not work.
Scientific research is about obtaining new knowledge, but what kind of knowledge becomes science and how is it obtained? In school, many of us have completed tasks that were called research. They generally involved a visit to the school library to collect information on a topic and summarizing this information in writing. Being able to find information is an important skill for researchers, and scientists do spend considerable time studying their literature, but simply collecting information from books is not research. Since the aim is to acquire new knowledge, it requires something beyond moving facts from one place to another, structuring them neatly and referencing the sources.
Science is often connected with measurements. Are measurements the crucial difference between science and non-science? Measurements are made in different parts of society, from laboratories to the fruit market. In applied research fields, academic researchers and development engineers in industry often make the same kind of measurements. How can it be that a Ph.D. student gets a fancy academic title after using a measurement technique for a few years at a university, while a young engineer making the same type of measurements in industry would be lucky to get a pat on the back for a job well done? If we are not to blame this difference on old tradition, and thereby deprive the Ph.D. degree of its value as an academic merit, there must be a central difference between what the engineer and the scientist do. Something beyond the everyday activities (like calibrating instruments, meticulously following procedures to assure good data quality and taking measurements) in the laboratory. To begin to cast some light over this difference I am going to borrow the following example from Molander [1]:
The main character of the example, Mr. Green, is a keen gardener who one day decides to count the apples in his garden. He goes about the task systematically and methodically and finds out that there are 1493 apples. This is definitely new knowledge, previously unknown to humanity, but is it science? Most people that I ask agree that it is not, even though they may have difficulties explaining why they think so. Those who have published their results in scientific journals and know how results are scrutinized in peer review processes may say that Mr. Green's chances of getting his results published are very slim. But why? It is not because Mr. Green does not have the proper scientific training – even with a Ph.D. in plant physiology he would not get these results published. For his results to become scientifically interesting they need to be incorporated into a greater context, a theoretical framework that gives them generality and helps us better understand some aspect of the world. If Mr. Green had counted his apples every year while also recording information about temperature, precipitation and hours of sun per day he could have searched for relationships in the data. That would approach a scientific way of obtaining new knowledge. When we judge scientific quality it also involves appraising the value of the new knowledge. What is it worth to know something about the number of apples in Mr. Green's garden? Is the garden unique in some sense, for example regarding microclimate, soil, or the type of apples grown there?
Collecting data is an important aspect of research, but it is also important in technical development, politics and other activities that are not considered to be science. There is a wealth of things that we do not yet know and that we could find out, like the oxygen content of the water in a particular bay, or the number of flowers in a particular field. Finding these things out requires planning, meticulous data collection and possibly statistical analysis to provide an unbiased picture of reality. Still, when finding them out we are only describing what we see, we are answering “what” questions. Science goes beyond pure description. It aspires to explain what happens in the world and to predict what will happen under certain circumstances. In other words, it aspires to answer what Phillips and Pugh [2] call “why” questions. Why is the oxygen level low in the bay? Why are there fewer flowers in the field some years? Answering such questions requires something more than careful collection of information. It requires a scientific approach. If you are an engineer working towards a strict deadline in a product development project, or a politician dealing with a problem that suddenly attracts media attention, your need to act can often be more pressing than your need to understand. For the scientist it is always the other way around. In research, trying to find a solution before you understand the nature of the problem is a bit like tying your shoelaces before you put your shoes on.
This means that there are two central and intimately interconnected aspects of science: one that has to do with investigating the world and one that is about interpreting what we see. By investigation we hope to acquire hard facts about reality, and we hope to obtain understanding by interpreting these facts. Interpretation is done within a theoretical framework that allows us to explain the facts. Philosophy books about scientific method often use the words observation and theory instead of investigation and interpretation. Observation sounds more passive than investigation and certainly has a narrower meaning. This is perhaps significant of the fact that such books often focus on how theories are developed. Both aspects are, however, two sides of a coin.
The next few sections are about two basic approaches to research that are described in the philosophy of science. They are written from a philosophical point of view. Even when I criticize the ideas I speak with the voice of a philosopher. In the remaining parts of this chapter I will again take a scientist's point of view by being a little bit more practically oriented, recognizing the fundamental role of observation in science. I hope to show that the philosophical concepts are useful for understanding the nature of scientific knowledge, although they do not cover all aspects of practical research. We can learn important things from these ideas but should not be too worried if some research does not fit perfectly into their framework.
2.2 The Inductive Method
It is a popular notion that scientists begin by collecting facts through organized observation and then, somehow, derive theories from them. In logic, going from a large set of specific observations to a general, theoretical conclusion in this manner is called induction and the approach is, therefore, often called the inductive method. Logic is a branch of philosophy that dates back to Aristotle. It deals with how arguments are made and how to determine if they are true or false. This brief description of the inductive approach follows closely to Chalmers [3], whose book is one of the most widely read introductory books about the philosophy of science. It is useful to describe his rather extreme form of “naive inductivism” in order to highlight some characteristics of the approach, especially those that are generally considered to be its weaknesses.
The inductivist version of science begins with observations, carefully recorded in the form of observation statements. These are always singular statements, meaning that they refer to something that was observed at a particular place and time and in a particular situation. For example, an astronomer might state that the planet Mars was observed at a certain position in the sky at a certain time. The rental car customer from the beginning of this chapter might state that, on a particular day, the engine started only after walking counter-clockwise around the car. To be able to explain some aspect of the world researchers must make generalizations from such singular statements to obtain universal ones, for example that all planets move in elliptic orbits around the sun, or that cars with automatic transmissions require the brake pedal to be pushed down in order to start. The inductivist maintains that it is legitimate to make generalizations from singular observations granted a few conditions are met. Firstly, the number of observations must be large. Secondly, they must be made under a wide variety of conditions. Finally, no observation can be in conflict with the conclusion drawn.
As mentioned above, inductive reasoning moves from a set of specific observations to a general conclusion. You may, for example, note that you become wet when you jump into the water. From a large set of such observations, made under varying conditions, you may infer the general conclusion that water always makes you wet. Going the opposite way, from general statements to specific conclusions, is called deduction. Consider the following example of deductive logic:
Here, the conclusion is a necessary logical consequence of the two premises. Premise 1 may be held to be a general law, derived from a large set of observations by induction. The problem is that the truth of the conclusion depends on the premises being true, which they obviously (at least to me) are not in this case. The weak link is induction because it cannot be logically justified. Even if you have made a very large number of consistent observations under various conditions, it does not follow that every future observation will be consistent with them. They provide some degree of support for your conclusion, but do not prove it to be true. The principles of induction and deduction are described graphically in Figure 2.1.
Figure 2.1 Induction moves from a set of specific observations to a general conclusion. Deduction moves from general statements to a set of expected, specific observations.
Chalmers uses a moral tale to drive this point home in an elaboration of Bertrand Russell's story of the inductivist turkey [2]:
A turkey found that, on his first morning at the turkey farm, he was fed at 9 a.m. Being a good inductivist, he did not jump to conclusions. He waited until he had collected a large number of observations of the fact that he was fed at 9 a.m., and he made these observations under a wide variety of circumstances: on Wednesdays and Thursdays, on warm days and cold days, on rainy days and dry days. Each day, he added another observation statement to his list. Finally, his inductivist conscience was satisfied and he carried out an inductive inference to conclude, “I am always fed at 9 a.m.”. Alas, this conclusion was shown to be false in no uncertain manner when, on Christmas Eve, instead of being fed, he was killed.
So, the inductivist's first condition is problematic: however large the number observations they can never ensure the truth of the conclusion. Let's look at the next condition and ask what counts as significant variation in circumstances. The turkey in the example made his observations on different weekdays and under varying weather conditions but not under sufficient variation of holiday seasons to find out what happens to turkeys on Christmas Eve. Determining the boiling point of water is perhaps an example more relevant to science. What should be changed to fulfill the criterion of sufficiently varying circumstances in that case? Is it necessary to vary the ambient pressure, or the purity of the water? Should we try different methods of heating, or different weekdays for the measurements? Most people would agree that the first two variables, pressure and purity, are sensible candidates, whereas the latter two make less sense. The point is that there are infinite ways to vary the circumstances, so how do we know which ones to choose? Well, we probably have some expectations of what will affect the boiling point of water and these expectations are based on some level of theoretical knowledge about the world. Theory could thereby be claimed to play a role prior to observation. In that case the inductivist assumption that science starts with observation does not hold true.
In addition to this, the observation statements we make to describe our observations could be said to rely on theory. Statements are always made in some language that contains definitions and they are thereby theoretically colored. As Chalmers puts it, even a simple statement such as “the gas will not light” is based on a theory that divides substances into classes, of which gases is one. It also presupposes that some gases are combustible. These theoretical notions have not always been available. In modern science the theoretical concepts are less commonplace and the observation statements may be more obviously theory-colored. Critics of the inductivist approach argue that observation statements are no more precise than the theory they rely on. If the theory turns out to be false, they say, the observation statements will no longer be valid.
We could summarize the criticism raised against the inductive method so far in two points. Firstly, observation is not a secure basis for knowledge. This is because induction in itself cannot be logically justified. We may make as many observations of something as we like, we still cannot use them to prove the truth of our conclusion. There is always the possibility that one day we will make an observation that contradicts the conclusion, as our inductivist turkey did on Christmas Eve. Secondly, it has been argued that observations and observation statements involve theory and, therefore, are just as fallible as theories are.
Besides these points there is another important weakness of a purely inductive approach that has to do with the type of knowledge that it produces. The problem is how singular observation statements can be generalized into theories that are scientifically useful. Science aims to explain and predict. Theories without explanatory power are thereby of limited use in science. If we note that we become wet every time we jump in the water we may infer that water makes people wet, but we cannot obtain theories about the underlying properties of water and people that explain why we get wet. The astronomer who observes the position of the planet Mars in the sky is perhaps a more illuminating example – how does the inductivist generalize from such observations to obtain Kepler's first law, that planets move in elliptic orbits with the sun as a focus? At the beginning of this chapter I said that the two solutions to the problem with the rental car represented different types of knowledge. The customer knew that walking around the car worked, at least if it was done in the counter-clockwise direction, but was not able to explain why. This is the type of knowledge that an inductive approach can provide us with. The woman at the rental car office knew that pushing down the brake pedal worked but she could also explain that this was a result of a certain safety feature of the particular car model. Can you think of any way to obtain this knowledge by a purely inductive approach?
Humankind has long noticed that there are regularities in Nature. Things tend to happen in certain ways and not in others, as if they were guided by machinery hidden from our eyes. The ultimate aim of science is to explain, rather than just describe, what happens in the world. As researchers we must therefore aspire to understand something of that hidden machinery. Since it is hidden from our eyes this task requires a more imaginative approach than just watching what goes on in the world and making notes of it.
2.3 The Hypothetico-Deductive Method
When the young Isaac Newton was toying with prisms in 1666 he noticed that they dispersed a white beam of sunlight into all the colors of the rainbow. If the colored light from a part of this rainbow was passed through a second prism it did not result in a new rainbow, only in further dispersion of the colored beam. The experiment is described schematically in Figure 2.2. Newton concluded that the prism did not add the rainbow colors to the light. Maybe the white light of a sunbeam already contained all the colors and all the prism did was to spread them out over the wall opposite his window? As a consequence, he thought, re-combining the colors should produce white light [4]. He set up an experiment to do just this and, as most readers are probably aware, his thought was confirmed.
Figure 2.2 Schematic representation of Newton's experimentum crucis. The second prism does not produce rainbow colors, as the first one does. Newton concluded that prisms do not add color to white light, but the colors are constituents of white light.
This is an example of working out the consequence of an hypothesis by logical deduction and testing it by an experiment (although Newton himself would not agree that this is the full story of what he did [4]). An hypothesis can be seen as a preliminary attempt to explain something – a theoretical statement without much observational support. We shall now discuss a method based on this procedure, where scientific inquiry begins with theory instead of observation. According to it, a scientist faced with a seemingly inexplicable phenomenon should begin by proposing speculative theories, or hypotheses, about its cause. These can then be tested rigorously against observation, for example by controlled experiments, to see if the consequences that follow from them correspond with reality. Working out consequences from premises is called deduction, speculative theories are called hypotheses and, accordingly, the method is called hypothetico-deductive. My description of the method closely follows how Chalmers describes falsificationism [3], which is a variant of the method proposed by Karl Popper. Most working scientists seem to apply some of the method's central elements, not least since they often discuss hypotheses and generally agree that hypotheses must be testable in order to be scientifically useful. In the words of Popper, hypotheses must be falsifiable.
If an hypothesis is formulated in a way that makes it theoretically impossible to disprove, it is not falsifiable. For example, a recruitment firm that throws half of the applications for a job into the wastebasket with the motivation “we don't want to recruit unlucky people” bases its decision on a non-falsifiable hypothesis. It is non-falsifiable because people whose applications end up in the wastebasket are, by definition, unlucky. Why would their applications otherwise be thrown away? It is a bit like saying that all sides of an equilateral triangle have equal length. If they did not it would not be an equilateral triangle – the statement is logically impossible to contradict. Non-falsifiable hypotheses are statements that are true whatever properties the world may have. Such hypotheses tell us nothing about the world and are, therefore, of no use in science.
According to the hypothetico-deductivist, research begins with an hypothesis and proceeds by testing it against observation. If our observations contradict the hypothesis we discard it and look for a new one. If the hypothesis is supported by our observations we keep it, but continue testing it under various conditions. The following classical example from Hempel [5] gives a flavor of how it could work.
Example 2.1: Semmelweis and the solution to childbed fever
