38,39 €
Become an efficient data scientist with R
This is the perfect tutorial for anyone who is new to statistical programming and modeling. Anyone with basic programming and data processing skills can pick this book up to systematically learn the R programming language and crucial techniques.
R is a high-level functional language and one of the must-know tools for data science and statistics. Powerful but complex, R can be challenging for beginners and those unfamiliar with its unique behaviors. Learning R Programming is the solution - an easy and practical way to learn R and develop a broad and consistent understanding of the language. Through hands-on examples you'll discover powerful R tools, and R best practices that will give you a deeper understanding of working with data. You'll get to grips with R's data structures and data processing techniques, as well as the most popular R packages to boost your productivity from the offset.
Start with the basics of R, then dive deep into the programming techniques and paradigms to make your R code excel. Advance quickly to a deeper understanding of R's behavior as you learn common tasks including data analysis, databases, web scraping, high performance computing, and writing documents. By the end of the book, you'll be a confident R programmer adept at solving problems with the right techniques.
Developed to make learning easy and intuitive, this book comes packed with a wide variety of statistical and graphical techniques and a wealth of practical information for anyone looking to get started with this exciting and powerful language.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 660
Veröffentlichungsjahr: 2016
Copyright © 2016 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: October 2016
Production reference: 1181016
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78588-977-6
www.packtpub.com
Author
Kun Ren
Copy Editor
Karuna Narayanan
Reviewer
Kelly Black
Project Coordinator
Izzat Contractor
Commissioning Editor
Akram Hussain
Proofreader
Safis Editing
Acquisition Editor
Denim Pinto
Indexer
Tejal Daruwale Soni
Content Development Editor
Rohit Kumar Singh
Graphics
Kirk D'Penha
Disha Haria
Jason Monteiro
Technical Editor
Vivek Pala
Production Coordinator
Melwyn D'sa
Kun Ren has used R for nearly 4 years in quantitative trading, along with C++ and C#, and he has worked very intensively (more than 8-10 hours every day) on useful R packages that the community does not offer yet. He contributes to packages developed by other authors and reports issues to make things work better. He is also a frequent speaker at R conferences in China and has given multiple talks. Kun also has a great social media presence. Additionally, he has substantially contributed to various projects, which is evident from his GitHub account:
I would like to thank my wife, who encouraged me to start working on this book a year ago, and my parents, who support me with unconditional love. I would also like to express my gratitude to the editors, Rohit Kumar Singh, Vivek Pala, and those who also worked hard on this book. They did a great job making the writing and publishing smooth.
Kelly Black is a member of the department of mathematics at the University of Georgia. His focus is stochastic differential equations, and he makes use of statistical software in a wide range of contexts, ranging from data analysis from Monte Carlo simulations to educational assessment.
I am grateful for the kindness and patience of Izzat Contractor, who helped guide the final production of this book.
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www.packtpub.com/mapt
Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.
R is designed for statistical computing, data analysis, and visualization. In recent years, it has become the most popular language for data science and statistics. R programming heavily involves data processing and it can be a challenge to program in R for those who are unfamiliar with the behaviors of the R language.
As a dynamic language, R allows extremely flexible use of data structures that are not as strict as compiled languages, such as C++, Java, and C#. When I started using R to process and analyze data, I found R’s behavior quirky, unpredictable, and sometimes very inconsistent.
In those data analysis projects, most effort was not spent running models. Instead, data cleaning, wrangling, and visualization took a major part of my time. In fact, it is most time consuming to find what’s wrong with the code that produced weird results or died in unexpected errors. Dealing with programming rather than field problems can be frustrating, especially when you have fought against bugs for hours without a clue.
However, as I work on more projects and gain more experience, I gradually know more about the behavior of objects and functions, and find that R is much more beautiful and consistent than I thought. That’s why I've written this book—to share my perspective on programming in R.
Through this book, you will develop a universal and consistent understanding of R as a programming language along with its vast set of tools. You will learn the best practices to boost your productivity, develop a deeper understanding of working with data, and become more confident about programming in R and solving problems with the right techniques.
Chapter 1, Quick Start, discusses a few basic facts about R, how to deploy an R environment, and how to code in RStudio.
Chapter 2, Basic Objects, introduces basic R objects and their behaviors.
Chapter 3, Managing Your Workspace, introduces the methods of managing the working directory, R environment, and the library of extension packages.
Chapter 4, Basic Expressions, covers the basic expressions of the R language: assignment, condition, and loop.
Chapter 5, Working with Basic Objects, discusses the basic functions each analyst should know in order to work with basic objects in R.
Chapter 6, Working with Strings, talks about R objects related with strings, and a number of string manipulation techniques.
Chapter 7, Working with Data, explains simple read/write data functions with some practical examples on various topics using basic objects and functions.
Chapter 8, Inside R, discusses R’s evaluation model by introducing what lazy evaluation, environment, function, and lexical scoping work is.
Chapter 9, Metaprogramming, introduces the metaprogramming techniques to help understand language objects and nonstandard evaluation.
Chapter 10, Object-Oriented Programming, describes the numerous object-oriented programming systems in R: S3, S4, RefClass, and community-provided R6.
Chapter 11, Working with Databases, shows how R works with popular relational databases such as SQLite and MySQL, and non-relational databases such as MongoDB and Redis.
Chapter 12, Data Manipulation, introduces techniques of manipulating relational data using data.table and dplyr, and non-relational data using rlist.
Chapter 13, High Performance Computing, discusses performance issues in R and several methods to boost computing performance.
Chapter 14, Web Scraping, talks about the basic structure of web pages, CSS, and XPath selectors and how to use the rvest package to scrape data from simple web pages.
Chapter 15, Boosting Productivity, demonstrates how R Markdown and shiny app, combined with interactive graphics, can boost productivity in the reporting and presentation of data analysis.
To run the example code in this book, you will need to install R 3.3.0 or newer. RStudio is the recommended development environment.
For Chapter 11, Working with Databases, a working MongoDB server and a Redis instance is required to run examples.
For Chapter 13, High Performance Computing, Rtools 3.3 is required to build an Rcpp code under Windows, and a gcc toolchain is required under Linux or macOS.
This book targets those who work on data-related projects and want to boost productivity but may not be familiar with the programming language and related tools.
This book also targets professional data analysts who want to systematically learn the R programming language, related techniques, and recommended packages and practices.
Although several chapters are a bit advanced for beginners, you don't have to be a computer expert or a professional data analyst to read those chapters, but I assume you will have a general idea of basic programming concepts and a basic experience of data processing.
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "The apply function also supports array input and matrix output."
The style of inline code words (variables and function names) and code blocks is set as follows:
x <- c(1, 2, 3) class(x) ## [1] "numeric" typeof(x) ## [1] "double" str(x) ## num [1:3] 1 2 3There will be a highlight on certain areas of the code whenever a point is being pointed out:
x <- rnorm(100) y <- 2 * x + rnorm(100) * 0.5 m <- lm(y ~ x) coef(m)New terms and important words are shown in bold.
Warnings or important notes appear in a box like this.
Tips and tricks appear like this.
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail [email protected], and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
You can download the code files by following these steps:
You can also download the code files by clicking on the Code Files button on the book's webpage at the Packt Publishing website. This page can be accessed by entering the book's name in the Search box. Please note that you need to be logged in to your Packt account.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/learningrprogramming. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at [email protected] with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.
Data analysis is difficult without the proper tools. It is almost impossible to extract patterns directly from a large set of numbers aligned in rows and columns and draw any conclusion, even for experts. A suitable tool, such as R, will remarkably boost your productivity in working with data. From my experience, learning a programming language is somehow like learning a human language. It is probably not a good idea to jump right into the details of vocabulary and grammar before looking at the big picture, getting motivated, and starting small. This chapter gives you a quick start by taking an overview of the R programming language in depth.
In this chapter, we will cover the following topics:
As soon as the software and tools are ready to go, you will write a simple R program to experience how it basically works. Once this is done, the R journey will unfold from the basics to advanced techniques and applications.
R is a powerful programming language and environment for statistical computing, data exploration, analysis, and visualization. It is free, open source, and has a strong, rapidly growing community where users and developers share their experience and actively contribute to the development of more than 7,500 packages, so that R can deal with problems in a wide range of fields (refer to https://cran.r-project.org/web/views/).
Although the origin of the R programming language dates back to 1993, its general adoption in R programming language data-related research industry has grown rapidly in the last decade and has become the lingua franca of data science.
In general, R should be viewed as more than just a programming language; it is a comprehensive computing environment, a strong and active community, and a rapidly growing and expanding ecosystem.
R, as a programming language, has been evolving and developing over the last 20 years. Its goal is quite clear to make it easy and flexible to perform comprehensive statistical computing, data exploration, and visualization.
However, ease of use and flexibility usually create conflicts. It can be very easy to click a few buttons to finish a variety of tasks in statistical analysis, but it won't be flexible if you need customization, automation, and your work needs to be reproducible. It can be very flexible to use tens of functions to transform data and make complicated graphics, but it won't be easy to learn and combine these functions correctly. R stands out for its well-positioned balance.
R, as a computing environment, is lightweight and ready to use. Compared to some other famous statistical software, for example, Matlab and SAS, R is much smaller and easier to deploy.
In this book, we will use RStudio to handle almost all our work in R. This integrated development environment provides rich features such as syntax-highlighting, auto-completion, package management, graphics viewer, help viewer, environment viewer, and debugging. These features hugely boost your productivity.
R, as a community, is strong and active. You can visit Try R (http://tryr.codeschool.com/) immediately and get a first impression of R basics through an interactive tutorial. In practice, when you are coding, you probably won't solve every problem by yourself. You may google an R question and find that it almost always has answers in StackOverflow (http://stackoverflow.com/questions/tagged/r). If your question is not fully addressed, you can ask it and probably get an answer in a couple of minutes.
If you need to use a package but also want to see how it works in detail, you can visit the source code at its online repository (or repo). Many repos are hosted by GitHub (https://www.github.com). In GitHub, you can do much more. When you find that a package is not working correctly, you can report a bug by filing an issue on the problem. If you need a feature that fits the purpose of the package, you can request a feature also by filing an issue for your demand. If you are interested in contributing to the package by resolving bugs and implementing features, you can fork the project, edit the code, and send merge requests so that your changes can be accepted by the owner. If your changes are accepted, congratulations, you have become a contributor to the package! Amazingly, R and its thousands of packages are built by contributors all over the world.
R, as an ecosystem, is rapidly growing and expanding in all data-related areas beyond the IT industry. The majority of its users are not professional developers but data analysts and statisticians. These users may not write the best-quality code, but they may contribute cutting-edge tools to the ecosystem in R language, and everyone else has free access to these tools without having to reinvent the wheels.
For example, let's say an econometrician writes an extension package that includes a new method to detect a category of time series patterns; it may attract several users who find it interesting and useful. Some professional users may improve the original code to make it faster and more general-purpose. A while later, a quantitative investor may find it helpful to incorporate this method into a trading strategy because it can detect patterns that usually causes risks in his/her portfolio. At the end of the day, the econometrician's tool is applied in a real-world industry, and the investor finds the portfolio less risky.
That is how the ecosystem works. And that is one of the reasons why R rocks in these areas: it has the ability to quickly adapt cutting-edge knowledge outside the IT industry (usually data science, Academia, and Industry) to generally available and applicable tools in the ecosystem. In other words, it facilitates conversion from the field knowledge and data science to productivity and value.
R stands out from a wide variety of statistical software for the following reasons:
In most scientific research areas and many industrial applications, reproducibility is necessary for many reasons. R scripts can precisely describe what you do with the computing environment and data so that it is fully reproducible from scratch.
Rich resources: R has a huge, rapidly increasing number of online resources. One type of resource is extension packages. There are, at the time of writing this, more than 7,500 packages available at CRAN (short for Comprehensive R Archive Network), a world-wide network of mirror servers from which you can get identical, up-to-date, R distributions and packages.These packages are created and maintained by more than 4,500 package developers in almost all data-related areas, such as multivariate analysis, time series analysis, econometrics, Bayesian inference, optimization, finance, genetics, chemometrics, computational physics, and many others. Take a look at CRAN Task View (https://cran.r-project.org/web/views/) for a good summary.
In addition to the enormous number of packages, there are also a great number of authors who regularly write personal blogs and Stack Overflow answers and share their thoughts, experiences, and recommended practices. Plus, there are a lot of websites specializing in R, such as R-bloggers (http://www.r-bloggers.com/), R documentation (http://www.rdocumentation.org/), and METACRAN (http://www.r-pkg.org/).
Strong community: The community of R consists of not only R developers but also, (the majority), R users from a wide range of backgrounds such as statistics, econometrics, finance, bioinformatics, mechanical engineering, physics, medicine, and so on.A great number of R developers actively contribute to open source projects or packages written in R. The goal of the community is to make data analysis, exploration, and visualization easier and more interesting.
If you are stuck in a problem in R, just google what puzzles you; probably, there are already some answers to your question. If not, just ask a question on Stack Overflow and you will get a response in a very short time.
Cutting-edge: Many R users are professional researchers in statistics, econometrics, or other disciplines. Quite often, authors publish their new papers along with a new package that includes the cutting-edge techniques presented in the paper. Maybe it's a new statistical test, a pattern recognition method, or a better-tuned optimization algorithm.No matter what it is, the R community has the privilege of applying cutting-edge data science knowledge in the real world often ahead of everyone else, improving its functionality and revealing its potential.
To install R, you need to visit R's official website (https://www.r-project.org/), download R (https://cran.r-project.org/mirrors.html), choose a nearby mirror, and download a version for your operating system. At the time of writing, the latest version is 3.2.3. The examples in this book are created and run under this version in Windows and Linux, but there should not be significant differences between the output in previous versions or other supported operating systems.
If you are using Windows, just download an installer for the latest version. To install R, run the Windows installer that you just downloaded. The installation process is easy to handle, but many users may still face problems with several steps.
In the Windows drop-down, when choosing the components to install, the installer lists four components. Here, Core files means the core libraries of R, and the Message translations component provides many versions of translations of warning and error messages in a list of supported languages. However, what may confuse you is the 32-bit files and 64-bit files options. Just don't worry; you only need to know that 64-bit R can handle much more data in a single process than its 32-bit counterpart. If you are using a modern computer purchased in recent years, it is most likely to support 64-bit programs and should be running a 64-bit operating system, so the default option will be 64-bit files. If you are using a 32-bit operating system, unfortunately, you cannot use 64-bit R unless you install a 64-bit system if your hardware supports it.
Anyway, I recommend that you install the default options, as shown in the following screenshot:
Another option you may feel confused about is whether to save the R version number in the registry. Checking these options makes it easier for other programs to detect which R version is installed. If you are sure you only use R in its own, just go ahead with the defaults.
Then, the installation starts copying files to your hard drive.
Finally, R is deployed to your computer. At the moment, you only have two ways to use R: In a command prompt (or terminal) or in the R GUI.
If you allowed the installer to create program shortcuts on your desktop, you will find two R shortcuts there. R runs in the Command Prompt and RGUI runs in an extremely simple GUI.
Although you can start to use R right now, it does not mean you have to use it in this way. I strongly recommend RStudio for editing and debugging R scripts. Actually, this book is also written in R Markdown in RStudio. Although RStudio is powerful, it does not work without a proper installation of R. In other words, R is the backend and RStudio is a frontend that helps you better work with the backend.
If you are using Windows, you may also install Rtools (http://cran.rstudio.com/bin/windows/Rtools/) so that you can write C++ code, compile it, and call it in R, and you can install and compile packages that contain C/C++ code from their sources.
RStudio is a powerful user interface for R programming. It's free, open source, and works on multiple platforms including Windows, Mac, and Linux.
RStudio has very powerful features that hugely boost your productivity in data analysis and visualization. It supports syntax highlighting, autocompletion, multi-tabbed views, file management, graphics viewport, package management, integrated help viewer, code formatting, version control, interactive debugging, and many more features.
You can download the latest release of RStudio at https://www.rstudio.com/products/rstudio/download. If you want to try the preview version with new features, download it from https://www.rstudio.com/products/rstudio/download/preview. Note that RStudio does not include R, so you need to make sure that you have R installed while working in RStudio.
In followings sections, I'll give you a brief introduction to the user interface of RStudio.
The following screenshot shows the RStudio user interface in the Windows operating system. If you are using Mac OS X or a supported version of Linux, the screen should look almost the same.
You may notice that the main window consists of several parts. Each part is called a pane and performs different functions. These panes are well designed for data analysts to work with data.
The following screenshot shows the R console embedded in RStudio. In most cases, the console works exactly like a Command Prompt or terminal. In fact, when you type in a command at the console, RStudio will submit the request to the R engine. It is the R engine that executes all the commands. The role of RStudio is to stand in the middle, take inputs from user to the R engine, and present the results it returns.
Using the console, you can easily execute a command, define a variable, or evaluate an expression interactively to compute a statistical measure, transform data, or produce charts.
Typing in commands at the console is not the usual way we work with data. Instead, we write scripts, a set of commands representing a logic flow that can be read from a file and executed by the R engine. The editor is useful for editing R scripts, markdown documents, web pages, many types of configuration files, and even C++ source code.
The functionality of the code editor is much more than a plain text editor: it supports syntax highlighting, autocompletion of R code, debugging with breakpoints, and so on. More specifically, when editing R scripts you can use the following shortcut keys:
The Environment pane shows the variables and functions that you have created and are available for repeated use. By default, it shows the variables in the global environment, that is, the user workspace in which you are working.
Each time you create a new object (a variable or function), a new entry will appear in the Environment pane. The entry shows the variable name and a short description of its value. When you change the value of a symbol or even remove that symbol, you actually modify the environment so that the environment pane reflects your change.
The History pane shows the previous expressions evaluated in the console. You can repeat the task performed previously by simply pressing up in the console.
The history may be stored in the .Rhistory file in the working directory.
The File pane shows the files in the folder. You can navigate between folders, create new folders, delete or rename folders or files, and so on.
If you are working on an RStudio project, the File pane is handy for viewing and organizing project files.
The Plots pane is used to show graphics produced by R code. If you produce more than one plot, the previous ones are stored and you can navigate back and forth to view all plots (until you clear them).
When you resize the plot pane, graphics will adapt to its size so that they look as nice as they did before resizing. You can also export a plot to a file for future use.
Much of R's power derives from its packages. The Packages pane shows all installed packages. You can also easily install or update packages from CRAN or remove an existing package from your library.
A lot of R's power also derives from its detailed documentation. The Help pane shows the documentation so that you can easily learn how to use functions.
There are numerous ways to View a function's documentation:
In practice, you don't have to remember all of R's functions; you only need to remember how to get help with a function you are not familiar with.
The Viewer pane is a new feature; it was introduced as an increasing number of R packages combine the functionality of both R and existing JavaScript libraries to make rich and interactive presentations of data.
The following screenshot is an example of my formattable (http://renkun.me/formattable) package that provides a simple implementation of conditional formatting in Excel with data frames in R:
If you are using a supported version of Linux, you can easily set up a server version of RStudio, or RStudio Server. It runs on a host server (probably much more powerful and stable than your laptop) and you can run an R session in RStudio in your web browser. The user interface is mostly the same but you have access to the computing and memory resources of the server, as if you were using a local computer.
In this chapter, you learned some basic facts about R and its major strengths. We learned how to install R in a Windows operating system. To make R programming easier, we chose to use RStudio and went through the user interface of RStudio, and you learned that the functionality of each pane is its main window. Finally, we ran several R commands to fit a model and to plot simple graphics, getting an initial impression of the way we work with R.
In the next chapter, we will go through the basic concepts and data structures in R to help you get familiar with the behavior of basic R objects. Only then can you easily represent, manipulate, and work with a wide variety of data.
