23,99 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Lebensstil
- Sprache: Englisch
Web scraping is a technique to extract data from websites. It simulates the behavior of a website user to turn the website itself into a web service to retrieve or introduce new data. This book gives you all you need to get started with scraping web pages using R programming.
You will learn about the rules of RegEx and Xpath, key components for scraping website data. We will show you web scraping techniques, methodologies, and frameworks. With this book's guidance, you will become comfortable with the tools to write and test RegEx and XPath rules.
We will focus on examples of dynamic websites for scraping data and how to implement the techniques learned. You will learn how to collect URLs and then create XPath rules for your first web scraping script using rvest library. From the data you collect, you will be able to calculate the statistics and create R plots to visualize them.
Finally, you will discover how to use Selenium drivers with R for more sophisticated scraping. You will create AWS instances and use R to connect a PostgreSQL database hosted on AWS. By the end of the book, you will be sufficiently confident to create end-to-end web scraping systems using R.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 88
Veröffentlichungsjahr: 2018
Ähnliche
R Web Scraping Quick Start Guide
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor:Amey VarangaonkarAcquisition Editor:Noyonika DasContent Development Editor:Kirk DsouzaTechnical Editor: Shweta JadhavCopy Editor: Safis EditingProject Coordinator: Hardik BhindeProofreader: Safis EditingIndexer:Priyanka DhadkeGraphics:Alishon MendonsaProduction Coordinator: Arvindkumar Gupta
First published: October 2018
Production reference: 1301018
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78913-873-3
www.packtpub.com
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
Packt.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the author
Olgun Aydin is a PhD candidate at the Department of Statistics at Mimar Sinan University, and is studying deep learning for his thesis. He also works as a data scientist.
Olgun is familiar with big data technologies, such as Hadoop and Spark, and is a very big fan of R. He has already published academic papers about the application of statistics, machine learning, and deep learning.
He loves statistics, and loves to investigate new methods and share his experience with other people.
About the reviewer
Ezgi Nazman is a Research Assistant in Gazi University, Statistics where she has been studying for a PhD since 2015. She did her bachelor's degree at Ege University, Statistics in 2012. She was Erasmus student in University of Hradec Kralove, Informatics and Management between September 2011 - June 2012. From 2013-2015, she studied Master of Science in Statistics, Gazi University.
Her research interests lie in the area of statistics, data mining, and machine learning, ranging from theory to implementation. She has had five articles published on SCI and other international indexes, and eight presented papers in international conferences. She is one of the authors of a book titled Arastirmacilar icin SPSS Uygulamali Deney Tasarimi.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
R Web Scraping Quick Start Guide
Dedication
Packt Upsell
Why subscribe?
Packt.com
Contributors
About the author
About the reviewer
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
Introduction to Web Scraping
Learning about data on the internet
Introduction to XPath (XML Path)
Data extraction systems
Web scraping techniques
Traditional copy and paste
Text grabbing and regular expression
Document Object Model (DOM)
Semantic annotation recognition
Web scraping tools
JavaScript tools
Web crawling frameworks
Web crawling environment in R
Summary
XML Path Language and Regular Expression Language
XML Path (XPath)
Nodes
Relationships between nodes
Parent
Child
Sibling
Ancestor
Descendant
Predicates
Selecting unknown nodes
Selecting several paths
Regular expression language (Regex)
How to match a single character
How to match the characters of a set
How to match words
Exercises on RegEx and XPath
RegEx exercises
XPath exercises
Summary
Web Scraping with rvest
Introducing rvest
Step-by-step web scraping with rvest
Writing XPath rules
Writing your first scraping script
Playing with data
Summary
Web Scraping with Rselenium
Advantages and disadvantages of using Selenium for web scraping
RSelenium
Step-by-step web scraping with RSelenium
Collecting data with RSelenium
Summary
Storing Data and Creating Cronjob
Cloud engine models
Infrastructure as a service (IaaS)
Platform as a service (PaaS)
Software as a service (SaaS)
Mobile backend as a service (MBaaS)
Function as a service (FaaS)
Some of the cloud services
Amazon Web Services (AWS)
Google Cloud
Cronjob
Storing data and creating schedule jobs for web scraping
Creating an AWS RDS Instance
Connecting to the PostgreSQL database on AWS
Creating cronjob
Summary
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
This book is for R programmers looking to quickly get started with web scraping. Some fundamental knowledge of R is required. This book will give you a quick, hands-on introduction to web scraping and how to use popular R libraries, such asrvestand RSelenium. Right from the initial environment setup to quickly scraping HTML web pages for useful information, this book will cover only the absolute fundamentals of web scraping without going into too much depth. By the end of the book, you will have the understanding that's necessary for scraping any web page using R programming.
Who this book is for
This book is for R programmers looking to quickly get started with web scraping, as well as data analysts who want to learn about scraping using R. Some fundamental knowledge of R is all that is required to get started with this book.
What this book covers
Chapter 1, Introduction to Web Scraping, introduces web scraping techniques, which are getting more and more popular, since data is as valuable as oil in the 21st century. In this chapter, you can find detailed information about web scraping technologies. We also take an overview of some of the key languages for web scraping, such as XPath and regEX. We'll also look into some web scraping libraries for R, such asrvestand RSelenium technologies.
Chapter 2, Working with the XML Path Language and the Regular Expression Language, looks at XPath and regEX rules, which are quite important to know when scraping a web page. In this chapter, you can find useful information about these languages and also have a chance to write XPath and regEX rules from scratch.
Chapter 3, Web Scraping with rvest, covers the rvest library. Scraping a web page with R is straightforward thanks to the rvest library, which was developed by Hadley Wickham. In this chapter, you can find tips and tricks about the library and learn how to write an R script by using the rvest library to scrape a web page from scratch.
Chapter 4, Web Scraping with RSelenium, explores RSelenium. RSelenium is a technology for testing, but it's also useful for scraping web pages. In this chapter, you can find an overview of Selenium and learn how to scrape a web page using RSelenium library.
Chapter 5, Storing Data and Creating Cronjobs, deals with the matter of storage. After collecting data, you should store the dataset somewhere; it would be good if you could use a cloud-based solution, such as AWS RDS, EC2, Google Cloud Platform, or Microsoft Azure. Also, if you would like to schedule the collection of data, it's possible to create cronjob that will help you do so. In this chapter, you can find an overview of databases and cloud platforms, and you'll also learn how to connect databases and schedule cronjobs using R.
To get the most out of this book
To get the most out of this book, its important that you have an idea of what web scraping is. Also, it is advised that you have good hands-on experience with R programming.
If you have R and RStudio ready on your PC to get started, you will find the information of all packages that are required for scraping data within the chapters.
Download the example code files
You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packt.com/support and register to have the files emailed directly to you.
