39,59 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Harness the latest features of MongoDB 3 with this collection of 80 recipes – from managing cloud platforms to app development, this book is a vital resource
About This Book
- Get to grips with the latest features of MongoDB 3
- Interact with the MongoDB server and perform a wide range of query operations from the shell
- From administration to automation, this cookbook keeps you up to date with the world's leading NoSQL database
Who This Book Is For
This book is engineered for anyone who is interested in managing data in an easy and efficient way using MongoDB. You do not need any prior knowledge of MongoDB, but it would be helpful if you have some programming experience in either Java or Python.
What You Will Learn
- Install, configure, and administer MongoDB sharded clusters and replica sets
- Begin writing applications using MongoDB in Java and Python languages
- Initialize the server in three different modes with various configurations
- Perform cloud deployment and introduce PaaS for Mongo
- Discover frameworks and products built to improve developer productivity using Mongo
- Take an in-depth look at the Mongo programming driver APIs in Java and Python
- Set up enterprise class monitoring and backups of MongoDB
In Detail
MongoDB is a high-performance and feature-rich NoSQL database that forms the backbone of the systems that power many different organizations – it's easy to see why it's the most popular NoSQL database on the market. Packed with many features that have become essential for many different types of software professionals and incredibly easy to use, this cookbook contains many solutions to the everyday challenges of MongoDB, as well as guidance on effective techniques to extend your skills and capabilities.
This book starts with how to initialize the server in three different modes with various configurations. You will then be introduced to programming language drivers in both Java and Python. A new feature in MongoDB 3 is that you can connect to a single node using Python, set to make MongoDB even more popular with anyone working with Python. You will then learn a range of further topics including advanced query operations, monitoring and backup using MMS, as well as some very useful administration recipes including SCRAM-SHA-1 Authentication. Beyond that, you will also find recipes on cloud deployment, including guidance on how to work with Docker containers alongside MongoDB, integrating the database with Hadoop, and tips for improving developer productivity.
Created as both an accessible tutorial and an easy to use resource, on hand whenever you need to solve a problem, MongoDB Cookbook will help you handle everything from administration to automation with MongoDB more effectively than ever before.
Style and approach
Every recipe is explained in a very simple set-by-step manner yet is extremely comprehensive.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 571
Veröffentlichungsjahr: 2016
Ähnliche
Table of Contents
MongoDB Cookbook Second Edition
MongoDB Cookbook Second Edition
Copyright © 2016 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: November 2014
Second Edition: January 2016
Production reference: 1060116
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78528-998-9
www.packtpub.com
Credits
Authors
Cyrus Dasadia
Amol Nayak
Reviewers
Christopher Dambamuromo
Laurence Putra Franslay
Jason Nichols
Commissioning Editor
Dipika Gaonkar
Acquisition Editor
Shaon Basu
Content Development Editor
Zeeyan Pinheiro
Technical Editor
Bharat Patil
Copy Editor
Tasneem Fatehi
Project Coordinator
Francina Pinto
Proofreader
Safis Editing
Indexer
Mariammal Chettiyar
Graphics
Disha Haria
Production Coordinator
Arvindkumar Gupta
Cover Work
Arvindkumar Gupta
About the Authors
Cyrus Dasadia always liked tinkering with open source projects since 1996. He has been working as a Linux system administrator and part-time programmer for over a decade. He works at InMobi, where he loves designing tools and platforms. His love for MongoDB started in 2013, when he was amazed by its ease of use and stability. Since then, almost all of his projects are written with MongoDB as the primary backend. Cyrus is also the creator of an open source alert management system called CitoEngine. He likes spending his spare time trying to reverse engineer software, playing computer games, or increasing his silliness quotient by watching reruns of Monty Python.
To my wife, Nilu, provider of unconditional love and support; recipient of bad jokes.
Amol Nayak is a MongoDB certified developer and has been working as a developer for over 8 years. He is currently employed with a leading financial data provider, working on cutting-edge technologies. He has used MongoDB as a database for various systems at his current and previous workplaces to support enormous data volumes. He is an open source enthusiast and supports it by contributing to open source frameworks and promoting them. He has made contributions to the Spring Integration project, and his contributions are the adapters for JPA, XQuery, MongoDB, Push notifications to mobile devices, and Amazon Web Services (AWS). He has also made some contributions to the Spring Data MongoDB project. Apart from technology, he is passionate about motor sports and is a race official at Buddh International Circuit, India, for various motor sports events. Earlier, he was the author of Instant MongoDB, Packt Publishing.
I would like to thank everyone at Packt Publishing who has been involved with this book. It started when Luke Presland from Packt Publishing approached me to author a book on Mongo. I was skeptical to take up the opportunity due to other commitments and tight deadlines, but if it wasn't for my mom, friends, and office colleagues who convinced me to take up the opportunity, I would not have written this book. The chapters and content covered was a lot, and I had a tough time keeping up with the timelines. A special thanks to Priyanka, Rebecca, Mary, and Joel with whom I interacted the most; they were very flexible to my changes in delivery timelines. A big thanks to Douglas Duncan and other reviewers of the book for reviewing the book closely and helping improve the quality of the content drastically. Finally, I would like to thank the other staff at Packt Publishing who were involved in the book's publishing process but haven't interacted with me.
About the Reviewers
Christopher Dambamuromo is a MongoDB evangelist who is an active contributor to StackOverflow on the MongoDB tag. Chris frequently engages the open source community and foster adoption of MongoDB. He helps engineering teams build scalable and performant applications in MongoDB. He also assesses the health, scalability, and capacity of distributed systems and advises engineering teams on the schema design, architecture, and deployment planning. He is a passionate and proficient C#/BI developer with more than 10 years of experience producing code to a consistently high standard. Chris has constantly been honing his skills in the technical aspects of software and business intelligence engineering (JavaScript, Node.js, HTML5, CSS3, Python, R, Java, ASP.NET MVC, C# Microsoft BI stack—MS SQL server, SSIS, SSRS, SSAS, SharePoint, and PowerPivot), data modeling, systems architecture, as well as business applications for BI solutions. He has an MSc degree in intelligent computer systems from the University of Glamorgan (UK), an MSc degree in mathematics and computing for finance from Swansea University, UK, and a BSc degree with honours in applied mathematics from the National University of Science and Technology, Zimbabwe. Chris is very fond of anything closely or remotely related to data and mathematics, and as long as it can be represented as a string of ones and zeros and then analyzed and visualized, you've got his attention! His GitHub handle is http://github.com/chrisdamba and can be found on StackOverflow as http://stackoverflow.com/users/122005/chridam.
Laurence Putra Franslay is a software engineer working in Singapore and runs the Singapore MongoDB user group. In his free time, he hacks away on random stuff and picks up new technologies. His key interests lie in security and distributed systems. For more information, view his profile at http://geeksphere.net/.
www.PacktPub.com
Support files, eBooks, discount offers, and more
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at <[email protected]> for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www2.packtpub.com/books/subscription/packtlib
Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.
Why Subscribe?
Free Access for Packt account holders
If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access.
Preface
MongoDB is a document-oriented, leading NoSQL database, which offers linear scalability, thus making it a good contender for high-volume, high-performance systems across all the business domains. It has an edge over the majority of NoSQL solutions for its ease of use, high performance, and rich features.
This book provides detailed recipes that describe how to use the different features of MongoDB. The recipes cover topics ranging from setting up MongoDB, knowing its programming language API, and monitoring and administration, to some advanced topics such as cloud deployment, integration with Hadoop, and some open source and proprietary tools for MongoDB. The recipe format presents the information in a concise, actionable form; this lets you refer to the recipe to address and know the details of just the use case in hand without going through the entire book.
What this book covers
Chapter 1, Installing and Starting the Server, is all about starting MongoDB. It will demonstrate how to start the server in the standalone mode, as a replica set, and as a shard, with the provided start up options from the command line or configuration file.
Chapter 2, Command-line Operations and Indexes, has simple recipes to perform CRUD operations in the Mongo shell and create various types of indexes in the shell.
Chapter 3, Programming Language Drivers, discusses about programming language APIs. Though Mongo supports a vast array of languages, we will look at how to use the drivers to connect to the MongoDB server from Java and Python programs only. This chapter also explores the MongoDB wire protocol used for communication between the server and programming language clients.
Chapter 4, Administration, contains many recipes for administration or your MongoDB deployment. This chapter covers a lot of frequently used administrative tasks such as viewing the stats of the collections and database, viewing and killing long-running operations and other replica sets, and sharding-related administration.
Chapter 5, Advanced Operations, is an extension of Chapter 2, Command-line Operations and Indexes. We will look at some of the slightly advanced features such as implementing server-side scripts, geospatial search, GridFS, full text search, and how to integrate MongoDB with an external full text search engine.
Chapter 6, Monitoring and Backups, tells you all about administration and some basic monitoring. However, MongoDB provides a state-of-the-art monitoring and real-time backup service, MongoDB Monitoring Service (MMS). In this chapter, we will look at some recipes around monitoring and backup using MMS.
Chapter 7, Deploying MongoDB on the Cloud, covers recipes that use MongoDB service providers for cloud deployment. We will set up our own MongoDB server on the AWS cloud as well as run MongoDB in Docker containers.
Chapter 8, Integration with Hadoop, covers recipes to integrate MongoDB with Hadoop to use the Hadoop MapReduce API in order to run MapReduce jobs on the data residing in MongoDB data files and write the results to them. We will also see how to use AWS EMR to run our MapReduce jobs on the cloud using Amazon's Hadoop cluster, EMR, with the mongo-hadoop connector.
Chapter 9, Open Source and Proprietary Tools, is about using frameworks and products built around MongoDB to improve a developer's productivity or about simplifying some of the day-to-day jobs using Mongo. Unless explicitly mentioned, the products/frameworks that we will be looking at in this chapter are open source.
Appendix, Concepts for Reference, gives you a bit of additional information on the write concern and read preference for reference.
What you need for this book
The version of MongoDB used to try out the recipes is 3.0.2. The recipes are good for version 2.6.x as well. In case of some special feature specific to version 2.6.x, it would be explicitly mentioned in the recipe. Unless explicitly mentioned, all commands should be executed on Ubuntu Linux.
The samples where Java programming was involved were tested and run on Java Version 1.7, and Python code was run using Python v2.7 (compatible with Python 3). For MongoDB drivers, you can choose to use the latest available version.
These are pretty common types of software, and their minimum versions are used across different recipes. All the recipes in this book will mention the required software to complete it and their respective versions. Some recipes need to be tested on a Windows system, while some on Linux.
Who this book is for
This book is designed for administrators and developers who are interested in knowing MongoDB and using it as a high-performance and scalable data storage. It is also for those who know the basics of MongoDB and would like to expand their knowledge. The audience of this book is expected to have at least some basic knowledge of MongoDB.
Conventions
In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "Create the /data/mongo/db directory (or any of your choice)."
A block of code is set as follows:
Any command-line input or output is written as follows:
New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: "As we want to start a free micro instance, check the Free tier only checkbox on the left".
Note
Warnings or important notes appear in a box like this.
Tip
Tips and tricks appear like this.
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.
To send us general feedback, simply send an e-mail to <[email protected]>, and mention the book title via the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Downloading the example code
You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the erratasubmissionform link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.
Piracy
Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors, and our ability to bring you valuable content.
Questions
You can contact us at <[email protected]> if you are having a problem with any aspect of the book, and we will do our best to address it.
Chapter 1. Installing and Starting the Server
In this chapter, we will cover the following recipes:
Introduction
In this chapter, we will look at starting up the MongoDB server. Though it is a cakewalk to start the server with default settings for development purposes, there are numerous options available to fine-tune the start up behavior. We will start the server as a single node and then introduce various configuration options. We will conclude this chapter by setting up a simple replica set and running a sharded cluster. So, let's get started with installing and setting up the MongoDB server in the easiest way possible for simple development purposes.
Installing single node MongoDB
In this recipe, we will look at installing MongoDB in the standalone mode. This is the simplest and quickest way to start a MongoDB server, but it is seldom used for production use cases. However, this is the most common way to start the server for development purposes. In this recipe, we will start the server without looking at a lot of other startup options.
Getting ready
Well, assuming that we have downloaded the MongoDB binaries from the download site, extracted it, and have the resulting bin directory in the operating system's path variable. (This is not mandatory, but it really becomes convenient after doing so.) The binaries can be downloaded from http://www.mongodb.org/downloads after selecting your host operating system.
How to do it…
How it works…
If you see the following line on the console, you have successfully started the server:
Starting a server can't get easier than this. Despite the simplicity in starting the server, there are a lot of configuration options that can be used to tune the behavior of the server on startup. Most of the default options are sensible and need not be changed. With the default values, the server should be listening to port 27017 for new connections, and the logs will be printed out to the standard output.
See also
There are times where we would like to configure some options on server startup. In the Installing single node MongoDB recipe, we will use some more start up options.
Starting a single node instance using command-line options
In this recipe, we will see how to start a standalone single node server with some command-line options. We will see an example where we want to do the following:
As the server has been started for development purposes, we don't want to preallocate full-size database files. (We will soon see what this means.)
Getting ready
If you have already seen and executed the Installing single node MongoDB recipe, you need not do anything different. If all these prerequisites are met, we are good for this recipe.
How to do it…
How it works…
Ok, this wasn't too difficult and is similar to the previous recipe, but we have some additional command-line options this time around. MongoDB actually supports quite a few options at startup, and we will see a list of the most common and important ones in my opinion:
Option
Description
--help or -h
This is used to print the information of various start up options available.
--config or -f
This specifies the location of the configuration file that contains all the configuration options. We will see more on this option in a later recipe. It is just a convenient way of specifying the configurations in a file rather than on the command prompt; especially when the number of options specified is more. Using a separate configuration file shared across different MongoDB instances will also ensure that all the instances are running with identical configurations.
--verbose or -v
This makes the logs more verbose; we can put more v's to make the output even more verbose, for example, -vvvvv.
--quiet
This gives a quieter output; this is the opposite of verbose or the -v option. It will keep the logs less chatty and clean.
--port
This option is used if you are looking to start the server listening to some port other than the default 27017. We would be frequently using this option whenever we are looking to start multiple mongo servers on the same machine, for example, --port 27018 will start the server listening to port 27018 for new connections.
--logpath
This provides a path to a log file where the logs will be written. The value defaults to STDOUT. For example, --logpath /logs/server.out will use /logs/server.out as the log file for the server. Remember that the value provided should be a file and not a directory where the logs will be written.
--logappend
This option appends to the existing log file, if any. The default behavior is to rename the existing log file and then create a new file for the logs of the currently started mongo instance. Suppose that we have used the name of the log file as server.out, and on startup, the file exists, then by default this file will be renamed as server.out.<timestamp>, where <timestamp> is the current time. The time is GMT as against the local time. Let's assume that the current date is October 28th, 2013 and time is 12:02:15, then the file generated will have the following value as the timestamp: 2013-10-28T12-02-15.
--dbpath
This provides you with the directory where a new database will be created or an existing database is present. The value defaults to /data/db. We will start the server using /data /mongo/db as the database directory. Note that the value should be a directory rather than the name of the file.
--smallfiles
This is used frequently for development purposes when we plan to start more than one mongo instance on our local machine. Mongo, on startup, creates a database file of size 64 MB (on 64-bit machines). This preallocation happens for performance reasons, and the file is created with zeros written to it to fill out space on the disk. Adding this option on startup creates a preallocated file of 16 MB only (again, on a 64-bit machine). This option also reduces the maximum size of the database and journal files. Avoid using this option for production deployments. Additionally, the file sizes double to a maximum of 2 GB by default. If the --smallfile option is chosen, it goes up to a maximum of 512 MB.
--replSet
This option is used to start the server as a member of the replica set. The value of this arg is the name of the replica set, for example, --replSet repl1. You will learn more on this option in a later recipe where we will start a simple mongo replica set.
--configsvr
This option is used to start the server as a configuration server. The role of the configuration server will be made clearer when we set up a simple sharded environment in a later recipe in this chapter.
--shardsvr
This informs the started mongod process that this server is being started as a shard server. By giving this option, the server also listens to port 27018 instead of the default 27017. We will know more on this option when we start a simple sharded server.
--oplogSize
Oplog is the backbone of replication. It is a capped collection where the data being written to the primary instances is stored in order to be replicated to the secondary instances. This collection resides in a database named local. On initialization of the replica set, the disk space for oplog is preallocated, and the database file (for the local database) is filled with zeros as placeholders. The default value is 5% of the disk space, which should be good enough for most of the cases.
The size of oplog is crucial because capped collections are of a fixed size and they discard the oldest documents in them on exceeding their size, thereby making space for new documents. Having a very small oplog size can result in data being discarded before being replicated to secondary nodes. A large oplog size can result in unnecessary disk space utilization and large duration for the replica set initialization.
For development purposes, when we start multiple server processes on the same host, we might want to keep the oplog size to a minimum value, quickly initiate the replica set, and use minimum disk space.
--storageEngine
Starting with MongoDB 3.0, a new storage engine called Wired Tiger was introduced. The previous (default) storage engine is now called mmapv1. To start MongoDB with Wired Tiger instead of mmapv1, use the wiredTiger value with this option.
--dirctoryperdb
By default, MongoDB's database files are stored in a common directory (as provided in --dbpath). This option allows you to store each database in its own subdirectory in the aforementioned data directory. Having such granular control allows you to have separate disks for each database.
There's more…
For an exhaustive list of options that are available, use the --help or -h option. This list of options is not exhaustive, and we will see some more coming up in later recipes as and when we need them. In the next recipe, we will see how to use a configuration file instead of the command-line arguments.
See also
Connecting to a single node in the Mongo shell with JavaScript
This recipe is about starting the mongo shell and connecting to a MongoDB server. Here we also demonstrate how to load JavaScript code in the shell. Though this is not always required, it is handy when we have a large block of JavaScript code with variables and functions with some business logic in them that is required to be executed from the shell frequently and we want these functions to be available in the shell always.
Getting ready
Although it is possible to run the mongo shell without connecting to the MongoDB server using mongo --nodb, we would rarely need to do so. To start a server on the localhost without much of a hassle, take a look at the first recipe, Installing single node MongoDB, and start the server.
How to do it…
This should print out test to the console.
Now, type the following command in the shell:Note
Note: This book was written with MongoDB version 3.0.2. There is a good chance that you may be using a later version and hence see a different version number in the mongo shell.
How it works…
The JavaScript function that we executed here is of no practical use and is just used to demonstrate how a function can be preloaded on the startup of the shell. There could be multiple functions in the .js file containing valid JavaScript code—possibly some complex business logic.
On executing the mongo command without any arguments, we connect to the MongoDB server running on localhost and listen for new connections on the default port 27017. Generally speaking, the format of the command is as follows:
In cases where there are no arguments passed to the mongo executable, it is equivalent to the passing of the db address as localhost:27017/test.
Let's look at some example values of the db address command-line option and its interpretation:
Now, there are quite a few options available on the mongo client too. We will see a few of them in the following table:
Option
Description
--help or -h
This shows help regarding the usage of various command-line options.
--shell
When the .js files are given as arguments, these scripts get executed and the mongo client will exit. Providing this option ensures that the shell remains running after the JavaScript files execute. All the functions and variables defined in these .js files are available in the shell on startup. As in the preceding case, the sayHello function defined in the JavaScript file is available in the shell for invocation.
--port
The specifies the port of the mongo server where the client needs to connect.
--host
This specifies the hostname of the mongo server where the client needs to connect. If the db address is provided with the hostname, port, and database, then both the --host and --port options need not be specified.
--username or -u
This is relevant when security is enabled for mongo. It is used to provide the username of the user to be logged in.
--password or -p
This option is relevant when security is enabled for mongo. It is used to provide the password of the user to be logged in.
