45,59 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Expert techniques for building fast servers and scalable, real-time network applications with minimal effort; rewritten for Node.js 8 and Node.js 9
About This Book
- Packed with practical examples and explanations, Mastering Node.js contains everything you need to take your applications to the next level.
- Unleash the full potential of Node.js 9 to build real-time and scalable applications.
- Gain in-depth knowledge of asynchronous programming, event loops, and parallel data processing.
- Explore Node's innovative event-non-blocking design, and build professional applications with the help of detailed examples.
Who This Book Is For
This book is targeted at JavaScript developers who want to take an in-depth look at the latest Node.js framework to create faster, scalable, real-time backend applications. Basic JavaScript programming knowledge—and also some previous Node.js development experience—are mandatory to get the best out of this book
What You Will Learn
- Build an Electron desktop app using Node that manages a filesystem
- Explore Streams and understand how they apply to building networked services
- Develop and deploy an SMS-driven customer service application
- Use WebSockets for rapid bi-directional communication
- Construct serverless applications with Amazon Lambda
- Test and debug with headless browsers, CPU profiling, Mocha, Sinon, and more
- Scale applications vertically and horizontally across multiple cores and web services
In Detail
Node.js, a modern development environment that enables developers to write server- and client-side code with JavaScript, thus becoming a popular choice among developers.
This book covers the features of Node that are especially helpful to developers creating highly concurrent real-time applications. It takes you on a tour of Node's innovative event non-blocking design, showing you how to build professional applications. This edition has been updated to cover the latest features of Node 9 and ES6. All code examples and demo applications have been completely rewritten using the latest techniques, introducing Promises, functional programming, async/await, and other cutting-edge patterns for writing JavaScript code. Learn how to use microservices to simplify the design and composition of distributed systems. From building serverless cloud functions to native C++ plugins, from chatbots to massively scalable SMS-driven applications, you'll be prepared for building the next generation of distributed software.
By the end of this book, you'll be building better Node applications more quickly, with less code and more power, and know how to run them at scale in production environments.
Style and approach
Mastering Node.js contains all of the examples and explanations you'll need to build applications in a short amount of time and at a low cost, running rapidly and at scale.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 623
Veröffentlichungsjahr: 2017
Ähnliche
Second Edition
Mastering Node.js
Second Edition
Copyright © 2017 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: November 2013Second edition: December 2017
Production reference: 1271217
ISBN 978-1-78588-896-0
www.packtpub.com
Credits
Authors
Sandro Pasquali Kevin Faaborg
Copy Editor
Shaila Kusanale
Reviewer
Glenn Geenen
Project Coordinator
Devanshi Doshi
Commissioning Editor
Ashwin Nair
Proofreader
Safis Editing
Acquisition Editor
Reshma Raman
Indexer
Rekha Nair
ContentDevelopmentEditor
Onkar Wani
Graphics
Jason Monteiro
Technical Editor
Sachin Sunilkumar
Production Coordinator
Shantanu Zagade
About the Authors
Sandro Pasquali formed Simple.com in 1997, a technology company that sold the world's first JavaScript-based application development framework and was awarded several patents for deployment and advertising technologies that anticipated the future of internet-based software. Node represents, for him, the natural next step in the inexorable march toward the day when JavaScript powers nearly every level of software development.
Sandro has led the design of enterprise-grade applications for some of the largest companies in the world, including Nintendo, Major League Baseball, Bang and Olufsen, LimeWire, AppNexus, Conde Nast, and others. He has displayed interactive media exhibits during the Venice Biennial, won design awards, built knowledge management tools for research institutes and schools, and started and run several start-ups. Always seeking new ways to blend design excellence and technical innovation, he has made significant contributions across all levels of software architecture, from data management and storage tools to innovative user interfaces and frameworks.
He is the author of Deploying Node.js, also by Packt Publishing, which aims to help developers get their work in front of others.
Sandro runs a software development company in New York and trains corporate development teams interested in using Node and JavaScript to improve their products. He spends the rest of his time entertaining his beautiful daughter and his wife.
You can follow him on Twitter at @spasquali, at github.com/sandro-pasquali, and read more of his writing on Node, JavaScript, and software engineering at exploringnode.com.
Kevin Faaborg is a professional software developer and an avid software hobbyist. At Harvard, he learned C programming from visiting professor Brian Kernighan. He witnessed and contributed to how digital technology has shaped music distribution, working first at MTV Networks, then at Lime Wire LLC, and now at Spotify AB, where he designed and started the patent program.
Kevin travels frequently, spending time each year in San Francisco, Colorado, NYC, and Stockholm. You can follow him on Twitter at @zootella, at github.com/zootella, and read more of his writing on Node, JavaScript, and software engineering at exploringnode.com.
About the Reviewer
Glenn Geenen is a Node.js developer who lives in Belgium. He studied digital arts and entertainment at HOWEST (Hogeschool West Vlaanderen). He worked for several years as a mobile (iOS) developer. After discovering Node.js, he embraced web development and became a Node.js consultant for his own company, GeenenTijd.
www.PacktPub.com
For support files and downloads related to your book, please visit www.PacktPub.com. Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www.packtpub.com/mapt
Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.
Why subscribe?
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser
Customer Feedback
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at www.amazon.in/dp/178588896X.
If you'd like to join our team of regular reviewers, you can email us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!
Table of Contents
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Errata
Piracy
Questions
Understanding the Node Environment
Introduction – JavaScript as a systems language
The Unix design philosophy
POSIX
Events for everything
Standard libraries
Extending JavaScript
Events
Modularity
The network
V8, JavaScript, and optimizations
Flags
Optimizing your code
Numbers and tracing optimization/de-optimization
Objects and arrays
Functions
Optimized JavaScript
Help with variables
Arrow functions
String manipulation
The process object
The REPL
Summary
Understanding Asynchronous Event-Driven Programming
Node's unique design
Collaboration
Queueing
Understanding the event loop
Event loop ordering, phases, and priorities
Listening for events
Signals
Child processes
File events
Deferred execution
process.nextTick
setImmediate
Timers
setTimeout
setInterval
unref and ref
Concurrency and errors
Managing concurrency
Callbacks
Promises
async/await
Generators and Iterators
Errors and exceptions
Considerations
Building a Twitter feed using file events
Summary
Streaming Data Across Nodes and Clients
Why use streams?
Exploring streams
Implementing readable streams
Pushing and pulling
Writable streams
Duplex streams
Transforming streams
Using PassThrough streams
Creating an HTTP server
Making HTTP requests
Proxying and tunneling
HTTPS, TLS (SSL), and securing your server
Creating a self-signed certificate for development
Installing a real SSL certificate
The request object
The URL module
The Querystring module
Working with headers
Using cookies
Understanding content types
Handling favicon requests
Handling POST data
Creating and streaming images with Node
Creating, caching, and sending a PNG representation
Summary
Using Node to Access the Filesystem
Directories, and iterating over files and folders
Types of files
File paths
File attributes
Opening and closing files
fs.open(path, flags, [mode], callback)
fs.close(fd, callback)
File operations
fs.rename(oldName, newName, callback)
fs.truncate(path, len, callback)
fs.ftruncate(fd, len, callback)
fs.chown(path, uid, gid, callback)
fs.fchown(fd, uid, gid, callback)
fs.lchown(path, uid, gid, callback)
fs.chmod(path, mode, callback)
fs.fchmod(fd, mode, callback) ----
fs.lchmod(path, mode, callback)
fs.link(srcPath, dstPath, callback)
fs.symlink(srcPath, dstPath, [type], callback)
fs.readlink(path, callback)
fs.realpath(path, [cache], callback)
fs.unlink(path, callback)
fs.rmdir(path, callback)
fs.mkdir(path, [mode], callback)
fs.exists(path, callback)
fs.fsync(fd, callback)
Synchronicity
Moving through directories
Reading from a file
Reading byte by byte
fs.read(fd, buffer, offset, length, position, callback)
Fetching an entire file at once
fs.readFile(path, [options], callback)
Creating a readable stream
fs.createReadStream(path, [options])
Reading a file line by line
The Readline module
Writing to a file
Writing byte by byte
fs.write(fd, buffer, offset, length, position, callback)
Writing large chunks of data
fs.writeFile(path, data, [options], callback)
fs.appendFile(path, data, [options], callback)
Creating a writable stream
fs.createWriteStream(path, [options])
Caveats
Serving static files
Redirecting requests
Location
Content-Location
Implementing resource caching
Handling file uploads
Putting it all together
A simple file browser
Electron
Electron processes
The renderer process
Vue.js
Summary
Managing Many Simultaneous Client Connections
Understanding concurrency
Concurrency is not parallelism
Routing requests
Understanding routes
Using Express to route requests
Using Redis for tracking client state
Storing user data
Handling sessions
Cookies and client state
A simple poll
Authenticating connections
Basic authentication
Handshaking
Using JSON Web Tokens for authentication
Summary
Further reading
Creating Real-Time Applications
Introducing AJAX
Responding to calls
Creating a stock ticker
Bidirectional communication with socket.io
Using the WebSocket API
socket.io
Drawing collaboratively
Listening for Server Sent Events
Using the EventSource API
The EventSource stream protocol
Asking questions and getting answers
Building a collaborative document editing application
Summary
Using Multiple Processes
Node's single-threaded model
The benefits of single-threaded programming
Multithreading is already native and transparent
Creating child processes
Spawning processes
Forking processes
Buffering process output
Communicating with your child
Sending messages to children
Parsing a file using multiple processes
Using the cluster module
Cluster events
Worker object properties
Worker events
Using PM2 to manage multiple processes
Monitoring
Process files
Real-time activity updates of multiple worker results
Summary
Scaling Your Application
When to scale?
Network latency
Hot CPUs
Socket usage
Many file descriptors
Data creep
Tools for monitoring servers
Running multiple Node servers
Forward and reverse proxies
Using the http-proxy module
Deploying a NGINX load balancer on Digital Ocean
Installing and configuring NGINX
Message queues – RabbitMQ
Types of exchanges
Using Node's UDP module
UDP multicasting with Node
Using Amazon Web Services in your application
Authenticating
Errors
Using S3 to store files
Working with buckets
Working with objects
Using AWS with a Node server
Getting and setting data with DynamoDB
Searching the database
Sending mail via SES
Using Twilio to create an SMS bot on Heroku
Using Twilio webhooks to receive and send SMS messages
The switchboard
The ThankYou interface
Summary
Microservices
Why microservices?
From 3-Tiers to 4-Tiers
Monoliths
From monoliths to 3-Tiered architectures
How did this architecture come about?
Service-Oriented Architectures
4-Tiers and microservices
Deploying microservices
Microservices with Seneca
Serverless applications
AWS Lambda
Scaling with Claudia and API Gateway
Installing claudia and deploying a service
Containerized microservices
Getting started with Docker
Creating a Dockerfile
Running containers
Orchestrating Containers with Kubernetes
Creating a basic Kubernetes cluster
Declaring Pod deployments
Summary
Testing Your Application
Why testing is important
Unit tests
Functional tests
Integration tests
Native Node testing and debugging tools
Writing to the console
Formatting console output
The util.format(format, [arg, arg…]) method
The util.inspect(object, [options]) method
The Node debugger
The assert module
Sandboxing
Distinguishing between local scope and execution context
Using compiled contexts
Testing with Mocha, Chai, and Sinon
Mocha
Chai
Sinon
Spies
Stubs
Mocks
Headless testing with Nightmare and Puppeteer
Nightmare
Puppeteer
Testing the terrain
Testing processes, memory, and CPU
Profiling processes
Dumping the heap
Connecting Node to Chrome DevTools
CPU profiling
Live debugging
Summary
Organizing Your Work into Modules
How to load and use modules
The module object
Modules, exports, and module.exports
Modules and caching
How Node handles module paths
Creating a package file
Easy init
Adding scripts to package.json
npm as a build system using custom scripts
Registering package dependencies
Publishing and managing NPM packages
Global installs and binaries
Other repositories
Lockfiles
Creating Your Own C++ Add-ons
Hello World
A calculator
Using NAN
Hello, nan
Asynchronous add-ons
Closing thoughts
Links and resources
Preface
The internet is no longer a collection of static websites to be passively consumed. The browser (and mobile) user has come to expect a much richer and interactive experience. Over the last decade or so, network applications have come to resemble desktop applications. Also, recognition of the social characteristics of information has inspired the development of new kinds of interfaces and visualizations modeling dynamic network states, where the user is viewing change over real time rather than fading snapshots trapped in the past.
Even though our expectations for software have changed, the tools available to us as software developers have not changed much. Computers are faster, and multicore chip architectures are common. Data storage is cheaper, as is bandwidth. Yet we continue to develop with tools designed before billion-user websites and push-button management of cloud-based clusters of virtual machines.
The development of network applications remains an overly expensive and slow process because of this. Developers use different languages, programming styles, complicating code maintenance, debugging, and more. Very regularly, scaling issues arrive too early, overwhelming the ability of what is often a small and inexperienced team. Popular modern software features, such as real-time data, multiplayer games, and collaborative editing spaces, demand systems capable of carrying thousands of simultaneous connections without bending. Yet we remain restricted to frameworks designed to assist us in building CRUD applications, binding a single relational database on a single server to a single user running a multipage website in a browser on a desktop computer.
Node helps developers build more resilient network applications at scale. Built on C++ and bundled with Google's V8 engine, Node is fast, and it understands JavaScript. Node has brought together the most popular programming language in the world and the fastest JavaScript compiler around, and has given easy access to an operating system through C++ bindings. Node represents a change in how network software is designed and built.
What this book covers
Chapter 1, Understanding the Node Environment, gives a brief description of the particular problems Node attempts to solve, their history and roots in the Unix design philosophy, and Node's power as a systems language. We will also learn how to write optimized, modern JavaScript on V8 (the engine powering Node), including a brief tour of the newest features of the language that will help you upgrade your code.
Chapter 2, Understanding Asynchronous Event-Driven Programming, digs deep into the fundamental characteristics of Node's design: event-driven, asynchronous programming. By the end of this chapter, you will understand how events, callbacks, and timers are used in Node as well as how the event loop works to enable high-speed I/O across filesystems, networks, and processes. We'll also learn about modern concurrency modeling constructs, from the default Node callback pattern to Promises, Generators, async/await, and other flow control techniques.
Chapter 3, Streaming Data Across Nodes and Clients, describes how streams of I/O data are knitted through most network software, emitted by file servers or broadcast in response to an HTTP GET request. Here, you will learn how Node facilitates the design, implementation, and composition of network software, using examples of HTTP servers, readable and writable file streams, and other I/O focused Node modules and patterns. You will take a deep dive into the Streams implementation, mastering this fundamental part of the Node stack.
Chapter 4, Using Node to Access the Filesystem, lays out what you need to know when accessing the filesystem with Node, how to create file streams for reading and writing, along with techniques for handling file uploads and other networked file operations. You will also implement a simple file browsing application using Electron.
Chapter 5, Managing Many Simultaneous Client Connections, shows you how Node helps in solving problems accompanying the high-volume and high-concurrency environments that contemporary, collaborative web applications demand. Through examples, learn how to efficiently track user state, route HTTP requests, handle sessions, and authenticate requests using the Redis database and Express web application framework.
Chapter 6, Creating Real-Time Applications, explores AJAX, Server-Sent-Events, and the WebSocket protocol, discussing their pros and cons when building real-time systems, and how to implement each using Node. We finish the chapter by building a collaborative document editing application.
Chapter 7, Using Multiple Processes, teaches how to distribute clusters of Node processes across multicore processors, and other techniques for scaling Node applications. An investigation of the differences between programming in single and multithreaded environments leads to a discussion on how to spawn, fork, and communicate with child processes in Node, including a section on using the PM2 process manager. We also build an analytics tool that records, and displays, the mouse actions of multiple, simultaneous clients connected through a cluster of web sockets.
Chapter 8,Scaling Your Application, outlines some techniques for detecting when to scale, deciding how to scale, and scaling Node applications across multiple servers and cloud services, with examples, including how to use RabbitMQ as a message queue, using NGINX to proxy Node servers, and using Amazon Web Services in your application. The chapter closes with us building a robust customer service application deployed on Heroku, where you will learn how to use the Twilio SMS gateway with Node.
Chapter 9, Microservices, introduces the concept of microservices—small, independent services—and how we got from monolithic and 3-Tier stacks to large fleets of independent services whose collaboration patterns are dynamic. We'll learn how to use Seneca and Node to create an autodiscovering services mesh, AWS Lambda to create serverless applications infinitely scalable in the cloud, and finally, how to create Docker containers and orchestrate their deployment with Kubernetes.
Chapter 10, Testing Your Application, explains how to implement unit, functional, and integration tests with Node. We will go deep, exploring how to use native debugging and testing modules, heap dumps and CPU profiling, eventually building test suites with Mocha and Chai. We'll cover mocks, stubs, and spies with Sinon, live debugging of running Node processes with Chrome DevTools, and how to use tools like Puppeteer to implement headless testing of your UI code.
Appendix A, Organizing Your Work into Modules, gives tips on using the npm package management system. Here, you will learn how to create, publish, and manage packages.
Appendix B, Creating your own C++ Add-ons, provides a brief introduction on how to build your own C++ add-ons and how to use them from within Node. We also cover the new NAN (Native Abstractions for Node) tool and how that can help you with writing cross-platform, future-proofed add-ons.
What you need for this book
You will need to have some familiarity with JavaScript, and have a copy of Node installed on your development machine or server, Version 9.0 or higher. You should know how to install programs on this machine, as you will need to install Redis, along with other libraries like Docker. Having Git installed, and learning how to clone GitHub repositories, will greatly improve your experience.
You should install RabbitMQ so that you can follow with the examples using message queues. The sections on using NGINX to proxy Node servers will, of course, require that you can install and use that web server. To build C++ add-ons, you will need to install the appropriate compiler on your system.
The examples in this book are built and tested within UNIX-based environments (including Mac OS X), but you should be able to run all Node examples on Windows-based operating systems as well. You can obtain installers for your system, and binaries, fromhttp://www.nodejs.org.
Who this book is for
This book is for developers who want to build high-capacity network applications, such as social networks, collaborative document editing environments, real-time data-driven web interfaces, networked games, and other I/O-heavy software. If you're a client-side JavaScript developer, reading this book will teach you how to become a server-side programmer using a language you already know. If you're a C++ hacker, Node is an open source project built using that language, offering you an excellent opportunity to make a real impact within a large and growing community, even gaining fame, by helping to develop this exciting new technology.
This book is also for technical managers and others seeking an explanation of the capabilities and design philosophy of Node. The book is filled with examples of how Node solves the problems modern software companies are facing in terms of high-concurrency, real-time applications pushing enormous volumes of data through growing networks. Node has already been embraced by the enterprise, and you should consider it for your next project.
We are using the bleeding-edge version of Node (9.x at the time of writing). This is the only book you need to be ready for in the next few years as Node continues its march through the enterprise.
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail [email protected], and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors .
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Downloading the example code
You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
You can download the code files by following these steps:
Log in or register to our website using your e-mail address and password.
Hover the mouse pointer on the
SUPPORT
tab at the top.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box.
Select the book for which you're looking to download the code files.
Choose from the drop-down menu where you purchased this book from.
Click on
Code Download
.
You can also download the code files by clicking on the Code Files button on the book's webpage at the Packt Publishing website. This page can be accessed by entering the book's name in the Search box. Please note that you need to be logged in to your Packt account.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR / 7-Zip for Windows
Zipeg / iZip / UnRarX for Mac
7-Zip / PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Mastering-Node.js-Second-Edition. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at [email protected] with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
Questions
If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.
Understanding the Node Environment
The Unix design philosophy
As a network application scales, the volume of information it must recognize, organize, and maintain increases. This volume, in terms of I/O streams, memory usage, and processor load, expands as more clients connect. This expansion of information volume also burdens the software developer. Scaling issues appear, usually demonstrating a failure to accurately predict the behavior of a large system from the behavior of its smaller predecessors:
Can a data layer designed for storing a few thousand records accommodate a few million?
Are the algorithms used to search a handful of records efficient enough to search many more?
Can this server handle 10,000 simultaneous client connections?
The edge of innovation is sharp and cuts quickly, presenting less time for deliberation precisely when the cost of error is magnified. The shape of objects comprising the whole of an application becomes amorphous and difficult to understand, particularly as ad hoc modifications are made, reactively, in response to dynamic tension in the system. What is described in a specification as a small subsystem may have been patched into so many other systems, that its actual boundaries are misunderstood. When this happens, it becomes impossible to accurately trace the outline of the composite parts of the whole.
Eventually, an application becomes unpredictable. It is dangerous when one cannot predict all future states of an application, or the side effects of change. Any number of servers, programming languages, hardware architectures, management styles, and so on, have attempted to subdue the intractable problem of risk following growth, of failure menacing success. Oftentimes, systems of even greater complexity are sold as the cure. The hold that any one person has on information is tenuous. Complexity follows scale; confusion follows complexity. As resolution blurs, errors happen.
Node chose clarity and simplicity instead, echoing a philosophy from decades earlier:
From their experiences creating and maintaining Unix, Ken Thompson and Dennis Ritchie came up with a philosophy for how people should best build software. Using this philosophy as his guide, Ryan Dahl made a number of decisions in the design of Node:
Node's design favors simplicity over complexity
Node uses familiar POSIX APIs, rather than attempting an improvement
Node does everything with events, and doesn't need threads
Node leverages the existing C libraries, rather than trying to reimplement their functionality
Node favors text over binary formats
Text streams are the language of Unix programs. JavaScript got good at manipulating text from its beginning as a web scripting language. It's a natural fit.
POSIX
POSIX, the Portable Operating System Interface, defines the standard APIs for Unix. It's adopted in Unix-based operating systems and beyond. The IEEE created and maintains the POSIX standard to enable systems from different manufacturers to be compatible. Write your C program using POSIX APIs on your laptop running macOS, and you'll have an easier time later building it on a Raspberry Pi.
As a common denominator, POSIX is old, simple, and most importantly, well-known to developers of all stripes. To make a new directory in a C program, use this API:
int mkdir(const char *path, mode_t mode);
And here it is in Node:
fs.mkdir(path[, mode], callback)
The Node documentation for the filesystem module starts out by telling the developer, there's nothing new here:
For Node, Ryan Dahl implemented proven POSIX APIs, rather than trying to come up with something on his own. While such an attempt might be better in some ways, or some situations, it would lose the instant familiarity that POSIX gives to new Node developers trained in other systems.
In choosing POSIX for the API, Node is in no way limited to the standards from the 1970s. It's easy for anyone to write their own module that calls down to Node's API, while presenting a different one upwards. These fancier alternatives can then compete in a Darwinian quest to prove themselves better than POSIX.
Standard libraries
Node is built on standard open source C libraries. For example, the TLS and SSL protocols are implemented by OpenSSL. More than just adopting an API, the C source code of OpenSSL is included and complied into Node. When your JavaScript program hashes a cryptographic key, it's not JavaScript that's actually doing the work. Your JavaScript, run by Node, has called down to the C code of OpenSSL. Essentially, you are scripting the native library.
This design choice of using the existing and proven open source libraries helped Node in a number of ways:
It meant that Node could arrive on the scene very rapidly, with the core set of functionality systems programmers needed and expected already there
It ensures performance, reliability, and security continues to match the libraries
It also didn't break cross-platform use, as all of these C libraries have been written and maintained to compile for different architectures for years
Previous platforms and languages have made a different choice in trying to achieve software portability. The 100% Pure Java™ Standard, for instance, was a Sun Microsystems initiative to promote the development of portable applications. Rather than leveraging the existing code in a hybrid stack, it encouraged developers to rewrite everything in Java. Developers had to keep features, performance, and security up to the standard by writing and testing new code. Node, on the other hand, picked a design that gets this all for free.
Modularity
In his book, The Art of Unix Programming, Eric Raymond proposed the Rule of Modularity:
Large systems are hard to reason about, especially when the boundaries of internal components are fuzzy, and the interactions between them are complex. This principle of building large systems out of small, simple, and loosely-coupled pieces is a good idea for software and beyond. Physical manufacturing, management theory, education, and government, all have benefited from this design philosophy.
When developers began employing JavaScript for larger and more complex software challenges, they encountered this challenge. There was not yet a good way (and later, no common standard way) to assemble a JavaScript program from smaller ones. For example, you've probably seen HTML pages with tags like these at the top:
<head><script src="fileA.js"></script><script src="fileB.js"></script><script src="fileC.js"></script><script src="fileD.js"></script>...</head>
This works, but leads to a number of problems:
The page must declare all potential dependencies before any are needed or used. If, while running, your program encounters a situation where it needs an additional dependency, dynamically loading another module is possible, but a separate hack.
The scripts are not encapsulated. Code in every file writes to the same global object. Adding a new dependency may break an earlier one because of a name collision.
fileA
cannot address
fileB
as a collection. An addressable context like
fileB.function1
isn't available.
The <script> tag would be a nice place for useful module services such as dependency awareness and version control, but it doesn't have these features.
These difficulties and dangers made creating and using JavaScript modules feel more treacherous than effortless. A good module system with features like encapsulation and versioning can reverse this, encouraging code organization and sharing, and leading to a robust ecosystem of high-quality open source software components.
JavaScript needed a standard way to load and share discreet program modules, and found one in 2009 with the CommonJS Modules specification. Node follows this specification, making it easy to define and share bits of reusable code called modules or packages.
Choosing a delightfully simple design, a package is just a directory of JavaScript files. Metadata about the package, such as its name, version, and software license, lives in an additional file named package.json. The JSON contents of this file are easily both human and machine-readable. Let's take a look:
{ "name": "mypackage1", "version": "0.1.2", "dependencies": { "jquery": "^3.1.0", "bluebird": "^3.4.1", }, "license": "MIT"}
This package.json defines a package named mypackage1, which depends on two other packages: jQuery and Bluebird. Alongside the package names is a version number. Version numbers follow the Semantic Versioning (SemVer) rules, with a pattern like Major.Minor.Patch. Looking at the incremented version numbers of a package your code has been using, here's what that means:
Major:
There's a change in the purpose or outcome of the API. If your code calls an updated function, it may break or produce an unintended result. Figure out what's changed, and determine if it affects your code.
Minor:
The package has added functionality, but remains compatible. Run all your tests, and you're good to go. Check out the documentation if
you're curious
, as there might be new, more advanced parts of the API alongside the functions and objects you're familiar with.
Patch:
The package fixed a bug, improved performance, or refactored a little. Run all your tests, and you're good to go.
Packages enable the construction of large systems from many small, interdependent systems. Perhaps even more importantly, packages encourage sharing. More detailed information about SemVer is available in Appendix A, Organizing Your Work Into Modules, where npm and packages are discussed in more depth.
CommonJS gave JavaScript packages. With packages, the next thing JavaScript needed was a package manager. Node provided one with npm.
A registry of packages, npm is accessible in two ways. First, at the website www.npmjs.com, you can link to and search for packages, essentially shopping for the right one. Stats that count how many times a package has been downloaded in the last day, week, and month show popularity and usage. Most packages link to a developer profile page and open source code on GitHub, so you can see the code, visualize recent development activity, and judge the reputations of the authors and contributors.
The second way to access npm is through the command-line tool npm, which is installed with Node. Using npm as a traditional package manager for your workstation, you can install packages globally, creating new command-line tools on your shell's path. npm also knows how to create, read, and edit package.json files, and can start you out with a new, empty Node package, add the dependencies it needs, download all the code, and keep everything up to date.
In no small part, Node's success is due to the number and quality of packages available to you as a Node developer.
The key design philosophy to follow is this: build programs out of packages where possible, and share those packages when possible. The shape of your applications will be clearer and easier to maintain. Importantly, the efforts of thousands of other developers can be linked into applications via npm, directly by inclusion, and indirectly as shared packages are tested, improved, refactored, and repurposed by members of the Node community.
