39,59 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Learn the A to Z of building excellent Splunk applications with the latest techniques using this comprehensive guide
About This Book
- This is the most up-to-date book on Splunk 6.3 for developers
- Get ahead of being just a Splunk user and start creating custom Splunk applications as per your needs
- Your one-stop-solution to Splunk application development
Who This Book Is For
This book is for those who have some familiarity with Splunk and now want to learn how to develop an efficient Splunk application. Previous experience with Splunk, writing searches, and designing basic dashboards is expected.
What You Will Learn
- Implement a Modular Input and a custom D3 data visualization
- Create a directory structure and set view permissions
- Create a search view and a dashboard view using advanced XML modules
- Enhance your application using eventtypes, tags, and macros
- Package a Splunk application using best practices
- Publish a Splunk application to the Splunk community
In Detail
Splunk provides a platform that allows you to search data stored on a machine, analyze it, and visualize the analyzed data to make informed decisions. The adoption of Splunk in enterprises is huge, and it has a wide range of customers right from Adobe to Dominos. Using the Splunk platform as a user is one thing, but customizing this platform and creating applications specific to your needs takes more than basic knowledge of the platform.
This book will dive into developing Splunk applications that cater to your needs of making sense of data and will let you visualize this data with the help of stunning dashboards.
This book includes everything on developing a full-fledged Splunk application right from designing to implementing to publishing. We will design the fundamentals to build a Splunk application and then move on to creating one. During the course of the book, we will cover application data, objects, permissions, and more. After this, we will show you how to enhance the application, including branding, workflows, and enriched data. Views, dashboards, and web frameworks are also covered.
This book will showcase everything new in the latest version of Splunk including the latest data models, alert actions, XML forms, various dashboard enhancements, and visualization options (with D3). Finally, we take a look at the latest Splunk cloud applications, advanced integrations, and development as per the latest release.
Style and approach
This book is an easy-to-follow guide with lots of tips and tricks to help you master all the concepts necessary to develop and deploy your Splunk applications.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 232
Veröffentlichungsjahr: 2016
Ähnliche
Table of Contents
Splunk Developer's Guide Second Edition
Splunk Developer's Guide Second Edition
Copyright © 2016 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: May 2015
Second edition: January 2016
Production reference: 1190116
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78588-237-1
www.packtpub.com
Credits
Author
Kyle Smith
Reviewer
Marco Scala
Commissioning Editor
Veena Pagare
Acquisition Editor
Vinay Argekar
Content Development Editor
Amey Varangaonkar
Technical Editor
Taabish Khan
Copy Editor
Trishya Hajare
Project Coordinator
Suzanne Coutinho
Proofreader
Safis Editing
Indexer
Hemangini Bari
Graphics
Abhinash Sahu
Production Coordinator
Shantanu N. Zagade
Cover Work
Shantanu N. Zagade
About the Author
Kyle Smith is a self-proclaimed geek and has been working with Splunk extensively since 2010. He enjoys integrating Splunk with new sources of data and types of visualization. He has spoken numerous times at the Splunk User Conference (most recently in 2014 on Lesser Known Search Commands) and is an active contributor to the Splunk Answers community and also to the #splunk IRC channel. He was awarded membership into the SplunkTrust as a founding member. He has published several Splunk Apps and add-ons to Splunkbase, the Splunk community's premier Apps and add-ons platform. He has worked in both higher education and private industry; he is currently working as an integration developer for Splunk's longest running professional services partner. He lives in central Pennsylvania with his family.
I'd like to thank my wife who most graciously put up with all my BS during the writing of this book. Without her, this effort is meaningless.
About the Reviewer
Marco Scala has been working for more than 15 years delivering solutions to large enterprise customers, first in the APM and J2EE fields and, since 2009, in the fields of operational intelligence and Splunk. He has provided consultancy for big Splunk installations for major customers, focusing on the best and most effective solutions for each different customer's needs. Since 2012, he's also a certified Splunk trainer.
In the last few years, Marco's major focus has been to get Splunk customers to gain the maximum value from their IT data and provide the business a better view and insight. Big Data is another major field of interest, and his next challenge is using Splunk to give customers useful insights and a practical implementation and exploitation of Big Data.
www.PacktPub.com
Support files, eBooks, discount offers, and more
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at <[email protected]> for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www2.packtpub.com/books/subscription/packtlib
Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.
Why subscribe?
Free access for Packt account holders
If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access.
Instant updates on new Packt books
Get notified! Find out when new books are published by following @PacktEnterprise on Twitter or the Packt Enterprise Facebook page.
Preface
Splunk is awesome. Not only can you consume virtually any data with it, you can also extend and integrate Splunk with virtually any external system. Splunk uses sets of configurations that are referred to as applications or add-ons, which is the primary focus of this book. Leveraging these applications and add-ons is what gives Splunk its unique ability to extend, learn, analyze, and visualize information.
Splunk helps users to determine the root cause of a failure, a quick overview of system health, and dive deep into SQL statements and messages, just to name a few. The aggregation and centralization of log and event management is a growing trend in the Big Data space. By leveraging the combined intelligence gathered from correlating disparate sets of data, businesses or individuals can make data-based decisions. This book will help a Splunk developer, or even just a curious end user, to develop different methods of consuming new data, design new types of visualization, or even just offer tips and tricks that help the software development lifecycle.
Overview of what this book isn't
Most developer guides will tell you what their book is and/or does. We aim to explain what this book isn't, and allow you to fill in the rest with your imagination! Thus, proceed to this list:
Splunk basics will not be covered. These include concepts such as searching (finding data, using timecharts, stats, some eval commands, and so on), reporting (making basic pie charts or line charts via the GUI), data inputs (basic file monitoring, TCP and UDP inputs, Splunk forwarders, and so on), and configurations (GUI and web-based configuration editing), to name a few. Creating dashboards via the GUI? Nope. Python will be discussed and sample code will be provided, but this book will not cover the nuances of the code, nor will it teach you Python syntax. We will not cover statistical computation, other than how to practically apply some basic math to create value-based visualizations. We will not cover using the SDKs (software development kits) being used in custom Splunk applications that are external to Splunk (for example, Angular, PHP, .NET, and others). These are out of the scope of this book. Free as in beer? Nope, the choice of hops, starch, and oak-barrel aging for the creation of beer will not be discussed, but rather consumed during the writing and/or reading of this book.
Note
Unless otherwise stated, this book uses Splunk version 6.3 as the development environment.
What this book is
This book will guide you through many the different areas of Splunk App and add-on creation. We will start by looking at the design aspects of an App or add-on, how to create them, what knowledge objects are available for use within the App, ways to enhance your App with metadata and external data, and some basic views and dashboards. From there, we will move into the Splunk Web Framework, modular inputs, jQuery, web framework programs, and then packaging and publishing Apps and add-ons. At the tail end, we will highlight some areas of the Splunk community that prove to be very useful.
Assumptions
There are a few basic assumptions that we are going to make. Having purchased or otherwise obtained this book, we assume that you are interested in developing with Splunk, and have a basic understanding of Splunk and how to navigate around the software. Knowledge of saving searches, reports, and basic dashboarding is a must, since most concepts and examples will be built upon the basics. We also assume that you have basic knowledge of HTML, CSS, JS, and some XML. Here, XML will be limited to the Splunk XML framework specifically. We would also recommend you to have knowledge of, or proficiency in, Python, RequireJS, and other web technologies such as Bower, npm, and Gulp. We will demonstrate how to use these web technologies within a Splunk application.
What this book covers
Chapter 1, Application Design Fundamentals, discusses fundamental questions and considerations before diving into an App or add-on configuration.
Chapter 2, Creating Applications, discusses the basic methods of App and add-on creation, along with an explanation of the structure of an App or add-on.
Chapter 3, Enhancing Applications, discusses a few different configurations that help to enrich your data with Splunk knowledge objects, along with some basic App and add-on branding guidelines.
Chapter 4, Basic Views and Dashboards, goes through the basics of SimpleXML dashboard creation and development.
Chapter 5, The Splunk Web Framework, details the various SplunkJS Stack components and shows examples of how to use them within an HTML dashboard.
Chapter 6, Advanced Integrations and Development, reviews modular inputs, data models, the KV Store, and modular D3 visualizations.
Chapter 7, Packaging Applications, lists the items needed to package an App or add-on, in order to get it ready for publishing.
Chapter 8, Publishing Applications, explains step by step how to upload an App to Splunkbase, and includes some information on the great support community.
What you need for this book
To take full advantage of all the examples and code contained within this book, you should have the following items:
Who this book is for
This book will benefit both the casual Splunker and the experienced professional alike. Whether you are just starting Splunk Apps or add-on development, or have been developing for years, this book has tips and tricks to help with developing new integrations and Splunk Apps and add-ons. Even for quick modular input, this book provides quick tutorials on common integration techniques and code examples.
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Downloading the example code
You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
Questions
If you have a problem with any aspect of this book, you can contact us at <[email protected]>, and we will do our best to address the problem.
Chapter 1. Application Design Fundamentals
Hello there, Splunk developer! If you are like us, we know you have a love of Splunk and all of the endless possibilities that present themselves! The Big Data world is exploding around us, and it always feels like a tireless battle when keeping up to date with advances in technologies, platforms, and concepts. Here, we will discuss none of those. This book is dedicated solely to Splunk and the development of applications for Splunk. Onward and upward!
What is a Splunk application?
All that being said, let's talk Splunk applications. A Splunk application is nothing more than a structured set of configurations and assets used to achieve an end goal of data collection, indexing, and visualization. Furthermore, in order to create a valid Splunk application, you must include the ability to navigate. Without navigation within the application, you would be working on an add-on. According to Splunk, applications:
Note
This is taken from http://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/AppIntro.
Why applications?
Applications allow us to quickly share configurations, focus on the context of available data, limit data access to specific individuals or groups, and organize similar dashboards and views into a cohesive presentation of data within Splunk. Sharing applications can be as easy as just zipping it up and sending it out. Splunk applications could be said to be open source, due to the fact that almost all of the configurations, custom scripts, and any other knowledge object contained within the applications, are readable on the filesystem. This allows for customization for an individual instance while maintaining an overall master configuration.
Definitions
To get started, we should define a few naming conventions typically used when naming applications. Note that while we will use these naming conventions as the best practice, your application can really be named anything at all, which may conflict with other applications of the same name, or violate Splunk usage agreements or publishing guidelines. In particular, the name Splunk cannot be present in your application or add-on name. Additionally, in the past, Splunk has referred to add-ons as technology add-ons, and has since moved to just add-ons. The following list of add-on types is our way to distinguish the different uses of each add-on:
Tip
Follow the Splunk application design guidelines. Using a custom naming scheme may cause conflicts.
Thus end the official naming conventions as normally seen in a Splunk installation. We will now discuss some other naming conventions that have been found to help in the wild west of various Splunk installations. These two naming conventions are of the author's own design, which have helped in some of his deployments:
While this may not be a complete list of naming conventions, it should be enough to recognize any that are seen in the wild. An additional aspect of the naming conventions that we recommend is the addition of company information. This will help your Splunk admins differentiate between Splunk add-ons and custom add-ons. Just as an example, let's say you built a TA for Cisco, specific to your company (the ACME company). Splunk's provided add-on is entitled TA-cisco, but you don't want to modify a vendor's offering. So, your new add-on's name could be A-ACME-TA-cisco. This gives you two things: an easy-to-see custom TA that relates to Cisco and the ability to override any TA-cisco settings based on application precedence.
Let's discuss application precedence for a moment. Splunk uses a merged configuration when applying configurations that are installed via the applications. The methodology that Splunk chose to implement conflict resolution is pretty simple. There are two different methods of precedence. The first is directory structure. If you have an input located in the default folder of an application (more on default in the later chapters), you can place a matching configuration in the local folder of the application to override the default configuration. The same method is applied to the applications themselves. Splunk uses the ASCII values of the names to determine precedence. On *nix, you can sort the applications in the apps folder of Splunk using the LC_COLLATE=C ls command. This will show you the ASCII-sorted order of the applications, and the first in the list will be highest priority. A has a higher priority than Z, but Z has a higher priority than a. So, the A at the beginning of the add-on name gives your add-on the highest precedence, so you can override any setting as needed.
Note
From this point forward, both Splunk applications and add-ons will be referred to formally as Apps purely as a convenience.
Designing the App
So you've decided that you need an App? Congratulations! Now that you know that you need one, you need to decide on a few more items as well. It is important to do a little bit of planning, as even the simplest Apps can evolve into super-complicated Apps, with dashboards, saved searches, workflows, and more. Never assume "well, this'll just be a quick development", as, most of the time, it is not.
Identifying the use case
First and foremost, try to determine the scope of your App. Once you have the scope planned out, try to limit the amount of scope creep that occurs, if possible. You may just be trying to perform extractions on your data, and if that is your current end goal, stop there. Don't try to build a full-blown suite on your first attempt. Build the IA, then the TA, and then move on from there. Ask yourself these questions as you try to determine your scope:
These questions can help you spark an idea of what internal resources would need to be engaged, as well as any kind of documentation and educational requirements.
Identifying what you want to consume
Once you have determined the scope of the App, you will need to decide how and from where you will consume the data. Getting data into Splunk can happen in a very wide variety of ways. There is no set manner of input that will work on all data sources. You may have to develop a new script or modular input. Being aware of where your data is coming from is the key to getting it consumed correctly the first time. A few questions you may ask yourself could be:
There is a lot of data out in the wild, but not all of it may be relevant to your use case. You may find that of a service that has 100 endpoints available for data collection, you only need 10. Not only will you save on license usage, but your indexers will thank you for it as well.
Identifying what you want to brand
Another key thought process in App development is how far you want to brand your App. Splunk has a very robust architecture and framework, providing you with the ability to customize your Apps extensively. You can override any individual piece of CSS and extend SplunkJS Stack to include any number of different visualizations or third-party libraries. Additional questions you might ponder on would include:
There are so many options available to brand your App, but all customizations should conform to the Splunk branding guidelines for developers. Visit http://www.splunk.com/view/SP-CAAAFT9 to read through Splunk's guidelines.
Identifying what you want to display
Once you
