Cognitive Computing and Big Data Analytics E-Book

Judith S. Hurwitz

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Veröffentlichungsjahr: 2015

Beschreibung

A comprehensive guide to learning technologies that unlock the value in big data

Cognitive Computing provides detailed guidance toward building a new class of systems that learn from experience and derive insights to unlock the value of big data. This book helps technologists understand cognitive computing's underlying technologies, from knowledge representation techniques and natural language processing algorithms to dynamic learning approaches based on accumulated evidence, rather than reprogramming. Detailed case examples from the financial, healthcare, and manufacturing walk readers step-by-step through the design and testing of cognitive systems, and expert perspectives from organizations such as Cleveland Clinic, Memorial Sloan-Kettering, as well as commercial vendors that are creating solutions. These organizations provide insight into the real-world implementation of cognitive computing systems. The IBM Watson cognitive computing platform is described in a detailed chapter because of its significance in helping to define this emerging market. In addition, the book includes implementations of emerging projects from Qualcomm, Hitachi, Google and Amazon.

Today's cognitive computing solutions build on established concepts from artificial intelligence, natural language processing, ontologies, and leverage advances in big data management and analytics. They foreshadow an intelligent infrastructure that enables a new generation of customer and context-aware smart applications in all industries.

Cognitive Computing is a comprehensive guide to the subject, providing both the theoretical and practical guidance technologists need.

• Discover how cognitive computing evolved from promise to reality
• Learn the elements that make up a cognitive computing system
• Understand the groundbreaking hardware and software technologies behind cognitive computing
• Learn to evaluate your own application portfolio to find the best candidates for pilot projects
• Leverage cognitive computing capabilities to transform the organization

Cognitive systems are rightly being hailed as the new era of computing. Learn how these technologies enable emerging firms to compete with entrenched giants, and forward-thinking established firms to disrupt their industries. Professionals who currently work with big data and analytics will see how cognitive computing builds on their foundation, and creates new opportunities. Cognitive Computing provides complete guidance to this new level of human-machine interaction.

Leseprobe

Contents

Introduction

Human Intelligence + Machine Intelligence

Putting the Pieces Together

The Book’s Focus

Overview of the Book and Technology

How This Book Is Organized

Chapter 1 The Foundation of Cognitive Computing

Cognitive Computing as a New Generation

The Uses of Cognitive Systems

What Makes a System Cognitive?

Gaining Insights from Data

Artificial Intelligence as the Foundation of Cognitive Computing

Understanding Cognition

Two Systems of Judgment and Choice

Understanding Complex Relationships Between Systems

The Elements of a Cognitive System

Summary

Chapter 2 Design Principles for Cognitive Systems

Components of a Cognitive System

Building the Corpus

Bringing Data into the Cognitive System

Machine Learning

Hypotheses Generation and Scoring

Presentation and Visualization Services

Summary

Chapter 3 Natural Language Processing in Support of a Cognitive System

The Role of NLP in a Cognitive System

Semantic Web

Applying Natural Language Technologies to Business Problems

Summary

Chapter 4 The Relationship Between Big Data and Cognitive Computing

Dealing with Human-Generated Data

Defining Big Data

The Architectural Foundation for Big Data

Analytical Data Warehouses

Hadoop

Data in Motion and Streaming Data

Integration of Big Data with Traditional Data

Summary

Chapter 5 Representing Knowledge in Taxonomies and Ontologies

Representing Knowledge

Defining Taxonomies and Ontologies

Explaining How to Represent Knowledge

Models for Knowledge Representation

Implementation Considerations

Summary

Chapter 6 Applying Advanced Analytics to Cognitive Computing

Advanced Analytics Is on a Path to Cognitive Computing

Key Capabilities in Advanced Analytics

Using Advanced Analytics to Create Value

Impact of Open Source Tools on Advanced Analytics

Summary

Chapter 7 The Role of Cloud and Distributed Computing in Cognitive Computing

Leveraging Distributed Computing for Shared Resources

Why Cloud Services Are Fundamental to Cognitive Computing Systems

Characteristics of Cloud Computing

Cloud Computing Models

Delivery Models of the Cloud

Managing Workloads

Security and Governance

Data Integration and Management in the Cloud

Summary

Chapter 8 The Business Implications of Cognitive Computing

Preparing for Change

Advantages of New Disruptive Models

What Does Knowledge Mean to the Business?

The Difference with a Cognitive Systems Approach

Meshing Data Together Differently

Using Business Knowledge to Plan for the Future

Answering Business Questions in New Ways

Building Business Specific Solutions

Making Cognitive Computing a Reality

How a Cognitive Application Can Change a Market

Summary

Chapter 9 IBM’s Watson as a Cognitive System

Watson Defined

Advancing Research with a “Grand Challenge”

Preparing Watson for

Jeopardy!

Preparing Watson for Commercial Applications

The Components of DeepQA Architecture

Summary

Chapter 10 The Process of Building a Cognitive Application

The Emerging Cognitive Platform

Defining the Objective

Defining the Domain

Understanding the Intended Users and Defining their Attributes

Defining Questions and Exploring Insights

Creating and Refining the Corpora

Training and Testing

Summary

Chapter 11 Building a Cognitive Healthcare Application

Foundations of Cognitive Computing for Healthcare

Constituents in the Healthcare Ecosystem

Learning from Patterns in Healthcare Data

Building on a Foundation of Big Data Analytics

Cognitive Applications across the Healthcare Ecosystem

Starting with a Cognitive Application for Healthcare

Using Cognitive Applications to Improve Health and Wellness

Using a Cognitive Application to Enhance the Electronic Medical Record

Using a Cognitive Application to Improve Clinical Teaching

Summary

Chapter 12 Smarter Cities: Cognitive Computing in Government

How Cities Have Operated

The Characteristics of a Smart City

The Rise of the Open Data Movement Will Fuel Cognitive Cities

The Internet of Everything and Smarter Cities

Understanding the Ownership and Value of Data

Cities Are Adopting Smarter Technology Today for Major Functions

Smarter Approaches to Preventative Healthcare

Building a Smarter Transportation Infrastructure

Using Analytics to Close the Workforce Skills Gap

Creating a Cognitive Community Infrastructure

The Next Phase of Cognitive Cities

Summary

Chapter 13 Emerging Cognitive Computing Areas

Characteristics of Ideal Markets for Cognitive Computing

Vertical Markets and Industries

Summary

Chapter 14 Future Applications for Cognitive Computing

Requirements for the Next Generation

Technical Advancements That Will Change the Future of Cognitive Computing

What the Future Will Look Like

Emerging Innovations

Summary

Glossary

Title page

Copyright

Dedication

About the Technical Editors

About the Authors

Acknowledgments

EULA

List of Tables

Chapter 4

Table 4.1

Chapter 5

Table 5.1

Chapter 6

Table 6.1

Table 6.2

Table 6.3

Chapter 9

Table 9.1

Table 9.2

Table 9.3

Chapter 10

Table 10.1

Table 10.2

Table 10.3

Chapter 11

Table 11.1

Table 11.2

Table 11.3

List of Illustrations

Chapter 1

Figure 1.1

Interaction between intuitive thinking and deep analysis

Figure 1.2

Elements of a cognitive system

Chapter 2

Figure 2.1

Architecture of a cognitive system

Figure 2.2

The continuous machine learning process

Chapter 4

Figure 4.1

Big data technology stack

Figure 4.2

Example of a Hadoop cluster

Figure 4.3

Workflow and data movement in a small Hadoop cluster

Chapter 5

Figure 5.1

Motor vehicle types 

Figure 5.2

Representing a chess game

Figure 5.3

Automotive diagnostics and repair

Figure 5.4

Taxonomy of nature

Figure 5.5

Taxonomies Evolve—Autism in the Diagnostic and Statistical Manual of Mental Disorders

Chapter 6

Figure 6.1

Converging technologies: analytics and artificial intelligence

Figure 6.2

Refining raw data to create business value

Chapter 7

Figure 7.1

Foundations of a cloud architecture

Figure 7.2

Hybrid cloud architecture

Chapter 9

Figure 9.1

IBM Watson DeepQA Architecture

Figure 9.2

Parsing two sentences using English Slot Grammar

Figure 9.3

Hypothesis Generation in Watson’s DeepQA Architecture

Chapter 10

Figure 10.1

Improving accuracy of the models

Chapter 11

Figure 11.1

Foundations of cognitive computing for healthcare

Figure 11.2

Healthcare ecosystems data sources

Figure 11.3

Welltok training architecture

Figure 11.4

Welltok high-level architecture and data flow: data flow content acquisition

Chapter 12

Figure 12.1

Foundations of cognitive computing for smarter cities

Figure 12.2

Data/knowledge management for cities

Figure 12.3

Modern city data sources and managers

Chapter 14

Figure 14.1

The life cycle of knowledge management

Guide

Cover

Table of Contents

Chapter

Pages

xvii

xviii

xix

xx

xxi

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

221

222

223

224

225

226

227

228

229

230

231

232

233

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

251

252

253

254

255

256

257

258

259

Introduction

With huge advancements in technology in the last 30 years, the ability to gain insights and actions from data hasn’t changed much. In general, applications are still designed to perform predetermined functions or automate business processes, so their designers must plan for every usage scenario and code the logic accordingly. They don’t adapt to changes in the data or learn from their experiences. Computers are faster and cheaper, but not much smarter. Of course, people are not much smarter than they were 30 years ago either. That is about to change, for humans and machines. A new generation of an information system is emerging that departs from the old model of computing as process automation to provide a collaborative platform for discovery. The first wave of these systems is already augmenting human cognition in a variety of fields. Acting as partners or collaborators for their human users, these systems may derive meaning from volumes of natural language text and generate and evaluate hypotheses in seconds based on analysis of more data than a person could absorb in a lifetime. That is the promise of cognitive computing.

Human Intelligence + Machine Intelligence

Traditional applications are good at automating well-defined processes. From inventory management to weather forecasting, when speed is the critical factor in success and the processes are known in advance, the traditional approach of defining requirements, coding the logic, and running an application is adequate. That approach fails, however, when we need to dynamically find and leverage obscure relationships between data elements, especially in areas in which the volume or complexity of the data increases rapidly. Change, uncertainty, and complexity are the enemies of traditional systems.

Cognitive computing—based on software and hardware that learns without reprogramming and automates cognitive tasks—presents an appealing new model or paradigm for application development. Instead of automating the way we already conduct business, we begin by thinking about how to augment the best of what the human brain can do with new application capabilities. We start with processes for ingesting data from inside and outside the enterprise, and add functions to identify and evaluate patterns and complex relationships in large and sometimes unstructured data sets, such as natural language text in journals, books, and social media, or images and sounds. The result is a system that can support human reasoning by evaluating data in context and presenting relevant findings along with the evidence that justifies the answers. This approach makes users more efficient—like a traditional application—but it also makes them more effective because parts of the reasoning and learning processes have been automated and assigned to a tireless, fast collaborator.

Like the fundamentals of traditional computing, the concepts behind smart machines are not new. Even before the emergence of digital computers, engineers and scientists speculated about the development of learning machines that could mimic human problem solving and communications skills. Although some of the concepts underlying the foundation technologies—including machine intelligence, computational linguistics, artificial intelligence, neural networks, and expert systems—have been used in conventional solutions for a decade or more, we have seen only the beginning. The new era of intelligent computing is driven by the confluence of a number of factors:

The growth in the amount of data created by systems, intelligent devices, sensors, videos, and such

The decrease in the price of computer storage and computing capabilities

The increasing sophistication of technology that can analyze complex data as fast as it is produced

The in-depth research from emerging companies across the globe that are investigating and challenging long-held beliefs about what the collaboration of humans and machines can achieve

Putting the Pieces Together

When you combine Big Data technology and the changing economics of computing with the need for business and industry to be smarter, you have the beginning of fundamental change. There are many names for this paradigm shift: machine learning, cognitive computing, artificial intelligence, knowledge management, and learning machines. But whatever you call it, this change is actually the integration of the best of human knowledge about the world with the awesome power of emerging computational systems to interpret massive amounts of a variety of types of data at an unprecedented rate of speed. But it is not enough to interpret or analyze data. Emerging solutions for cognitive computing must gather huge amounts of data about a specific topic, interact with subject matter experts, and learn the context and language of that subject. This new cognitive era is in its infancy, but we are writing this book because of the significant and immediate market potential for these systems. Cognitive computing is not magic. It is a practical approach to supporting human problem solving with learning machines that will change markets and industries.

The Book’s Focus

This book takes a deep look at the elements of cognitive computing and how it is used to solve problems. It also looks at the human efforts involved in evolving a system that has enough context to interpret complex data and processes in areas such as healthcare, manufacturing, transportation, retail, and financial services. These systems are designed as collaboration between machines and humans. The book examines various projects designed to help make decision making more systematic. How do expertly trained and highly experienced professionals leverage data, prior knowledge, and associations to make informed decisions? Sometimes, these decisions are the right ones because of the depth of knowledge. Other times, however, the decisions are incorrect because the knowledgeable individuals also bring their assumptions and biases into decision making. Many organizations that are implementing their first cognitive systems are looking for techniques that leverage deep experience combined with mechanization of complex Big Data analytics. Although this industry is young, there is much that can be learned from these pioneering cognitive computing engagements.

Overview of the Book and Technology

The authors of this book, Judith Hurwitz, Marcia Kaufman, and Adrian Bowles are veterans of the computer industry. All of us are opinionated and independent industry analysts and consultants who take an integrated perspective on the relationship between different technologies and how they can transform businesses and industries. We have approached the writing of this book as a true collaboration. Each of us brings different experience from developing software to evaluating emerging technologies, to conducting in-depth research on important technology innovations.

Like many emerging technologies, cognitive computing is not easy. First, cognitive computing represents a new way of creating applications to support business and research goals. Second, it is a combination of many different technologies that have matured enough to become commercially viable. So, you may notice that most of the technologies detailed in the book have their roots in research and products that have been around for years or even decades. Some technologies or methods such as machine learning algorithms and natural language processing (NLP) have been seen in artificial intelligence applications for many decades. Other technologies such as advanced analytics have evolved and grown more sophisticated over time. Dramatic changes in deployment models such as cloud computing and distributed computing technology have provided the power and economies of scale to bring computing power to levels that were impossible only a decade ago.

This book doesn’t attempt to replace the many excellent technical books on individual topics such as machine learning, NLP, advanced analytics, neural networks, Internet of Things, distributed computing and cloud computing. Actually, we think it is wise to use this book to give you an understanding of how the pieces fit together to then gain more depth by exploring each topic in detail.

How This Book Is Organized

This book covers the fundamentals and underlying technologies that are important to creating cognitive system. It also covers the business drivers for cognitive computing and some of the industries that are early adopters of cognitive computing. The final chapter in the book provides a look into the future.

Chapter 1: “The Foundation of Cognitive Computing.”

This chapter provides perspective on the evolution to cognitive computing from artificial intelligence to machine learning.

Chapter 2: “Design Principles for Cognitive Systems

.” This chapter provides you with an understanding of what the architecture of cognitive computing is and how the pieces fit together.

Chapter 3: “Natural Language Processing in Support of a Cognitive System.”

This chapter explains how a cognitive system uses natural language processing techniques and how these techniques create understanding.

Chapter 4: “The Relationship Between Big Data and Cognitive Computing.”

Big data is one of the pillars of a cognitive system. This chapter demonstrates the Big Data technologies and approaches that are fundamental to a cognitive system.

Chapter 5: “Representing Knowledge in Taxonomies and Ontologies.”

To create a cognitive system there needs to be organizational structures for the content. This chapter examines how ontologies provide meaning to unstructured content.

Chapter 6: “Applying Advanced Analytics to Cognitive Computing.”

To assess meaning of both structured and unstructured content requires the use of a wide range of analytical techniques and tools. This chapter provides insights into what is needed.

Chapter 7: “The Role of Cloud and Distributed Computing in Cognitive Computing.”

Without the ability to distribute computing capability and resources, it would be difficult to scale a cognitive system. This chapter explains the connection between Big Data, cloud services, and distributed analytic services.

Chapter 8: “The Business Implications of Cognitive Computing.”

Why would a business need to create a cognitive computing environment? This chapter explains the circumstances in which an organization or business would benefit from cognitive computing.

Chapter 9: “IBM’s Watson as a Cognitive System.”

IBM began building a cognitive system by initiating a “grand challenge.” The grand challenge was designed to see if it could take on the best Jeopardy! players in the world. The success of this experiment led to IBM creating a cognitive platform called Watson.

Chapter 10: “The Process of Building a Cognitive Application.”

What does it take for an organization to create its own cognitive system? This chapter provides an overview of what the process looks like and what organizations need to consider.

Chapter 11: “Building a Cognitive Healthcare Application.”

Each cognitive application will be different depending on the domain. Healthcare is the first area that was selected to create cognitive solutions. This chapter looks at the types of solutions that are being created.

Chapter 12:

“

Smarter Cities: Cognitive Computing in Government.”

Using cognitive computing to help streamline support services in large cities has huge potential. This chapter looks at some of the initial efforts and what technologies come into play to support metropolitan areas.

Chapter 13: “Emerging Cognitive Computing Areas.”

Many different markets and industries can be helped through a cognitive computing approach. This chapter demonstrates which markets can benefit.

Chapter 14:

“

Future Applications for Cognitive Computing.”

It is clear that we are early in the evolution of cognitive computing. The coming decade will bring many new software and hardware innovations to stretch the limits of what is possible.

CHAPTER 1The Foundation of Cognitive Computing

Cognitive computing is a technology approach that enables humans to collaborate with machines. If you look at cognitive computing as an analog to the human brain, you need to analyze in context all types of data, from structured data in databases to unstructured data in text, images, voice, sensors, and video. These are machines that operate at a different level than traditional IT systems because they analyze and learn from this data. A cognitive system has three fundamental principles as described below:

Learn

—A cognitive system learns. The system leverages data to make inferences about a domain, a topic, a person, or an issue based on training and observations from all varieties, volumes, and velocity of data.

Model

—To learn, the system needs to create a model or representation of a domain (which includes internal and potentially external data) and assumptions that dictate what learning algorithms are used. Understanding the context of how the data fits into the model is key to a cognitive system.