8403229 Dzone Guide Artificialintelligence 2017

User Manual:

Open the PDF directly: View PDF .
Page Count: 38

Download
Open PDF In Browser	View PDF

BROUG HT TO YOU IN PA RT NERSHIP WITH

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DEAR READE R ,
Although AI isn’t new as a concept, it’s still very much in its

TABLE OF CONTE NTS
3

Executive Summary

Key Research Findings

TensorFlow for Real-World Applications

infancy, and for the first time, as a society, we’re beginning to
shift towards an AI-first realm. With endless possibilities and so
much unchartered territory to explore, it’s no wonder that the
race for AI supremacy is on. For driven industry professionals
of all fields, AI presents an exciting challenge to develop new
technologies, set industry standards, and create new processes
and workflows, as well as propose new use cases that will

8
12

satisfactorily addressed. As with any emerging field, it’s difficult

Changing Attitudes and Approaches Towards Privacy, AI, and IoT
BY IRA PASTERNAK

and a tech crisis that we’re not equipped to face just yet.

isn’t it? Two seemingly simple questions that have yet to be

AI-Powered NLP: The Evolution of Machine Intelligence from
Machine Learning
BY TUHIN CHATTOPADHYAY, PH.D.

others, like Elon Musk, are concerned that this national AI

questions that need answering. What exactly is AI? What

Data Integration & Machine Learning for Deeper Customer Insights
BY BOB HAYES

While some are willing to charge full force ahead at any cost,

Projections aside, there are still a number of very elemental

BY G. RYAN SPAIN

BY TIM SPANN

enhance the state of human-AI relationships as we know them.

dominance competition could result in unimaginable conflicts

BY MATT WERNER

Infographic: The Rob-Office

Reinforcement Learning for the Enterprise
BY SIBANJAN DAS

to set the tone and agree on a consensus or set direction. As

Diving Deeper into AI

AI undergoes major shifts, it reshapes our world, as well as our

Learning Neural Networks Using Java Libraries

human experience, and as a result our understanding of AI is
being challenged every single day.

BY DANIELA KOLAROVA

BY SARAH DAVIS

As it stands, AI should be used as an extension of humans,
and implemented so as to foster contextually personalized

whether it’s designed with the intent to assist or substitute.
And, contrary to popular belief, AI isn’t designed to replace
humans at all, but rather to replace the menial tasks performed

Executive Insights on Artificial Intelligence And All of its Variants
BY TOM SMITH

symbiotic human-AI experiences. In other words, AI should
be developed in a manner that is complementary to humans,

Checklist: Practical Uses of AI

AI Solutions Directory

Glossary

PRODUCTION

BUSINESS

Chris Smith

Rick Ross

DIRECTOR OF PRODUCTION

CEO

Andre Powell

Matt Schmidt

SR. PRODUCTION COORDINATOR

PRESIDENT

G. Ryan Spain

Jesse Davis

Moreover, as we move forward with AI developments,

PRODUCTION PUBLICATIONS EDITOR

EVP

maintaining the current open and democratized mindset that

Ashley Slate

Gordon Cervenka

by humans. As a result, our AI-powered society will open the
door to new jobs and career paths, allowing man to unlock
greater possibilities and reach new developmental heights. In
short, AI will augment our human experience.

large organizations like Open AI and Google promote will be
critical to addressing the ethical considerations involved with
these integrative technologies. When it comes to AI, there
are more questions than there are answers, and in this guide,
you’ll find a balanced take on the technical aspect of AI-first
technologies along with fresh perspectives, new ideas, and
interesting experiences related to AI. We hope that these stories
inspire you and that these findings allow you to redefine your
definition of AI, as well as empower you with knowledge
that you can implement in your AI-powered developments
and experiments.

DESIGN DIRECTOR

Billy Davis

SALES

MARKETING

DIRECTOR OF BUSINESS DEV.

Kellet Atkinson

DIRECTOR OF MARKETING

Lauren Curatola

MARKETING SPECIALIST

Kristen Pagàn

MARKETING SPECIALIST

Natalie Iannello

MARKETING SPECIALIST

Miranda Casey
Julian Morris

MARKETING SPECIALIST

ACCOUNT MANAGER

Tom Martin

ACCOUNT MANAGER

EDITORIAL

Caitlin Candelmo

DIRECTOR OF CONTENT AND
COMMUNITY

Matt Werner

PUBLICATIONS COORDINATOR

PRODUCTION ASSISSTANT

MARKETING SPECIALIST

With this collaborative spirit in mind, we also hope that this

COO

Ana Jones

Matt O’Brian
Alex Crafts
DIRECTOR OF MAJOR ACCOUNTS

Jim Howard
SR ACCOUNT EXECUTIVE

Jim Dyer

Michael Tharrington

CONTENT AND COMMUNITY MANAGER

Kara Phelps

CONTENT AND COMMUNITY MANAGER

Mike Gates

SR. CONTENT COORDINATOR

Sarah Davis

ACCOUNT EXECUTIVE

CONTENT COORDINATOR

Andrew Barker

Tom Smith

ACCOUNT EXECUTIVE

Brian Anderson
ACCOUNT EXECUTIVE

Chris Brumfield
SALES MANAGER

RESEARCH ANALYST

Jordan Baker

CONTENT COORDINATOR

Anne Marie Glen

CONTENT COORDINATOR

guide motivates you and your team to push forward and share
your own experiences, so that we can all work together
towards building ethically responsible technologies that
improve and enhance our lives.

BY CHARLES-ANTOINE RICHARD
DZONE ZONE LEADER, AND MARKETING DIRECTOR, ARCBEES

Want your solution to be featured in coming guides?
Please contact research@dzone.com for submission information.
Like to contribute content to coming guides?
Please contact research@dzone.com for consideration.
Interested in becoming a dzone research partner?
Please contact sales@dzone.com for information.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Special thanks to our topic
experts, Zone Leaders,
trusted DZone Most Valuable
Bloggers, and dedicated
users for all their help and
feedback in making this
guide a great success.

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Executive
Summary
BY MATT WERNER
PUBLICATIONS COORDINATOR, DZONE

In the past few years, Artificial Intelligence and Machine
Learning technologies have both become more prevalent
and feasible than ever before. Open source frameworks
like TensorFlow helped get developers excited about their
own applications, and after years of experimenting with
recommendation engines and predictive analytics, some
major organizations like Facebook and Google are trying
to break new ground while others, like Tesla, warn of the
possibility for harm. There’s also been worry that using
AI to automate tasks and jobs could cause significant
harm to hundreds of people. But, how are developers
approaching these new tools and ideas, and why are they
interested? To find out, we asked 463 DZone readers to
share their motivations for exploring AI, as well as the
challenges they face.

WHY AI?
DATA Developers using AI primarily use it for prediction
(47%), classification (35%), automation (30%), and detection
(28%). Organizations are trying to achieve predictive
analytics (74%), task automation (50%), and customer
recommendation engines (36%).
IMPLICATIONS Those who are using AI for personal

use are working on features that they may have seen
in other places, such as a recommendation section on
an eCommerce site or streaming service that suggests
items to buy based on previous user behavior. Most
organizations, on the other hand, are mostly focused on
predictive analytics, which can help detect fraudulent
behavior, reduce risk, and optimize messaging and design
to attract customers.
RECOMMENDATIONS Experimenting with AI frameworks

and libraries to mimic features in other applications is a
great way to get started with the technology. Developers
looking for fruitful careers in the space would also benefit
by looking at “big picture” applications, such as predictive
analytics, that organizations as a whole are interested in.

LIBRARIES AND FRAMEWORKS
DATA The most popular languages for developing AI apps

are Java (41%), Python (40%), and R (16%). TensorFlow is the
most popular framework at 25%, SparkMLLib at 16%, and
Amazon ML at 10%.
IMPLICATIONS Thanks to familiarity with the language

and popular tools like Deeplearning4j and OpenNLP, Java
is the most popular language for developing AI apps.
Python is close behind for similar reasons: it’s a generalpurpose language with several easily available data
science tools, such as NumPy. TensorFlow quickly took the
lead as the most popular framework due to its versatility
and functionality, which has created a large community
that continues to improve upon it.
RECOMMENDATIONS A good way to reduce the amount

of time it takes to become familiar with AI and ML
development is to start with general purpose languages
developers are familiar with. Open source tools like
OpenNLP and SparkMLLib have been built for developing
these kinds of apps, so monetary cost is not a factor
either. Developers, especially those working with Java
and Python, can greatly benefit from exploring the
communities and tools that currently exist to start
building their own projects and sharing their successes
and struggles with the community as it grows.

WHAT’S KEEPING AI DOWN?
DATA Organizations that are not pursuing AI do so due to
the lack of apparent benefit (60%), developer experience
(38%), cost (35%), and time (28%).
IMPLICATIONS While factors regarding investment into

AI are contributing factors to why organizations aren’t
interested in pursuing AI, the perceived lack of benefit
to the organization is the greatest factor. This suggests
either a lack of education around the benefits of AI or that
the potential gains do not outweigh potential losses at
this point.
RECOMMENDATIONS Developers who are playing with

AI technologies in their spare time have the ability to
create change in their organizations from the bottomup. Showing managers AI-based projects that simplify
business processes could have a significant impact on
the bottom line, as well as educating managers on how
developers can get started through open source tools tied
to existing languages, as explained above. Encouraging
other developers to play with these libraries and
frameworks either on company or their spare time is a
good way to overcome the experience and cost objections,
since these tools don’t cost money. As developers learn
more about the subject, it may be more profitable for
organizations to actively invest in AI and incorporate it
into their applications.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Key
Research
Findings

•• 17% of respondents work at organizations

with more than 10,000 employees; 25% work
at organizations between 1,000 and 10,000
employees; and 23% work at organizations
between 100 and 1,000 employees.
•• 75% develop web applications or services; 46%

develop enterprise business apps; and 28%
develop native mobile applications.
EXPERIENCE
40% of respondents say they have used AI or machine

BY G . R YA N S PA I N
P R O D U C T I O N C O O R D I N ATO R , DZO N E

learning in personal projects, 23% say they have used one
of these in their organization, and 45% of respondents
say they have not used AI or machine learning at all;

463 software professionals completed DZone’s
2017 AI/Machine Learning survey. Respondent
demographics are as follows:

however, responses to later questions indicate that some
respondents may have experimented with machine
learning tools or concepts while not considering
themselves as using AI or machine learning in their

•• 36% of respondents identify as developers or

engineers, 17% identify as developer team
leads, and 13% identify as software architects.

development. For example, only 34% of respondents
selected “not applicable” when asked what algorithms they
have used for machine learning. 61% of respondents at an
organization interested or actively invested in machine
learning (59% of total respondents) said their organization

•• The average respondent has 13 years of

is training developers to pursue AI.

experience as an IT professional. 52% of
respondents have 10 years of experience or
more; 19% have 20 years or more.

TOOLS OF THE TRADE
One of the most interesting survey findings is about

•• 33% of respondents work at companies

the languages respondents have used for AI/ML. 41% of

headquartered in Europe; 36% work in
companies headquartered in North America.

respondents said they have used Java for AI or machine
learning, while 40% said they have used Python. Of the

 Which languages do you use for machine learning
development?
50

 For what purposes are you using machine learning?

Automation

Prediction

Optimization

Personalization

Detection

Classification

Java

Python

Javascript

C++

Scala

Other

n/a

Other

28
0

n/a

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Java users, 73% said that Java is the primary language

INTEREST AND CHALLENGES

they use at work, considerably higher than the 54%

While interest in machine learning is certainly present,

among all respondents. But among respondents who

it still has a long way to go before it is ubiquitous. Of

said they have used AI or machine learning at their

respondents who have never used AI or machine learning,

organization, Python usage increased to 68%. R was a

54% said there is no current business use case for it, and

distant third, with 16% saying they have used R for AI/ML.

40% say they or their organization lacks knowledge on

As far as libraries and frameworks go, TensorFlow was the

the subject. Respondents who have no personal interest

most popular with 25% of responses; 16% of respondents

in AI/ML (28%) cite lack of time (48%), ML development

said they have used Spark MLlib. For machine learning

experience (40%), and practical benefit (28%) as the

APIs, Google Prediction beat out Watson 17% to 12%.

major reasons they aren’t interested. 17% of respondents

21% of respondents said they have used an AI/machine

say their organization has no interest in AI or machine

learning library not listed in our survey, and 18% said they

learning, and 24% aren’t sure if their organization has

have used an API not listed, indicating the fragmentation

any interest. Among those whose organizations are not

of a still-new tooling landscape.

interested, factors preventing interest included not seeing
organizational benefit (60%), cost (38%), and time (28%).
For those who said their organization is interested or

USE CASES AND METHODS
When asked what purposes they are using AI/machine

invested in AI/machine learning, common challenges

learning from, almost half (47%) of respondents said they

organizations face for adoption and use include lack of

were using it for prediction. Other popular use cases were
classification (35%), automation (30%), and detection
(28%). 74% of respondents who said their organization
was interested and/or invested in ML said that
predictive analytics was their main use case, followed
by automating tasks (50%). Customer recommendations
were less sought after at 36%. The most popular type of
machine learning among respondents was supervised

data scientists (43%), attaining real-time performance in
production (40%), developer training (36%), and limited
access to usable data (32%). Organization size did have
an impact on responses; for example, 64% of respondents
who said their organization is actively invested in AI or
machine learning said they work at companies with over
1,000 employees, and 81% said they work in companies
with over 100 employees.

learning (47%), while unsupervised learning (21%) and
reinforcement learning (12%) didn’t see as much use.
The most commonly used algorithms/machine learning
methods were neural networks (39%), decision trees
(37%), and linear regression (30%).

 Is your organization currently invested or interested
in AI or machine learning?
My organization is

 What issues prevent your organization from being
interested in AI/machine learning?
60

actively invested and
interested in AI/
Not sure

machine learning
projects

17
My organization
is neither invested
nor interested in
AI/machine learning

My organization
AI/machine
learning, but not
invested

is interested in

Cost

Time

Developer
Experience

Does not see
organizational
benefit

Data
Scientist
availability

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

6
Other

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

TensorFlow
for Real-World
Applications

QUICK VIEW
01

TensorFlow and deep learning are
now something corporations must
embrace and begin using.

The coming flood of audio,
video, and image data and their
applications are key to new
business and continued success.

Images can be versioned by using
image tags — this can include
both the artifact version and
other base image attributes, like
the Java version, if you need to
deploy in various permutations.

BY TIM SPANN
SOLUTIONS ENGINEER, HORTONWORKS AND DZONE ZONE LEADER

I have spoken to thought leaders at a number of large

as analyzing images, generating data, natural language

corporations that span across multiple industries such

processing, intelligent chatbots, robotics, and more.

as medical, utilities, communications, transportation,
retail, and entertainment. They were all thinking
about what they can and should do with deep learning
and artificial intelligence. They are all driven by what
they’ve seen in well-publicized projects from wellregarded software leaders like Facebook, Alphabet,
Amazon, IBM, Apple, and Microsoft. They are starting

For corporations of all types and sizes, the use cases that fit
well with TensorFlow include:

•• Speech recognition

•• Detection of flaws

•• Image recognition

•• Text summarization

•• Object tagging videos

•• Mobile image and video

•• Self-driving cars
•• Sentiment analysis

processing

•• Air, land, and sea drones

to build out GPU-based environments to run at scale. I
have been recommending that they all add these GPUrich servers to their existing Hadoop clusters so that
they can take advantage of the existing productionlevel infrastructure in place. Though TensorFlow

For corporate developers, TensorFlow allows for development
in familiar languages like Java, Python, C, and Go. TensorFlow
is also running on Android phones, allowing for deep learning
models to be utilized in mobile contexts, marrying it with the
myriad of sensors of modern smart phones.

is certainly not the only option, it’s the first that is
mentioned by everyone I speak to. The question they
always ask is, “How do I use GPUs and TensorFlow
against my existing Hadoop data lake and leverage the

Corporations that have already adopted Big Data have the use
cases, available languages, data, team members, and projects to
learn and start from.

data and processing power already in my data centers

The first step is to identify one of the use cases that fits your

and cloud environments?” They want to know how to

company. For a company that has a large number of physical

train, how to classify at scale, and how to set up deep

assets that require maintenance, a good use case is to detect

learning pipelines while utilizing their existing data

potential issues and flaws before they become a problem. This

lakes and big data infrastructure.

is an easy-to-understand use case, potentially saving large
sums of money and improving efficiency and safety.

So why TensorFlow? TensorFlow is a well-known open source

The second step is to develop a plan for a basic pilot project.

library for deep learning developed by Google. It is now in

You will need to acquire a few pieces of hardware and a team

version 1.3 and runs on a large number of platforms used by

with a data engineer and someone familiar with Linux and

business, from mobile, to desktop, to embedded devices, to

basic device experience.

cars, to specialized workstations, to distributed clusters of

corporate servers in the cloud and on premise. This ubiquity,

This pilot team can easily start with an affordable Raspberry Pi

openness, and large community have pushed TensorFlow

Camera and a Raspberry Pi board, assuming the camera meets

into the enterprise for solving real-world applications such

their resolution requirements. They will need to acquire the

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

hardware, build a Raspberry Pi OS image, and install a number
of open source libraries. This process is well-documented here.
The first test of this project would be to send images from the
camera on regular intervals, analyzed with image recognition,
and the resulting data and images sent via Apache MiniFi to
cloud servers for additional predictive analytics and learning.
The combination of MiniFi and TensorFlow is flexible enough
that the classification of images via an existing model can be
done directly on the device. This example is documented here
at Hortonworks and utilizes OpenCV, TensorFlow, Python,
MiniFi, and NiFi.
After obtaining the images and Tensorflow results, you can
now move onto the next step, which is to train your models
to understand your dataset. The team will need to capture
good state images in different conditions for each piece of
equipment utilized in the pilot. I recommend capturing these
images at different times of year and at different angles. I also
recommend using Apache NiFi to ingest these training images,
shrink them to a standard size, and convert them to black and
white, unless color has special meaning for your devices. This

majority of the work is already complete. There are welldocumented examples of this available at DZone for you to start
with. The tools necessary to ingest, process, transform, train,
and store are the same you will start with.

can be accomplished utilizing the built-in NiFi processors:

TensorFlow and Apache NiFi are clustered and can scale to

ListFiles, ResizeImage, and a Python script utilizing OpenCV

huge number of real-time concurrent streams. This gives you a

or scikit-image.

production-ready supported environment to run these millions

The team will also need to obtain images of known damaged,
faulty, flawed, or anomalous equipment. Once you have these,
you can build and train your custom models. You should
test these on a large YARN cluster equipped with GPUs.
For TensorFlow to utilize GPUs, you will need to install the
tensorflow-gpu version as well as libraries needed by your GPU.
For NVidia, this means you will need to install and configure
CUDA. You may need to invest in a number of decent GPUs for
initial training. Training can be run on in-house infrastructure
or by utilizing one of the available clouds that offer GPUs. This
is the step that is most intensive, and depending on the size
of the images and the number of data elements and precision
needed, this step could take hours, days, or weeks; so schedule
time for this. This may also need to run a few times due to
mistakes or to tweak parameters or data.
Once you have these updated models, they can be deployed
to your remote devices to run against. The remote devices do
not need the processing power of the servers that are doing
the training. There are certainly cases where new multicore GPU devices available could be utilized to handle faster
processing and more cameras. This would require analyzing
the environment, cost of equipment, requirements for timing,
and other factors related to your specific use case. If this is for
a vehicle, drone, or a robot, investing in better equipment will
be worth it. Don’t put starter hardware in an expensive vehicle
and assume it will work great. You may also need to invest in
industrial versions of these devices to work in environments
that have higher temperature ranges, longer running times,
vibrations, or other more difficult conditions.

One of the reasons I recommend this use case is that the

of streaming deep learning operations. Also, by running
TensorFlow directly at the edge points, you can scale easily as
you add new devices and points to your network. You can also
easily shift single devices, groups of devices, or all your devices
to processing remotely without changing your system, flows,
or patterns. A mixed environment where TensorFlow lives at
the edges, at various collection hubs, and in data centers make
sense. For certain use cases, such as training, you may want
to invest in temporary cloud resources that are GPU-heavy to
decrease training times. Google, Amazon, and Microsoft offer
good GPU resources on-demand for these transient use cases.
Google, being the initial creator of TensorFlow, has some really
good experience in running TensorFlow and some interesting
hardware to run it on.
I highly recommend utilizing Apache NiFi, Apache MiniFi,
TensorFlow, OpenCV, Python, and Spark as part of your Artificial
Intelligence knowledge stream. You will be utilizing powerful,
well-regarded open source tools with healthy communities
that will continuously improve. These projects gain features,
performance and examples at a staggering pace. It’s time for your
organization to join the community by first utilizing these tools
and then contributing back.
Tim Spann is a Big Data Solution Engineer. He helps educate
and disseminate performant open source solutions for Big Data
initiatives to customers and the community. With over 15 years
of experience in various technical leadership, architecture, sales
engineering, and development roles, he is well-experienced in all facets
of Big Data, cloud, IoT, and microservices. As part of his community
efforts, he also runs the Future of Data Meetup in Princeton.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS




DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

QUICK VIEW

Data Integration and

The goal of analytics is to “find
patterns” in data. These patterns take
the form of statistical relationships
among the variables in your data.

The key to discovering new insights
is to connect the dots across your
individual data silos.

Data scientists are limited by their
ability to manually sift through the
data to find meaningful insights.

Data scientists rely on the power
of machine learning to quickly and
accurately uncover the patterns—the
relationships among variables—in
their data.

Machine Learning for
Deeper Customer Insights
BY BOB HAYES
PRESIDENT, BUSINESS OVER BROADWAY

In this Big Data world, a major goal for businesses
is to maximize the value of all their customer
data. In this article, I will argue why businesses
need to integrate their data silos to build better
models and how machine learning can help them
uncover those insights.
THE VALUE OF DATA IS INSIGHT
The goal of analytics is to “find patterns” in data. These
patterns take the form of statistical relationships among the
variables in your data. For example, marketing executives want
to know which marketing pieces improve customer buying
behavior. The marketing executives then use these patterns—
statistical relationships—to build predictive models that help
them identify which marketing piece has the greatest lift on
customer loyalty.

a lot of facts about a few people. Data sets about the human
genome are good examples of these types of data sets. For data
sets in the lower right quadrant, we know a few facts about a lot
of people (e.g. the U.S. Census). Data silos in business are good

Our ability to find patterns in data is limited by the number of
variables to which we have access. So, when you analyze data
from a single data set, the breadth of your insights is restricted
by the variables housed in that data set. If your data are
restricted to, say, attitudinal metrics from customer surveys,
you have no way of getting insights about how customer
attitude impacts customer loyalty behavior. Your inability to
link customers’ attitudes with their behaviors simply prevents
any conclusions you can make about how satisfaction with the
customer experience drives customer loyalty behaviors.

TWO DIMENSIONS OF YOUR DATA
You can describe the size of data sets along two dimensions: 1)
the sample size (number of entities in the data set) and 2) the
number of variables (number of facts about each entity). Figure
1 includes a good illustration of different data sets and how

For data sets in the upper left quadrant of Figure 1, we know

examples of these types of data sets.
Mapping and understanding all the genes of humans allows
for deep personalization in healthcare through focused drug
treatments (i.e. pharmacogenomics) and risk assessment of
genetic disorders (e.g. genetic counseling, genetic testing). The
human genome project allows healthcare professionals to look
beyond the “one size fits all” approach to a more tailored approach
of addressing the healthcare needs of a particular patient.

THE NEED FOR INTEGRATING DATA SILOS
In business, most customer data are housed in separate
data silos. While each data silo contains important pieces of
information about your customers, if you don’t connect those
pieces across those different data silos, you’re only seeing parts
of the entire customer puzzle.

they fall along these two size-related dimensions (you can see

Check out this TED talk by Tim Berners-Lee on open data that

an interactive graphic version here).

illustrates the value of merging/mashing disparate data sources

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

together. Only by merging different data sources together can

Because these integrated data sets are so large, both with

new discoveries be made—discoveries that are simply not

respect to the number of records (i.e. customers) and variables

possible if you analyze individual data silos alone.

in them, data scientists are simply unable to efficiently sift
through the sheer volume of data. Instead, to identify key

HIGH

KEY ACCOUNTS
• You know a lot of things about a
few customers
• Analytic results hard to
generalize to entire customer
base

DATA INTEGRATION
• You know a lot of things about
all customers - customer
genome
• Analytics build better models for
all customers
• True CX personalization

ONE-OFF DATA PROJECTS

LOW

NUMBER OF THINGS KNOWN ABOUT
EACH CUSTOMER (VARIABLES) (DEPTH)

Data Integration:
Your Customer Genome Project

DEPARTMENT SILOS

• You know a few things about a
few customers

• You know few things about all
customers

• Analytics less valuable due to
lack of generalizability and poor
models due to omitted metrics

• Analytics builds general rules
for broad customer segment

LOW

• Underspecified models

variables and create predictive models, data scientists rely
on the power of machine learning to quickly and accurately
uncover the patterns—the relationships among variables—in
their data.
Rather than relying on the human efforts of a single data
scientist, companies can now apply machine learning. Machine
learning uses statistics and math to allow computers to find
hidden patterns (i.e. make predictions) among variables
without being explicitly programmed where to look. Iterative
in nature, machine learning algorithms continually learn from
data. The more data they ingest, the better they get at finding
connections among the variables to generate algorithms that
efficiently define how the underlying business process works.

HIGH

NUMBER OF CUSTOMERS (SAMPLE SIZE)

In our case, we are interested in understanding the drivers
behind customer loyalty behaviors. Based on math, statistics,

Siloed data sets prevent business leaders from gaining a

and probability, algorithms find connections among variables

complete understanding of their customers. In this scenario,

that help optimize important organizational outcomes—in this

analytics can only be conducted within one data silo at a time,

case, customer loyalty. These algorithms can then be used to

restricting the set of information (i.e. variables) that can be used

make predictions about a specific customer or customer group,

to describe a given phenomenon; your analytic models are likely

providing insights to improve marketing, sales, and service

underspecified (not using the complete set of useful predictors),

functions that will increase business growth.

thereby decreasing your model’s predictive power/increasing
your model’s error. The bottom line is that you are not able to
make the best prediction about your customers because you

The Bottom Line: the application of machine learning to
uncover insights is an automated, efficient way to find the

don’t have all the necessary information about them.

important connections among your variables.

The integration of these disparate customer data silos helps

SUMMARY

your analytics team to identify the interrelationships among
the different pieces of customer information, including their
purchasing behavior, values, interests, attitudes about your
brand, interactions with your brand, and more. Integrating
information/facts about your customers allows you to gain an
understanding about how all the variables work together (i.e.
are related to each other), driving deeper customer insight about
why customers churn, recommend you, and buy more from you.
The Bottom Line: the total, integrated, unified data set is greater than the sum of its data silo parts. The key to discovering
new insights is to connect the dots across your data silos.

MACHINE LEARNING

The value of your data is only as good as the insights you
are able to extract from it. These insights are represented by
relationships among variables in your data set. Sticking to a
single data set (silo) as the sole data source limits the ability
to uncover important insights about any phenomenon you
study. In business, the practice of data science to find useful
patterns in data relies on integrating data silos, allowing access
to all the variables you have about your customers. In turn,
businesses can leverage machine learning to quickly surface
the insights from the integrated data sets, allowing them to
create more accurate models about their customers. With
machine learning advancements, the relationships people
pursue (and uncover) are limited only by their imagination.

After the data have been integrated, the next step involves
analyzing the entire set of variables. However, with the

Bob E. Hayes (Business Over Broadway) holds a PhD in industrial-

integration of many data silos, including CRM systems, public

organizational psychology. He is a scientist, blogger and author (TCE:

data (e.g. weather), and inventory data, there is an explosion of

Total Customer Experience, Beyond the Ultimate Question and Measuring

possible analyses that you can run on the combined data set. For

Customer Satisfaction and Loyalty). He likes to solve problems through

example, with 100 variables in your database, you would need

the application of the scientific method and uses data and analytics to

to test around 5000 unique pairs of relationships to determine
which variables are related to each other. The number of tests
grows exponentially when you examine unique combinations of
three or more variables, resulting in millions of tests that have

help make decisions that are based on fact, not hyperbole. He conducts
research in the area of big data, data science, and customer feedback (e.g.
identifying best practices in CX/Customer Success programs, reporting
methods, and loyalty measurement), and helps companies improve how
they use their customer data through proper integration and analysis.

to be conducted.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS




OR LET ANODOT FIND THEM FOR YOU

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

SPONSORED OPINION

own AI solution for time series anomaly detection can tie up

Discover “Unknown

experienced data scientists and developers for years.
Anodot’s AI analytics brings your team their most important

Unknowns” with AI Analytics

business insights, automatically learning the normal behavior
of your time series metrics, and alerting on abnormal behavior.
By continuously analyzing all of your business data, Anodot

and Anomaly Detection

detects the business incidents that matter, and identifies why
they are happening by correlating across multiple data sources
to give you critical insights.

Tracking hundreds of thousands of metrics can easily

AI analytics frees your data scientists from building an

become overwhelming. Traditional monitoring tools like

AI solution from scratch, and unburdens your analysts

BI dashboards, only show a subset or aggregation of your

from trying to manually spot critical anomalies while your

data, so you may be looking at the wrong thing, or missing

business is moving forward, often at breakneck speed. AI

significant details. Static thresholds can set off alert storms,

analytics can eliminate business insight latency, and give

forcing you to spend way too much time searching for the

your business the vital information to turn your time series

root cause. Meanwhile, an important business service could

data into a competitive advantage.

be performing poorly, or worse, be down!
Explore the Ultimate Guide to Building an Anomaly Detection
Yet if you track everything, you can detect anything. AI can

System.

accurately and automatically zero-in on anomalies from time
series data – even for millions of individual metrics, finding

WRITTEN BY IRA COHEN

even the issues you didn’t know to look for. But building your

CHIEF DATA SCIENTIST AND CO-FOUNDER, ANODOT

Anodot AI Analytics
Illuminate business blind spots with AI analytics, so you will never miss another revenue
leak or brand-damaging incident
CATEGORY

NEW RELEASES

OPEN SOURCE

STRENGTHS

AI Analytics

Every 3 weeks

•

Prevent revenue leaks and brand-damage by automatically
gaining actionable insights in real time

CASE STUDY

•

With 13 trillion monthly bid requests, 55,000 CPUs and 7 data

by illuminating “data blind spots”

centers, Rubicon Project needed to monitor and control its data
with confidence. With Anodot AI Analytics, Rubicon easily

•

Galan, Director of Analytics.
The company was already using Graphite for monitoring, so

•

No configuration required and no alert thresholds necessary

NOTABLE CUSTOMERS

it simply pulled Graphite data into Anodot and immediately

•

Rubicon Project

•

Microsoft

•

Waze

benefitted from streamlining and automating the data analytics.

•

Lyft

•

Comcast

•

VF Corporation

WEBSITE anodot.com

Turn data into actionable business insights without data science
expertise by leveraging built-in data science

would have taken at least six of our data scientists and engineers
more than a year to build something of this caliber,” said Rich

Get alerts on anomalies or business incidents by using
automated machine learning algorithms

“We generally prefer to build all our tools internally, but after
working with Anodot, our Chief Data Scientist estimated that it

Gain a complete picture of business drivers by correlating data
from multiple sources

tracks all of its data in real time to remedy urgent problems and
capture opportunities.

Discover the metrics that matter in an overwhelming sea of data

TWITTER @TeamAnodot

BLOG anodot.com/blog

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

QUICK VIEW

AI-Powered NLP:

Classical machine learning techniques
are used for text mining to
accomplish sentiment analysis, topic

The Evolution of

modelling, TF–IDF, NER, etc.

techniques, MI objectives like

Machine Intelligence
from Machine Learning

With the advent of deep learning
automated real-time questionanswering, emotional connotation,
fighting spam, machine translation,
summarization, and information
extraction are achieved.

Word embeddings, recurrent neural
networks, and long short-term
memory (LSTM) are used for content
creation in author’s style.

BY TUHIN CHATTOPADHYAY, PH.D.
BUSINESS ANALYTICS EVANGELIST

This article will illustrate the transition of the NLP

syntactic analysis, and content classification. Before diving

landscape from a machine learning paradigm to

further into the underlying deep learning algorithms, let’s

the realm of machine intelligence and walk the

take a look at some of the interesting applications that AI

readers through a few critical applications along

contributes to the field of NLP.

with their underlying algorithms. Nav Gill’s blog

To start with the craziest news, artificial intelligence is

on the stages of AI and their role in NLP presents a

writing the sixth book of A Song of Ice and Fire. Software

good overview of the subject. A number of research

engineer Zack Thoutt is using a recurrent neural network to

papers have also been published to explain how

help wrap up George R. R. Martin’s epic saga. Emma, created

to take traditional ML algorithms to the next

by Professor Aleksandr Marchenko, is an AI bot for checking

level. Traditionally, classical machine learning
techniques like support vector machines (SVM),
neural networks, naïve Bayes, Bayesian networks,

plagiarism that amalgamates NLP, machine learning, and
stylometry. It helps in defining the authorship of write-up by
studying the way people write. Android Oreo has the ability
to recognize text as an address, email ID, phone number,

Latent Dirichlet Allocation (LDA), etc. are used for

URL, etc. and take the intended action intelligently. The

text mining to accomplish sentiment analysis, topic

smart text selection feature uses AI to recognize commonly

modelling, TF–IDF, NER, etc.

copied words as a URL or business name. IBM Watson
Developer Cloud’s Tone Analyzer is capable of extracting the

However, with the advent of open-source APIs like

tone of any documents like tweets, online reviews, email

TensorFlow, Stanford’s CoreNLP suite, Berkeley AI

messages, interviews, etc. The analysis output is a dashboard

Research’s (BAIR) Caffe, Theano, Torch, Microsoft’s

with visualizations of the presence of multiple emotions

Cognitive Toolkit (CNTK), and licenced APIs like api.ai, IBM’s

(anger, disgust, fear, joy, sadness), language style (analytical,

Watson Conversation, Amazon Lex, Microsoft’s Cognitive

confident, tentative), and social tendencies (openness,

Services APIs for speech (Translator Speech API, Speaker

conscientiousness, extraversion, agreeableness, emotional

Recognition API, etc.), and language (Linguistic Analysis API,

range). The tool also provides sentence level analysis to

Translator Text API etc.), classical text mining algorithms

identify the specific components of emotions, language style,

have evolved into deep learning NLP architectures like

and social tendencies embedded in each sentence.

recurrent and recursive neural networks. Google Cloud,

through its Natural Language API (REST), offers sentiment

ZeroFox is leveraging AI on NLP to bust Twitter’s spam

analysis, entity analysis, entity sentiment analysis,

bot problem and protect social and digital platforms for

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

enterprises. Google Brain is conducting extensive research on

term memory (LSTM) in generating the text through

understanding natural language, and came up with unique

“memories” of a priori information. A number of research

solutions like autocomplete suggestions, autocomplete for

and development initiatives are currently going on the

doodles, and automatically answered e-mails, as well as the

artificial natural language processing to match the human

RankBrain algorithm to transform Google search. Google’s

processing of language and eventually improve it.

Neural Machine Translation reduces translation errors
by an average of 60% compared to Google’s older phrase-

The Stanford Question Answering Dataset (SQuAD) is

based system. Quora conducted a Kaggle competition to

one such initiative, with 100,000+ question-answer pairs

detect duplicate questions where the modellers reach 90%

on 5222300+ articles which were also shared in a Kaggle

accuracy. Last but not least, seamless question-answering

competition. Dynamic Co-attention Network (DCN),

is accomplished through a number of artificially intelligent

which combines a co-attention encoder with a dynamic

natural language processors like Amazon’s Alexa Voice

pointing decoder, gained prominence as the highest

Service (AVS), Lex, and Polly, along with api.ai, archie.ai, etc.

performer (Exact Match 78.7 and F1 85.6) in SQuAD and in

that can be embedded in devices like Echo and leveraged for

automatically answering questions about documents. Other

virtual assistance through chatbots.

applications of deep learning algorithms that generate
machine intelligence in the NLP space include bidirectional
long short-term memory (biLSTM) models for non-factoid
answer selection, convolutional neural networks (CNNs)

“While the focus of ML is natural
language understanding (NLU), MI
is geared up for natural language
generation (NLG) that involves
text planning, sentence planning,
and text realization.”

for sentence classification, recurrent neural networks for
word alignment models, word embeddings for speech
recognition, and recursive deep models for semantic
compositionality. Yoav Goldberg’s magnum opus and all the
dedicated courses [Stanford, Oxford, and Cambridge] on the
application of deep learning on NLP further bear testimony
to the paradigm shift from ML to MI in the NLP space.
With the evolution of human civilization, technological
advancements continue to complement the increasing
demands of human life. Thus, the progression from machine
learning to machine intelligence is completely in harmony
with the direction and pace of the development of the
human race. A few months ago, Nav Gill’s blog on the stages
of AI and their role in NLP observed that we have reached

Thus, the shift in gears from machine learning to machine
intelligence is achieved through automated real-time
question-answering, emotional analysis, spam prevention,
machine translation, summarization, and information
extraction. While the focus of ML is natural language
understanding (NLU), MI is geared up for natural language
generation (NLG) that involves text planning, sentence
planning, and text realization. Conventionally, Markov

the stage of machine intelligence, and the next stage is
machine consciousness. Of late, AI has created a lot of
hype by some who see it as the greatest risk to civilization.
However, like any technology, AI can do more good for
society than harm — when used correctly. Instead of the
predicted cause of the apocalypse, AI may turn out to be
the salvation of civilization with a bouquet of benefits, from
early cancer detection to better farming.

chains are used for text generation through the prediction
of the next word from the current word. A classic example
of a Markov chain is available at SubredditSimulator.
However, with the advent of deep learning models, a
number of experiments were conducted through embedded
words and recurrent neural networks to generate text
that can keep the style of the author intact. The same
research organization, Indigo Research, published a blog
recently that demonstrates the application of long short-

Tuhin Chattopadhyay is a business analytics and data
science thought leader. He was awarded Analytics and Insight
Leader of the Year in 2017 by KamiKaze B2B Media and was

featured in India’s Top 10 Data Scientists 2016 by Analytics India

Magazine. Tuhin spent the first ten years of his career in teaching
business statistics, research, and analytics at a number of reputed
schools. Currently, Tuhin works as Associate Director at The Nielsen
Company and is responsible for providing a full suite of analytics
consultancy services to meet the evolving needs of the industry.
Interested readers may browse his website for a full profile.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Changing Attitudes
and Approaches
Towards Privacy, AI,
and IoT

QUICK VIEW
01

In the last couple of years
there has been a big shift in
the approach toward privacy,
especially in the eyes of users.

Big Data, IoT, and AI
technologies have all
contributed to the widespread
collection and use of personal
information.

The privacy debate is at a
crossroads, where the public,
the authorities, and big
companies must decide which
direction the industry will turn.

BY IRA PASTERNAK
PRODUCT MANAGER, NEURA INC.

Privacy differs from culture to culture,
and changes along with technological
advancements and sociopolitical events.
Privacy today is a very fluid subject—a result
of major changes that took place in the last
five or so years.

with regulatory crackdowns on big companies and public
demand for better protection. By late 2016, it was clear that
the European Union was set to approve the new General Data
Protection Regulation.
Privacy views continued to evolve in 2016. A survey of
American consumers showed a drastic change in public
opinion from only one year earlier. Ninety-one percent of

The big bang of privacy awareness happened in June
2013, when the Snowden leaks came to light. The public
was exposed to surveillance methods executed by the
governments of the world, and privacy became a hot topic.
Meanwhile, data collection continued, and by 2015, almost 90
percent of the data gathered by organizations was collected
within only two years. Compare this with only 10 percent
of data being collected before 2013. People started to realize

respondents strongly agreed that users had lost control of how
their data was collected and used. When asked again whether
collecting data in exchange for improved services was okay,
47 percent approved, while only 32 percent thought it wasn’t
acceptable—a drop of 39 percent in just one year. The feelings
of powerlessness for “losing control of their data” changed to
a more businesslike approach; users were willing to cooperate
with the data collection in exchange for better services.

that a person could be analyzed according to online behavior,

This shift continues with the realization that users are

and a complete profile of social parameters like social

willing to exchange their data for personalized services and

openness, extraversion, agreeableness, and neuroticism

rewards. A survey conducted by Microsoft found that 82

could be created from just ten likes on Facebook.

percent of participants were ready to share activity data, and
79 percent were willing to share their private profile data,

Google Chief Economist Hal Varian wrote in 2014, “There is

like gender, in exchange for better services. This correlated

no putting the genie back in the bottle. Widespread sensors,

with the change in the willingness to purchase adaptive

databases, and computational power will result in less

products. Fifty-six percent stated they were more likely to

privacy in today’s sense, but will also result in less harm due

buy products that were adapting to their personal lives,

to the establishment of social norms and regulations about

rather than non-adaptive products.

how to deal with privacy issues.”
This correlates with the first real commercial use of an AI

In 2015, at the height of The Privacy Paradox, the general

service to personalize user apps and IoT devices to match

belief was that privacy would soon reach a tipping point

users’ physical world personas, preferences, and needs. As

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

users have seen the value of personalized experience, they
have relaxed their grip on accessing their personalized

10+

data. It should be noted, this is not the same thing as more

7-10

targeted advertising. When users think they are allowing

14%

access to their data or relevant notifications and products
that anticipate their needs, and receive advertising instead,

1-3

they are disappointed, annoyed, and in some cases, hostile.

41%

In other words, if the user feels they’ve been deceived, they
are less likely to trust that brand and possibly other AI-

4-6
38%

enhanced apps and products in the near future.
As companies plan to integrate AI into their apps and IoT
devices, they must be aware of the changes in privacy
cultural norms and newly enacted laws. Prior to 2017, the
most common reply regarding private data collection was,
“you don’t have to be afraid if you don’t have anything
to hide.” In 2017, we realized the power lies not in the
secrets one might have, but in understanding one’s daily
routines and behaviors. We have moved beyond the issue of
individuals being tracked for the sake of ads. It has become a
story of tracking for the sake of building social-psychological
profiles and executing micro-campaigns, so users will act
the way you want them to in the real world.

Fig 1. The number of devices I own that connect to the internet
(incl. computers, phones, fitness trackers, internet-connected cars,
appliances, Wi-Fi routers, cable boxes, etc).

The average person uses various digital services and
technologies that provide a lot of data to whomever collects it.
Since most of the services by themselves are not harmful, or at
least don’t mean any harm, there should be no problem, right?
Well, not exactly.
Today’s massive data collection has brought us to a place

Two important privacy-related acts of 2017 were the

where our privacy is at risk. It is dependent on a partnership

removal of restrictions on data trading in the US and

between organizations and consumers to ensure cultural

stricter regulation on data trading in the EU. Companies

and legal privacy standards are met.

will need to know both to navigate privacy regulations in
Since there is so much at stake, companies need to take a

the global economy.

stand regarding their approach toward privacy. The right
The most obvious, basic difference between the two

solution is a model of transparency and collaboration with

approaches is that the European law includes the right to

the users. This model assumes that private data should be

be forgotten, while the American law doesn’t. The European

owned by the users, and anyone who wishes to approach

model says there should be strict regulations, followed

the users’ private data should ask their permission and

by heavy penalties to the disobedient, to protect the end

explain why the data is needed. This way we provide

user from data collectors. The American model is more

transparency and understanding of the data sharing to all

of a free market approach where everything is for sale,

sides. This is particularly important when collecting data

and in the end, the market will create the balance that is
needed. It’s no coincidence that Europe, with its historical
understanding of the dangers of going without privacy
protection, has privacy laws that are much stricter than in
the US. Juxtaposed with both approaches is the Chinese/
Russian model, which says the state is the owner of the
data, not the companies or the citizens.
And yet, despite of all their fears and worries, most of

that will learn a user’s persona and predict their needs or
actions. AI holds great potential for user awareness and
personalized experience that result in increased engagement
and reduced churn. However, technology innovators must
understand the benefits of AI can only be realized if users
are willing, possibly even enthusiastic participants. It’s up
to organizations collecting and utilizing user data to follow
culture norms and legal requirements. Only then will AIenhanced apps and products reach their full potential.

the participants are not afraid to use the technology, and
have more than four devices connected to the internet. For

Ira Pasternak heads product management at Neura Inc., the

example, 90 percent of young American adults use social

leading provider of AI for apps and IoT devices. With a strong

media on a daily basis, and online shopping has never been

background in mobile user experience and consumer behavior, Ira

better—almost 80 percent of Americans are making one

focuses on turning raw sensory data from mobile and IoT into real-

purchase per month. It seems that on one hand, users are

world user aware insights that fuel intuitive digital experiences in

aware of the risks and problems the technology presents

mHealth, Smart Cars, Connected Homes, and more. Ira is passionate

today, and on the other hand, most are heavy consumers of
that technology.

about the psychology behind human interface with technology and
the way it shapes our day-to-day life.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS



AI/ML may be a newer, growing technology, but one day you might find that it is your greatest ally in the office. There have been
plenty of robots in movies, TV, and literature that warn us about the dangers of AI, but not nearly as many to demonstrate how AI
can help create value for your applications and organizations. Here, DZone presents the Rob-office to walk through the most
popular use cases for AI technology with our readers, and what they're used for.

28%
28%of
ofrespondents
respondentsuse
useAI/ML
AI/MLfor
fordetection.
detection.
Detecting
anomalies
can
be
incredibly
strenuous
Detecting anomalies can be incredibly strenuouson
onhumans
humans
trying
trying to
to keep
keep track
trackof
ofmore
moredata
datathan
thanthey
theycan
canhandle,
handle,but
butan
anAI
AI

20%
20% of
of respondents
respondents use
use AI/ML
AI/ML for
for optimization.
optimization.

15% of users use AI/ML for personalization.

AI
AI applications
applications built
built to
to optimize
optimize are
are trying
trying to
to achieve
achieve aa task
task or
or

AI/ML
AI/MLcan
canhelp
helpto
topersonalize
personalizeUX
UXby
bylearning
learningfrom
froma

goal
goal the
the best
best itit can
can in
in the
the least
least amount
amount of
of time.
time. Based
Based on
on what
what the
the

auser's
user'spast
pastbehavior
behaviorand
andtailoring
tailoringthe
theapp
appto
to

application can
application
can identify
identify anomalies
anomalies in
in data
data and
and alert
alert aa customer
customer ifor
something
out of the
ordinary,
as when like
a credit
a service isif the
something
is the
out of such
the ordinary,
if youcard
buyis

AI
AI observes,
observes, itit will
will try
try to
to identify
identify and
and replicate
replicate whatever
whatever actions
actions have
have

improve their experience. A common example is

been
been taken
taken that
that lead
lead to
to the
the best
best responses.
responses. For
For example,
example, aa Roomba
Roomba

Netflix's
Netflix's suggested
suggested titles
titles to
to stream,
stream, which
which are
are

used to
buy something
in China
without
buying
a plane
something
in China
without
buying
a plane
ticketticket
first.first.

will
will try
try to
to map
map your
your floor
floor and
and learn
learn how
how to
to vacuum
vacuum itit in
in the
the

basedon
ontitles
titlesyou
youhave
haverated
ratedpositively
positivelyand
and
based

most
most efficient
efficient way
way possible.
possible.

whatyou've
you'vewatched
watchedrecently.
recently.
what

47% of respondents use AI/ML for prediction.
Prediction engines aim to extrapolate likely future results based
an existing learning set of data. Prediction engines are useful for
setting goals, analyzing application performance metrics, and
detecting anomalies. For example, a predictive engine may be
able to forecast how a stock's price may change.

35%
35%of
ofreaders
readersuse
useAI/ML
AI/MLfor
forclassification.
classification.
Classification
Classificationapplications
applicationscan
canbe
bevery
veryuseful
usefulto
tosort
sort
different
differentvariables
variablesinto
intodifferent
differentcategories.
categories.For
Rather
example,
than
rather
manually
than manually
analyzing
analyzing
responses
responses
to a piece
to of
a piece
news,of
annews,
AI
an AI
application
application
can
can
search
search
forfor
keywords
keywords
or or
phrases
phrases
and
and
recognize
recognize
whichwhich
comments
are positive
are positive
or negative.
or negative.

30% of DZone members use AI/ML
for
for automation.
automation
Using AI to automate tasks is a common goal for
individuals and organizations. If a simple, repeatable
task can be automated by an AI application, it can save
tremendous amounts of time and money.

CO PYR IGHT DZO NE.CO M 201 7

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

SPONSORED OPINION

The Evolution
of AI Products

1. Who its users are – persona, habits, connections, visited

places, etc.
2. What the users are doing – Have they just arrived

home? Are they at the gym?
Combining who the users are and what they’re doing
enables user aware products to address user needs like
never before.

There are many smart products around us, but not all of
them were created equal. There are different categories of AI
that smart products can fit into:
•

•

Think about a smart home that knows that a user is returning
from a run and cools the house a bit more. Or, a car audio
system that knows its driver is alone on the way to the office
and that under these conditions likes listening to podcasts.

Automated products are the simplest and can be
programed to operate at a specific time.

And, it’s not just IoT devices – it can be a coupon app that
knows that Sheila is an avid runner and will show her
discounts for running gear when she’s at the mall, or a
medication adherence app that reminds each user to take
their meds personally when they’re about to go to sleep.

Connected products are devices that you can control
them remotely – like switching a light bulb at home
from the office.
Smart products can detect user activity – like an AC that
detects when someone arrived home and starts cooling.

These aren’t visions for the future of AI, with the add-on
SDK we’ve developed at Neura, any company can integrate
AI into their product, instantly.

User-aware products - The ultimate phase in product IQ.
They understand who the users are and react to each
one personally.

Welcome to the next phase of AI.

In order for a product to be user-aware it needs to know
two things:

WRITTEN BY DROR BREN
PRODUCT MARKETING MANAGER, NEURA

Neura AI Service
Neura’s AI enables apps and IoT products to deliver experiences that adapt to who their users are
and react to what they do throughout the day to increase engagement and reduce churn.
CATEGORY

NEW RELEASES

OPEN SOURCE

STRENGTHS

Artificial Intelligence for
IoT and apps

Two Week Sprints

•

CASE STUDY
Through artificial intelligence (AI), Neura enables the Femtech app My Days to prompt
each user at the moments that are most appropriate for them. A side-by-side test
was created to measure the effectiveness of time-based reminders (the old way) and
Neura-enhanced AI fueled reminders.
The results were decisive with ignored notifications dropping by 414%. More
significant was the second finding of this test. When a user interacted with a Neuraenhanced push notification, they were significantly more likely to then engage directly
with the My Days app. The results were an increase in direct engagement of 928% and
total engagement of 968%.
Based on this test, My Days has deployed Neura to its full user base of 100s of
thousands of users.

WEBSITE theneura.com

TWITTER @theneura

•

Artificial intelligence engine
enhances IoT devices and apps to
provide personalized experiences
that anticipate a user’s needs and
preferences
Neura enhanced products are
proven to increase engagement and
retention
Machine learning provides deep
understand of a user’s typical life
throughout each day
The Neura AI Engine incorporates
data from more than 80 IoT data
sources.

BLOG theneura.com/blog

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Reinforcement
Learning for the
Enterprise
BY SIBANJAN DAS

QUICK VIEW
01 Reinforcement Learning is a first
step towards general artificial
intelligence that can survive in a
variety of environments instead of
being tied to certain rules or models.
02 Reinforcement Learning finds
extensive application in scenarios
where human interference is involved
and cannot be solved by current age
rule-based automation and traditional
machine learning algorithms.
03 Identify various open source
reinforcement learning libraries and
get started designing solutions for
your enterprise’s problems.

BUSINESS ANALYTICS AND DATA SCIENCE CONSULTANT AND DZONE ZONE LEADER

Humanity has a unique ability to adapt to dynamic

These achievements are always laid down in line with an

environments and learn from their surroundings

organization’s business goals. With the desire to win these

and failures. It is something that machines lack,
and that is where artificial intelligence seeks

prizes and excel in their careers, employees try to maximize
their potential and give their best performance. They might
not receive the award at their first attempt. However, their

to correct this deficiency. However, traditional

manager provides feedback on what they need to improve to

supervised machine learning techniques require

succeed. They learn from these mistakes and try to improve

a lot of proper historical data to learn patterns

their performance next year. This helps an organization

and then act based on them. Reinforcement
learning is an upcoming AI technique which
goes beyond traditional supervised learning to

reach its goals by maximizing the potential of its employees.
This is how reinforcement learning works. In technical terms,
we can consider the employees as agents, C&B as rewards,
and the organization as the environment. So, reinforcement

learn and improve performance based on the

learning is a process where the agent interacts with the

actions and feedback received from a machine’s

environments to learn and receive the maximum possible

surroundings, like the way humans learn.
Reinforcement learning is the first step towards
artificial intelligence that can survive in a

rewards. Thus, they achieve their objective by taking the best
possible action. The agents are not told what steps to take.
Instead, they discover the actions that yield maximum results.

variety of environments, instead of being tied to

There are five elements associated with reinforcement

certain rules or models. It is an important and

learning:

exciting area for enterprises to explore when they

1. An agent is an intelligent program that is the primary

want their systems to operate without expert

component and decision maker in the reinforcement

supervision. Let’s take a deep dive into what

learning environment.

reinforcement learning encompasses, followed by
some of its applications in various industries.

2. The environment is the surrounding area, which

has a goal for the agent to perform.

SO, WHAT CONSTITUTES REINFORCEMENT
LEARNING?

3. An internal state, which is maintained by an agent

Let’s think of the payroll staff whom we all have in our

4. Actions, which are the tasks carried out by the agent

organizations. The compensation and benefits (C&B) team

to learn the environment.

in an environment.

comes up with different rewards and recognition programs
every year to award employees for various achievements.

5. Rewards, which are used to train the agents.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

agent receives a reward (R). This reward can be of positive

OBSERVATIONS

or negative value (V). The goal to gain maximum rewards is
defined in the policy (P). Thus, the task of the agent is to get
the best rewards by choosing the correct policy.

REWARDS
ENVIRONMENTS

AGENT

Q-LEARNING
MDP forms the basic gist of Q-Learning, one of the methods
ACTIONS

FUNDAMENTALS OF THE LEARNING APPROACH

of Reinforcement Learning. It is a strategy that finds the
optimal action selection policy for any MDP. It minimizes
behavior of a system through trial and error. Q-Learning
updates its policy (state-action mapping) based on a reward.

I have just started learning about Artificial Intelligence.
One way for me to learn is to pick up a machine learning
algorithm from the Internet, choose some data sets, and
keep applying the algorithm to the data. With this approach,
I might succeed in creating some good models. However,
most of the time, I might not get the expected result. This
formal way to learn is the exploitation learning method,
and it is not the optimal way to learn. Another way to learn

A simple representation of Q-learning algorithm is as follows:
STEP 1: Initialize the state-action matrix (Q-Matrix), which
defines the possible actions in each state. The rows of
matrix Q represent the current state of the agent, and the
columns represent the possible actions leading to the next
state as shown in the figure below:
ACTION

is the exploration mode, where I start searching different

-1

algorithms and choose the algorithm that suits my data set.
However, this might not work out, either, so I have to find

STATE

a proper balance between the two ways to learn and create

the best model. This is known as an exploration-exploitation
trade off, and forms the rationale behind the reinforcement

-1

100

-1

100

-1

learning method. Ideally, we should optimize the trade-off
defining a good policy for learning.

Note: The -1 represents no direct link between the nodes. For example,
the agent cannot traverse from state 0 to state 3.

This brings us to the mathematical framework known

STEP 2: Initialize the state-action matrix (Q-Matrix) to zero

as Markov Decision Processes which are used to model

or the minimum value.

between exploration and exploitation learning methods by

decision using states, actions and rewards. It consists of:
STEP 3: For each episode:

S – Set of states

••

Choose one possible action.

R – Reward functions

••

Perform action.

P – Policy

••

Measure Reward.

••

Repeat STEP 2 (a to c) until it finds the action that

A – Set of actions

V – Value

yields maximum Q value.

So, in a Markov Decision Process (MDP), an agent (decision
maker) is in some state (S). The agent has to take action

••

Update Q value.

(A) to transit to a new state (S). Based on this response, the
STEP 4: Repeat until the goal state has been reached.

So, reinforcement learning is a process
where the agent interacts with the

GETTING STARTED WITH REINFORCEMENT LEARNING
Luckily, we need not code the algorithms ourselves. Various
AI communities have done this challenging work, thanks
to the ever-growing technocrats and organizations who are
making our days easier. The only thing we need to do is to

environments to learn and receive the

think of the problem that exists in our enterprises, map it to
a possible reinforcement learning solution, and implement

maximum possible rewards.

the model.

•• Keras-RL implements state-of-the art deep

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

reinforcement learning algorithms and integrates with
the deep learning library Keras. Due to this integration,
it can work either with Theano or Tensorflow and can be
used in either a CPU or GPU machines. It is implemented
in Python Deep Q-learning (DQN), Double DQN (removes
the bias from the max operator in Q-learning), DDPG,
Continuous DQN, and CEM.

•• PyBrain is another Python-based Reinforcement Learning,
Artificial intelligence, and neural network package that
implements standard RL algorithms like Q-Learning and
more advanced ones such as Neural Fitted Q-iteration.

Reinforcement learning finds
extensive applications in those
scenarios where human interference
is involved, and cannot be solved by
rule-based automation and traditional
machine learning algorithms.

It also includes some black-box policy optimization
methods (e.g. CMA-ES, genetic algorithms, etc.).

•• OpenAI Gym is a toolkit that provides a simple interface
to a growing collection of reinforcement learning tasks.

the notable examples in the recent past is an industrial

You can use it with Python, as well as other languages in

robot developed by a Japanese company, Faunc, that

the future.

learned a new job overnight. This industrial robot used
reinforcement learning to figure out on how to pick up

•• TeachingBox is a Java-based reinforcement learning

objects from containers with high precision overnight.

framework. It provides a classy and convenient

It recorded its every move and found the right path to

toolbox for easy experimentation with different

identify and select the objects.

reinforcement algorithms. It has embedded techniques
to relieve the robot developer from programming
sophisticated robot behaviors.

2. Digital Marketing

Enterprises can deploy reinforcement learning models
to show advertisements to a user based on his activities.
The model can learn the best ad based on user behavior
and show the best advertisement at the appropriate
time in a proper personalized format. This can take

Ideally, we should optimize the
trade-off between exploration and
exploitation learning methods by
defining a good policy for learning.

ad personalization to the next level that guarantees
maximum returns.
3. Chatbots

Reinforcement learning can make dialogue more
engaging. Instead of general rules or chatbots with
supervised learning, reinforcement learning can select
sentences that can take a conversation to the next level
for collecting long term rewards.
4. Finance

Reinforcement learning has immense

POSSIBLE USE CASES FOR ENTERPRISES
Reinforcement learning finds extensive applications in
those scenarios where human interference is involved, and

applications in stock trading. It can be used to
evaluate trading strategies that can maximize the
value of financial portfolios.

cannot be solved by rule-based automation and traditional
machine learning algorithms. This includes robotic process
automation, packing of materials, self-navigating cars,
strategic decisions, and much more.
1. Manufacturing

Reinforcement learning can be used to power up the
brains of industrial robots to learn by themselves. One of

Sibanjan Das is a Business Analytics and Data Science
consultant. He has over seven years of experience in the IT
industry working on ERP systems, implementing predictive

analytics solutions in business systems, and the Internet of Things.

Sibanjan holds a Master of IT degree with a major in Business
Analytics from Singapore Management University. Connect with him
at his Twiiter handle @sibanjandas to follow the latest news in Data
Science, Big Data, and AI.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Diving Deeper
INTO ARTIFICIAL INTELLIGENCE

ARTIFICIAL INTELLIGENCE-RELATED ZONES

TOP #ARTIFICIALINTELLIGENCE TWITTER ACCOUNTS

@aditdeshpande3

@karpathy

AI dzone.com/ai
The Artificial Intelligence (AI) Zone features all aspects of AI pertaining to
Machine Learning, Natural Language Processing, and Cognitive Computing.
The AI Zone goes beyond the buzz and provides practical applications of
chatbots, deep learning, knowledge engineering, and neural networks.

@AndrewYNg

@adampaulcoates

IoT dzone.com/iot
The Internet of Things (IoT) Zone features all aspects of this multifaceted

@mgualtieri

@soumithchintala

technology movement. Here you’ll find information related to IoT, including
Machine to Machine (M2M), real-time data, fog computing, haptics, open
distributed computing, and other hot topics. The IoT Zone goes beyond home
automation to include wearables, business-oriented technology, and more.

@demishassabis

@DrAndyPardoe

Big Data dzone.com/bigdata
The Big Data/Analytics Zone is a prime resource and community for Big
Data professionals of all types. We’re on top of all the best tips and news for
Hadoop, R, and data visualization technologies. Not only that, but we also

@bobehayes

@SpirosMargaris

give you advice from data science experts on how to understand and present
that data.

TOP ARTIFICIAL INTELLIGENCE REFCARDZ

TOP ARTIFICIAL INTELLIGENCE RESOURCES

Recommendations Using Redis

Linear Digressions

In this Refcard, learn to develop a simple

lineardigressions.com

recommendation system with Redis, based on user-

Covering a variety of topics related to data

by Martin Zinkevich

indicated interests and collaborative filtering. Use data

science and machine learning, this podcast

Learn how you can use machine learning to your bene-

structures to easily create your system, learn how to

features two experts who make the most

fit — even if you just have a basic working knowledge.

use commands, and optimize your system for real-

complicated AI concepts accessible.

Get a better understanding of machine learning termi-

time recommendations in production.

The O’Reilly Bots Podcast

Best Practices for Machine Learning
Engineering

nology and consider the process of machine learning
through three key phases.

oreilly.com/topics/oreilly-bots-podcast

Machine Learning
Covers machine learning for predictive analytics,
explains setting up training and testing data, and
offers machine learning model snippets.

TOP ARTIFICIAL INTELLIGENCE
PODCASTS

This assortment of podcasts discusses the
most recent advances that are
revolutionizing how we interact with
conversational robots.

Intro to AI
by Ansaf Salleb-Aouissi
In this free introductory course, learn the fundamentals of
artificial intelligence, building intelligent agents, solving
real AI problems with Python, and more.

R Essentials

Concerning AI

R has become a widely popular language because

concerning.ai

of its varying data structures, which can be more

If you’re interested in the more philosoph-

Video Lectures on Machine Learning

intuitive than data storage in other languages; its

ical, ethical aspect of artificial intelligence,

This wide assortment of machine learning videos will

built-in statistical and graphical functions; and its

this podcast is for you. Concerning AI will

teach you everything you need to know about machine

large collection of useful plugins that can enhance the

inspire you to think deeply about what AI

learning, from Bayesian learning to supervised learning to

language’s abilities in many different ways.

means for the future of society.

clustering and more.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Rainforest QA

The Leader in AI-Powered QA

Delivering a high-quality product can mean the difference between
a stellar customer experience and a sub-par one. Rainforest helps
you deliver the wow-factor apps at scale by ensuring that every
deployment meets your standards. Our testing solution combines
machine intelligence with over 60,000 experienced testers to deliver
on-demand, comprehensive QA test results in as fast as 30 minutes.

Find out how to sign off on software
releases with more confidence at
www.rainforestqa.com/demo.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

SPONSORED OPINION

How to Manage
Crowdsourcing at Scale
with Machine Learning
Anyone who has used crowdsourcing systems like mTurk
and Crowdflower has had the experience of getting results
that aren’t quite right. Whatever the reason, improving
and assuring the quality of microservice output can be a
challenge. We’ve implemented a machine learning model
to successfully scale crowdsourced tasks without losing
results quality.
WHY USE MACHINE LEARNING FOR SOFTWARE TESTING?
Manually checking output defeats the purpose of leveraging
microservices, especially at the scale that we use it. By
feeding every piece of work through our machine learning
algorithms, we can avoid many of the issues associated with
leveraging microservices efficiently.

catch sloppy work based on input patterns. We can catch
suspicious job execution behavior by analyzing mouse
movements and clicks, the time it takes to execute the task,
and other factors.
A CONSTANTLY GROWING TRAINING DATA SET
Almost every task executed with our platform becomes
training data for our machine learning mode, meaning that
our algorithm is constantly being refined and improved. It
goes through tagging and sorting process to ensure that it’s
labelled correctly, then feeds into our algorithms to further
refine our results.
REAL-TIME QUALITY CONFIRMATION
We want to give users results fast, whether they’re running
tests on Saturday at 2am or first thing Monday morning.
Machine learning management allows us to provide a
consistent bar for results quality at any moment, at any scale.
Incorporating machine learning into our crowdtesting model
helps us stabilize the quality of our test results. As a result,
we can more confidently integrate microservices into our
development workflow.

DETECTING BAD WORK FROM INPUT PATTERNS
By running every test result against our algorithms, we can

WRITTEN BY RUSS SMITH
CTO AND CO-FOUNDER, RAINFOREST QA

Rainforest QA
Like Automation but Good: Machine Efficiency with Human Context.
CATEGORY

NEW RELEASES

OPEN SOURCE

STRENGTHS

AI-Powered QA Platform

Continuous

•

Enables teams to take a streamlined, datadriven approach to QA testing

CASE STUDY

•

Guru is a knowledge management solution that helps teams capture,

machine-learning verified test results

share and access knowledge easily. Their development team manages
their own QA testing. In order to keep their team small as their

•

Integrates testing into development workflow

•

Provides clear, comprehensive test results for

product, Guru integrated Rainforest QA into its development workflow,
encouraging their engineers to write tests from their

faster issue resolution

code editor.
The Rainforest machine learning algorithm confirms all test results,
allowing Guru to have confidence in the quality of their test results. By

NOTABLE CUSTOMERS
•

Adobe

•

Oracle

have recovered 100+ hours of developer time from testing each month

•

BleacherReport

without sacrificing product quality.

•

StubHub

Read their story here.

•

TrendKite

leveraging Rainforest, Guru has scaled their developer-driven quality
process rather than hiring a dedicated QA manager. As a result, they

WEBSITE rainforestqa.com

Increases confidence in release quality with

TWITTER @rainforestqa

BLOG rainforestqa.com/blog

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Learning Neural
Networks Using
Java Libraries

QUICK VIEW
01

Learn about the evolution of
neural networks

A short guide to implement of
Neural Networks from scratch

A summary of popular Java
Neural Network libraries

BY DANIELA KOLAROVA
SYSTEM ARCHITECT, DXC TECHNOLOGY

As developers, we are used to thinking in
terms of commands or functions. A program is
composed of tasks, and each task is defined using
some programming constructs. Neural networks
differ from this programming approach in the
sense that they add the notion of automatic
task improvement, or the capability to learn and
improve similarly to the way the brain does.
In other words, they try to learn new activities
without task-specific programming.
Instead of providing a tutorial on writing a neural network
from scratch, this tutorial will be about neural nets
incorporating Java code. The evolution of neural nets

and 0 represented false. They assigned a binary threshold
activation to the neuron to calculate the neuron’s output.

Input x1

Σ | F(x)

Output

Input x2

The threshold was given a real value, say 1, which would
allow for a 0 or 1 output if the threshold was met or
exceeded. Thus, in order to represent the AND function, we
set the threshold at 2.0 and come up with the following table:
AND

starts from McCulloch and Pitt’s neuron, enhancing it with
Hebb’s findings, implementing the Rosenblatt’s perceptron,
and showing why it can’t solve the XOR problem. We will
implement the solution to the XOR problem by connecting
neurons, producing a Multilayer Perceptron, and making
it learn by applying backpropagation. After being able to
demonstrate a neural network implementation, a training
algorithm, and a test, we will try to implement it using
some open-source Java ML frameworks dedicated to deep

This approach could also be applied for the OR function
if we switch the threshold value to 1. So far, we have

learning: Neuroph, Encog, and Deeplearning4j.

classic linearly separable data as shown in the tables, as

The early model of an artificial neuron was introduced

McCulloch-Pitts neuron had some serious limitations.

by the neurophysiologist Warren McCulloch and logician

In particular, it could solve neither the “exclusive or”

Walter Pitts in 1943. Their paper, entitled, “A Logical

function (XOR), nor the “exclusive nor” function (XNOR),

Calculus Immanent in Nervous Activity,” is commonly

which seem to be not linearly separable. The next

regarded as the inception of the study of neural networks.

revolution was introduced by Donald Hebb, well-known

The McCulloch-Pitts neuron worked by inputting either

for his theory on Hebbian learning. In his 1949 book, The

a 1 or 0 for each of the inputs, where 1 represented true

Organization of Behavior, he states:

we can divide the data using a straight line. However, the

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

“When an axon of cell A is near enough to excite a cell B and
repeatedly or persistently takes part in firing it, some growth
process or metabolic change takes place in one or both cells
such that A’s efficiency, as one of the cells firing B,
is increased.”
In other words, when one neuron repeatedly assists in
firing another, the axon/connection of the first neuron
develops synaptic knobs or enlarges them if they already

tables, we can see that XOR turns to be equivalent to OR
and NOT AND functions representable by single neurons.
Let’s take a look at the truth tables again:
T

NOT AND

exist in contact with the second neuron. Hebb was not
connection between the neurons is strengthened — which
is known as the weight assigned to the connections
between neurons — but also that this activity is one of

XOR

only proposing that when two neurons fire together the

the fundamental operations necessary for learning and
memory. The McCulloch-Pitts neuron had to be altered

We can combine the two neurons representing NOT AND

to assign weight to each of the inputs. Thus, an input of

and OR and build a neural net for solving the XOR problem

1 may be given more or less weight, relative to the total

similar to the net presented below:

threshold sum.

INPUT

HIDDEN

OUTPUT

Later, in 1962, the perceptron was defined and described
by Frank Rosenblatt in his book, Principles of Neurodynamics.
This was a model of a neuron that could learn in the
Hebbean sense through the weighting of inputs and that
laid the foundation for the later development of neural
networks. Learning in the sense of the perceptron meant
initializing the perceptron with random weights and
repeatedly checking the answer after the activation was

The diagram represents a multiplayer perception, which

correct or there was an error. If it was incorrect, the

has one input layer, one hidden layer, and an output layer.

network could learn from its mistake and adjust

The connections between the neurons have associated

its weights.

weights not shown in the picture. Similar to the single
perception, each processing unit has a summing and

Input x1
Input x2

activation component. It looks pretty simple but we also

Σ | F(x)

Output

need a training algorithm in order to be able to adjust
the weights of the various layers and make it learn. With
the simple perception, we could easily evaluate how to
change the weights according to the error. Training a

Despite the many changes made to the original
McCulloch-Pitts neuron, the perceptron was still limited

multilayered perception implies calculation of the overall
error of the network.

to solving certain functions. In 1969, Minsky co-authored

In 1986, Geoffrey Hinton, David Rumelhart, and Ronald

with Seymour Papert, Perceptrons: An Introduction to

Williams published a paper, “Learning Representations by

Computational Geometry, which attacked the limitations

Backpropagating Errors”, which describes a new learning

of the perceptron. They showed that the perceptron

procedure, backpropagation. The procedure repeatedly

could only solve linearly separable functions and had not

adjusts the weights of the connections in the network so

solved the limitations at that point. As a result, very little

as to minimize a measure of difference between the actual

research was done in the area until the 1980s. What would

output vector of the net and the desired output vector. As

come to resolve many of these difficulties was the creation

a result of the weight adjustments, internal hidden units

of neural networks. These networks connected the inputs

— which are not part of the input or output — are used to

of artificial neurons with the outputs of other artificial

represent important features, and the regularities of the

neurons. As a result, the networks were able to solve

tasks are captured by the interaction of these units.

more difficult problems, but they grew considerably more

complex. Let’s consider again the XOR problem that wasn’t

It’s time to code a multilayered perceptron able to learn the

solved by the perceptron. If we carefully observe the truth

XOR function using Java. We need to create a few classes,

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

like a neuron interface named ProcessingUnit, Connection

repository with the XOR NeuralNet. It is obvious that

class, a few more activation functions, and a neural net

there will be less code written using one of these libraries

with a layer that is able to learn. The interfaces and classes

compared to the Java code needed for our example.

can be found in a project located in my GitHub repository.

Neuroph provides an API for datasets that allows for easier
training data initialization, learning rules hierarchy,
neural net serialization/persistence, and deserialization,
and is equipped with a GUI. Encog is an advanced machine

The McCulloch-Pitts neuron worked

learning framework that supports a variety of advanced

by inputting either a 1 or 0 for each

process data. However, its main strength lies in its neural

of the inputs, where 1 represented

wide variety of networks, as well as support classes to

true and 0 represented false.

algorithms, as well as support classes to normalize and
network algorithms. Encog contains classes to create a
normalize and process data for these neural networks.
Deeplearning4j is a very powerful library that supports
several algorithms, including distributed parallel versions
that integrate with Apache Hadoop and Spark. It is
definitely the right choice for experienced developers and
software architects. A XOR example is provided as part of

The NeuralNet class is responsible for the construction

the library packages.

and initialization of the layers. It also provides
functionality for training and evaluation of the activation
results. If you run the NeuralNet class solving the classical
XOR problem, it will activate, evaluate the result, apply
backpropagation, and print the training results.

With the simple perception, we

If you take a detailed look at the code, you will notice

could easily evaluate how to change

that it is not very flexible in terms of reusability. It would
be better if we divide the NeuralNet structure from
the training part to be able to apply various learning

the weights according to the error.

algorithms on various neural net structures. Furthermore,
if we want to experiment more with deep learning
structures and various activation functions, we will have
to change the data structures because for now, there
is only one hidden layer defined. The backpropagation
calculations have to be carefully tested in isolation in
order to be sure we haven’t introduced any bugs. Once
we are finished with all the refactoring, we will have to
start to think about the performance of deep neural nets.
What I am trying to say is that if we have a real problem
to solve, we need to take a look at the existing neural
nets libraries. Implementing a neural net from scratch
helps to understand the details of the paradigm, but one
would have to put a lot of effort if a real-life solution has
to be implemented from scratch. For this review, I have
selected only pure Java neural net libraries. All of them

Using one of the many libraries available, developers
are encouraged to start experimenting with various
parameters and make their neural nets learn. This article
demonstrated a very simple example with a few neurons
and backpropagation. However, many of the artificial
neural networks in use today still stem from the early
advances of the McCulloch-Pitts neuron and the
Rosenblatt perceptron. It is important to understand
the roots of the neurons as building blocks of modern
deep neural nets and to experiment with the ready-touse neurons, layers, activation functions, and learning
algorithms in the libraries.

are open-source, though Deeplearning4j is commercially
supported. All of them are documented very well with lots
of examples. Deeplearning4j has also CUDA support. A
comprehensive list of deep learning software for various
languages is available on Wikipedia, as well.
Examples using this library are also included in the GitHub

Daniela Kolarova is a senior application architect at DXC
Technology with more than 13 years experience with Java.
She has worked on many international projects using Core Java,
Java EE, and Spring. Because of her interests in AI she worked
on scientific projects at the Bulgarian Academy of Sciences
and the University. She also writes article on DZone, scientific
publications, and has spoken at AI and Java conferences.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS



DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Practical Uses of AI
BY SARAH DAVIS - CONTENT COORDINARTOR, DZONE

Too often regarded as a buzzword, artificial intelligence (AI) is a rapidly growing field that shows no signs of slowing down.
According to the Bank of America Corporation, the AI-based analytics market should be valued at $70 billion by 2020. It’s
hard to tell exactly what the future will look like – there are many pros and cons to weigh out – but one thing is for sure: AI
is well on its way to being a major part of the future.

 SECURITY
Machine learning models are being developed that can accurately predict
files that contain malware in order to help both prevent and predict
security breaches. For example, Deep Instinct is applying AI and deep
learning technology to detect threat attacks.

 EDUCATION

Natural language processing and machine learning are being used to
collect and analyze data from 911 calls, social media, gunshot sensors, and
more to create heat maps of where crimes are likely to occur.

 PERSONALIZATION AND RECOMMENDATIONS

IBM’s Teacher Advisor, based on Watson, allows math teachers to create

We’ve been seeing companies deploy marketing personalization through

personalized lesson plans for individual students. Another platform

AI for year, with sites like Amazon suggesting recommended purchases

developed by IBM Watson is Jill, an automated teaching assistant robot

after a user clicks on an item. This is advancing rapidly, though, and many

that responds to student inquiries for large online courses and that could

AI systems are using location data to determine things like when to give

improve student retention rates.

users push notifications and what coupons to send.

 ACCESSIBILITY
AI provides a great opportunity for wheelchair users and people with
autism, to name a few. For example, Autimood is an AI application that
helps children with autism better understand human emotions in a way
that’s both helpful and fun. Additionally, Robotic Adaption to Humans
Adapting to Robots (RADHAR) uses computer vision algorithms to make
navigating environments in a wheelchair easier.

 ENERGY AND THE ENVIRONMENT

 PUBLIC SAFETY

 HEALTHCARE

AI has huge potential in the healthcare industry since it can analyze
big data much more quickly than human doctors can. It can assist in
preventing, screening, treating, and monitoring diseases. For example, a
computer-assisted diagnosis can predict breast cancer in women a year
before their official diagnosis.

 CONVENIENCE

MIT researchers have developed a machine learning system that uses

AI programs use algorithms and machine learning to make general

predictive analytics to pick the best location for wind farms. Additionally,

life easier for consumers. For instance, machine learning systems

IBM researchers are using machine learning to analyze pollution data and

can analyze photos to suggest the best restaurant, apps can give

make predictions about air quality. AI is also being used to analyze forest

personalized financial advice, and household robots can read people’s

data to predict and stop deforestation before it starts.

facial expressions and provide the best possible interaction.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

SPONSORED OPINION

How Will AI
Impact Your BI

in a timely and useful way takes data science expertise and
programming horsepower your organization may have trouble
finding (or paying for). OpenText Magellan can help.

With relatively little effort, a company or
agency can amass petabytes of data – more
information than the entire human race had

We should be living in an information utopia. Ever more powerful
and affordable technology means you can gather data out of

collected until the 20TH century.

nearly any process, from overheating train brakes to citizen
comments about the quality of service at their local airport, and
share it widely and near-instantly.

Magellan is a flexible, pre-integrated AI-powered analytics
With relatively little effort, a company or agency can amass

platform that combines open source machine learning with

petabytes of data – more information than the entire human race

advanced analytics, enterprise-grade BI, and capabilities to

had collected until the 20th century. And information-collecting

acquire, merge, manage and analyze Big Data and Big Content

is vital, because in an increasingly competitive economy,

stored in your Enterprise Information Management (EIM)

companies need to take advantage of every possible insight in

systems. What this means in real-world terms is that you can

order to grow their business and stay ahead of competitors.

make decisions and take actions with the help of Magellan with
greater speed and scale than you could unassisted.

The problem is that having so much data can be overwhelming to
manage. Organizing enormous volumes of data, searching them

WRITTEN BY STANNIE HOLT

for patterns and relevant insights, and reporting those findings

MARKETING CONTENT WRITER, OPENTEXT

OpenText Magellan
The power of AI in a pre-configured platform that augments decision making and accelerates your business
CATEGORY

NEW RELEASES

OPEN SOURCE

CASE STUDY

An AI-powered analytics platform

Continuous

Yes

OpenText Magellan combines open

that combines open source machine

source machine learning with advanced

learning with advanced analytics

analytics, enterprise-grade BI, and
capabilities to acquire, merge, manage

STRENGTHS
•

and analyze Big Data and Big Content

A cohesive platform with pre-built components: Bundling technologies for advanced

stored in your Enterprise Information

analytics, machine learning, data modeling and preparation, and enterprise-grade BI into

Management systems.

a single infrastructure
•

The result is a flexible, cognitive

Built on an open foundation: Magellan lets you take advantage of the flexibility,
extensibility, and diversity of an open product stack while maintaining full ownership of
your data and algorithms.

•

software platform built on Apache
Spark that dramatically reduces the
time, effort and expertise required for
integrating these varied technologies –

Designed to drive autonomy: Magellan empowers IT to empower non-technical users—

to leverage the benefits and realize the

with a self-service interface enabling business analysts to apply sophisticated algorithms

value of advanced analytics for decision

and act on the insights they find.

making and task automation across
your EIM applications.

•

Infused with unstructured data analytics: Magellan includes powerful natural language
processing capabilities for Big Content like concept identification, categorization, entity
extraction, and sentiment analysis.

WEBSITE opentext.com

TWITTER @OpenText

BLOG bit.ly/2zAvtiY

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

QUICK VIEW

Executive Insights on
Artificial Intelligence
And All of its Variants
BY TOM SMITH
RESEARCH ANALYST, DZONE

To gather insights on the state of artificial intelligence (AI), and
all its variants, machine learning (ML), deep learning (DL), natural
language processing (NLP), predictive analytics, and neural
networks, we spoke with 22 executives who are familiar with AI.
GAURAV BANGA CEO, CTO, AND DR. VINAY SRIDHARA, BALBIX
ABHINAV SHARMA DIGITAL SERVICING GROUP LEAD, BARCLAYCARD US
PEDRO ARELLANO VP PRODUCT STRATEGY, BIRST
MATT JACKSON VP AND NATIONAL GENERAL MANAGER, BLUEMETAL
MARK HAMMOND CEO, BONSAI
ASHOK REDDY GENERAL MANAGER, MAINFRAME, CA TECHNOLOGIES
SUNDEEP SANGHAVI CO-FOUNDER AND CEO, DATARPM, A PROGRESS COMPANY
ELI DAVID CO-FOUNDER AND CHIEF TECHNOLOGY OFFICER, DEEP INSTINCT
ALI DIN GM AND CMO, AND MARK MILLAR, DIRECTOR OF RESEARCH AND
DEVELOPMENT, DINCLOUD
SASTRY MALLADI CTO, FOGHORN SYSTEMS
FLAVIO VILLANUSTRE VP TECHNOLOGY LEXISNEXIS RISK SOLUTIONS,
HPCC SYSTEMS
ROB HIGH CTO WATSON, IBM
JAN VAN HOECKE CTO, IMANAGE
ELDAR SADIKOV CEO AND CO-FOUNDER, JETLORE
AMIT VIJ CEO AND CO-FOUNDER, KINETICA
TED DUNNING PHD., CHIEF APPLICATION ARCHITECT, MAPR
BOB FRIDAY CTO AND CO-FOUNDER, MIST
JEFF AARON VP OF MARKETING, MIST
SRI RAMANATHAN GROUP VP AI BOTS AND MOBILE, ORACLE
SCOTT PARKER SENIOR PRODUCT MARKETING MANAGER, SINEQUA
MICHAEL O’CONNELL CHIEF ANALYTICS OFFICER, TIBCO

Like Big Data and IoT,
implementing a successful
artificial intelligence strategy
depends on identifying the
business problem you are
trying to solve.

Companies in any industry
benefit from AI by making
smarter, more informed
decisions by collecting,
measuring, and analyzing data.

People and companies are
beginning to use AI and all
of its variations to solve real
business problems, seeing a
tremendous impact on the
bottom line.

KEY FINDINGS
01 The key to having a successful AI business strategy is to
know what business problem you are trying to solve. Having the
necessary data, having the right tools, and having the wherewithal
to keep your models up-to-date are important once you’ve identified
specifically what you want to accomplish.
Start by looking at your most tedious and time-consuming
processes. Identify where you have the greatest risk exposure for
failing to fulfill compliance issues as well as the most valuable
assets you want to protect.
Once you’ve identified the business problem you are trying to solve,
you can begin to determine the data, tools, and skillsets you will
need. The right tool, technology, and type of AI depends on what you
are trying to accomplish.
02 Companies benefit from AI by making smarter, more informed
decisions, in any industry, by collecting, measuring, and analyzing
data to prevent fraud, reduce risk, improve productivity and
efficiency, accelerate time to market and mean time to resolution,
and improve accuracy and customer experience (CX).
Unlike before, companies can now afford the time and money
to look at the data to make an informed decision. You cannot do
this unless you have a culture to collect, measure, and value data.
Achieving this data focus is a huge benefit event without AI since
a lot of businesses will continue to operate on gut feel rather
than data. They view data as a threat versus an opportunity, and
ultimately these businesses will not survive.
Employee engagement and CX can be improved in every vertical
industry, and every piece of software can benefit. AI can replicate
day-to-day processes with a greater level of accuracy than any
human, without downtime. This will have a significant impact on
the productivity, efficiency, margins, and the risk profile of every
company pushing savings and revenue gains to the bottom line.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Companies will be able to get to market faster and cheaper, with
greater customer satisfaction and retention.

learning. Companies also have a hard time wrangling all of their
data, which may be stored in multiple places.

03 The biggest change in AI in the near-term has been the fact
that people and companies are beginning to use it, and all of its
variations, to solve real business problems. Tools and libraries
have improved. The cloud is enabling companies to handle data at
scale necessary for AI, machine learning (ML), deep learning (DL),
natural language processing (NLP), and recurring neural networks.
In addition, more investment is being made in AI initiatives as
companies see the dramatic impact it can have on the bottom line.

Brownfield equipment owners can be very uncomfortable
with anyone interfering with their very expensive and precise
equipment. Luckily, you don’t need to touch their equipment to
execute a proof of concept. Once a client becomes comfortable
with what AI can accomplish, they are open to automation. Once
end users see the data is more accurate than their experience, they
begin to trust the data and trust AI to improve the efficiency and
reliability of their equipment.

We’ve moved from machine learning to deep learning. We see more
AI/ML libraries that are more mature and scalable. We see larger
neural networks that are deeper and able to handle more data,
resulting in more knowledge and greater accuracy.

07 The greatest opportunities for the implementation of AI
are ubiquitous – it’s just a matter of which industries adopt and
implement it the quickest. All prospects have the same level of
opportunity with AI. How can businesses identify jobs that require
a lot of repetitive work and start automating them?

Today the cloud is a commodity, and it’s possible that this
will happen to AI as well, except faster, as consumers adopt
autonomous cars and manufacturers put hundreds of millions
of dollars on their bottom lines. AI improves quality of life for
individuals, making things simpler and easier while improving the
quality of life of workers and making companies significantly
more profitable.
04 The technical solutions mentioned most frequently with AI
initiatives are: TensorFlow, Python, Spark, and Google.ai. Spark,
Python, and R are mentioned most frequently as the languages
being used to perform data science while Google, IBM Watson, and
Microsoft Azure are providing plenty of tools for developers to work
on AI projects via API access.
05 The real-world problems being solved with AI are diverse
and wide-reaching, with the most frequently mentioned verticals
being finance, manufacturing, logistics, retail, and oil and gas. The
most frequently mentioned solutions were cybersecurity, fraud
prevention, efficiency improvement, and CX.
AI helps show what’s secure, what’s not, and every attack vector. It
identifies security gaps automatically freeing up security operations
to focus on more strategic issues while making security simpler
and more effective.
A large-scale manufacturer milling aircraft parts used to take
days to make the parts with frequent manual recalibrations of
the machine. Intelligent behavior has increased efficiency of the
operators, reduced time to mill a part, and reduced the deviations
in parts. AI automation provides greater support for the operators
and adds significant value to the bottom line.
06 The most common issues preventing companies from
realizing the benefits of AI are a lack of understanding, expertise,
trust, or data.
There’s fear of emerging technology and lack of vision. Companies
don’t know where to start, they are not able to see how AI can
improve their business. They need to start with a simple proof of
concept with measurable and actionable results.

There are opportunities in every industry. We see the greatest
opportunities in financial services, healthcare, and manufacturing.
In manufacturing and industrial IoT, ML is used to predict failures
so companies can take action before the failure, reduce downtime,
and improve efficiency.
There are several well-known fraud controls. Companies can know
what’s on the network, who’s on the network, what devices they are
accessing the network with, what apps they are running, whether
or not those devices are secure and have the latest security updates
and patches. This is very complex in a large organization, and AI
can handle these challenges quickly and easily.
08 The greatest concerns about AI today are the hype and issues
around privacy and security. The hype has created unrealistic
expectations. Most of the technology is still green. People are getting
too excited. There’s a real possibility that vendors may lose credibility
due to unrealistic expectations. Some vendors latch on to “hot” terms
and make it difficult for potential clients to distinguish between
what’s hype and what’s real.
As AI grows in acceptance, privacy and data security come into
play, since companies like Amazon and Google hoard data. Who
decides the rules that apply to a car when it’s approaching a
pedestrian? We’re not spending enough time thinking about the
legal implications for the consumer regarding cyberattacks and the
security of personally identifiable information (PII). We’ll likely see
more malware families and variants that are based on AI tools and
capabilities.
09 To be proficient in AI technologies, developers need to
know math. They should be willing and able to look at the data,
understand it, and be suspicious of it. You need to know math,
algebra, statistics, and calculus for algorithms; however, the skill
level required is falling as more tools become available. Depending
on the areas in which you want to specialize, there are plenty
of open source community tools, and the theoretical basics are
available on sites like Coursera.
Tom Smith is a Research Analyst at DZone who excels at gathering
insights from analytics—both quantitative and qualitative—to drive

Tremendous skillsets are required. There’s a shortage of talent and
massive competition for those with the skills. Most companies are
struggling to get the expertise they need for the application of deep

business results. His passion is sharing information of value to help people
succeed. In his spare time, you can find him either eating at Chipotle or
working out at the gym.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS




DZONE.COM/GUIDES

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Solutions Directory
This directory contains artificial intelligence and machine learning software, platforms, libraries, and
frameworks, as well as many other tools to assist your application security. It provides free trial data
and product category information gathered from vendor websites and project pages. Solutions
are selected for inclusion based on several impartial criteria, including solution maturity, technical
innovativeness, relevance, and data availability.

COMPANY

PRODUCT

CATEGORY

Accord.NET

.NET machine learning framework

AirFusion

AI-powered infrastructure monitoring

Open source

N/A

WEBSITE

accord-framework.net

airfusion.com

Alpine Data

Alpine Chorus 6

Data science, ETL, predictive analytics,
execution workflow design and management

Alteryx

Alteryx Designer

ETL, predictive analytics, spatial analytics,
automated workflows, reporting, and
visualization

Available by request

Amazon Machine Learning

Machine learning algorithms-as-a-service,
ETL, data visualization, modeling and
management APIs, batch and realtime
predictive analytics

Free tier available

Anodot

Real time analytics and AI-based anomaly
detection

Demo available by request

Apache Foundation

MADlib

Big data machine learning w/SQL

Open source

madlib.incubator.apache.org

Apache Foundation

Mahout

Machine learning and data mining on Hadoop

Open source

mahout.apache.org/

Apache Foundation

Singa

Machine learning library creation

Open source

singa.incubator.apache.org/en

Apache Foundation

Spark Mlib

Machine learning library for Apache Spark

Open source

spark.apache.org/mllib

Apache Foundation

OpenNLP

Machine learning toolkit for natural language
processing

Open source

opennlp.apache.org

Apache Foundation

Lucene

Text search engine library

Open source

lucene.apache.org/core

Amazon Web
Services

AirFusion

FREE TRIAL

Demo available by request

alpinedata.com/product

alteryx.com/products/alteryxdesigner

aws.amazon.com/machinelearning

anodot.com/product

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

COMPANY

PRODUCT

CATEGORY

FREE TRIAL

WEBSITE

Apache Foundation

Solr

Information retrieval library

Open source

lucene.apache.org/solr

Apache Foundation

UIMA

Unstructured data processing system

Open source

uima.apache.org

Apache Foundation

Joshua

Statistical machine translation toolkit

Open source

incubator.apache.org/projects/
joshua.html

Apache Foundation

PredictionIO

Machine learning server

Open source

predictionio.incubator.apache.org

Chatbot development platform

Free solution

api.ai

API.ai

Artificial Solutions

API.ai

Teneo Platform

NLI platform for chatbots

Demo available by request

artificial-solutions.com/teneo

BigML

Predictive analytics server and development
platform

Caffe2

Deep learning framework

Open source

caffe2.ai

Chainer

Neural network framework

Open source

chainer.org

Cisco

CLiPS Research
Center

Cloudera

DataRobot

EngineRoom.io

Gluru

Free tier available

MindMeld

NLP voice recognition and chatbot software

Available by request

Pattern

Python web mining, NLP, machine learning

Open source

Cloudera Enterprise Data Hub

Predictive analytics, analytic database, and
Hadoop distribution

Available by request

DataRobot

Machine learning model-building platform

Demo available by request

ORAC Platform

Gluru AI

AI and deep learning platform

AI support system

Available by request

Demo available by request

bigml.com

mindmeld.com

clips.uantwerpen.be/pattern

cloudera.com/products/enterprisedata-hub.html

datarobot.com/product

engineroom.io

gluru.co

Google

TensorFlow

Machine learning library

Open source

tensorflow.org

Grakn Labs

GRAKN.AI

Hyper-relational database for AI

Open source

grakn.ai

Grok

AI-based incident prevention

H2O

Open source prediction engine on Hadoop
and Spark

Open source

h2o.ai

Machine learning framework

Open source

heatonresearch.com/encog

Heaton Research

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Encog

14 days

grokstream.com

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

COMPANY
IBM

Infosys

Intel Nervana
JavaML

PRODUCT

CATEGORY

FREE TRIAL

WEBSITE

Watson

Artificial intelligence development platform

30 day free trial

ibm.com/watson

Nia

Artificial intelligence collection and analysis
platform

Available by request

infosys.com/nia

Intel Nervana Graph
Java-ML

Framework development library

Open source

intelnervana.com/intel-nervanagraph

Various machine learning algorithms for Java

Open source

java-ml.sourceforge.net

Open source

kaldi-asr.org

Kaldi

Speech recognition toolkit for C++

Kasisto

KAI

AI platform for chatbots

N/A

kasisto.com/kai

Keras

Deep learning library for Python

Open source

keras.io

Marvin

JavaScript callback AI

Open source

github.com/retrohacker/marvin

Convolutional neural networks for MATLAB

Open source

vlfeat.org/matconvnet

MatConvNet
Meya.ai

MatConvNet
Meya Bot Studio

Web-based IDE for chatbots

7 days

meya.ai
software.microfocus.com/en-us/
software/information-dataanalytics-idol

IDOL

Machine learning, enterprise search, and
analytics platform

Available by request

Microsoft

Cortana Intelligence Suite

Predictive analytics and machine learning
development platform

Free Azure account available

azure.microsoft.com/en-us/
services/machine-learning

Microsoft

CNTK (Cognitive Toolkit)

Open source

github.com/Microsoft/CNTK

Microsoft

Azure ML Studio

Microsoft

Distributed Machine Learning
Toolkit

Micro Focus

mlpack

mlpack 2

MXNet

Natural Language
Toolkit

Natural Language Tookit

Deep learning toolkit
Visual data science workflow app

Free tier available

studio.azureml.net

Machine learning toolkit

Open source

dmtk.io

Machine learning library for C++

Open source

mlpack.org

Deep learning library

Open source

mxnet.io

Natural language processing platform for
Python

Open source

nltk.org

Neura

AI-powered user retention platform

90 days

Neuroph

Neural network framework for Java

Open source

neuroph.sourceforge.net

OpenNN

Neural network library

Open source

opennn.net

OpenText

Magellan

The power of AI in a pre-configured platform

Open source

opentext.com/what-we-do/
products/analytics/opentextmagellan

Lambda architecture layers for building
machine learning apps

Open source

oryx.io

Neura

Oryx

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Oryx 2

theneura.com

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

COMPANY

PRODUCT

CATEGORY

Progress Software

DataRPM

Cognitive predictive maintenance for
industrial IoT

Rainbird

Cognitive reasoning platform

RainforestQA

RainforestQA Web App
Testing

AI-powered web testing platform

FREE TRIAL
Demo available by request

N/A

WEBSITE
datarpm.com/platform

rainbird.ai

Demo available by request

rainforestqa.com/product/webapp-testing

RapidMiner

RapidMiner Studio

Predictive analytics workflow and model
builder

Available by request

rapidminer.com/products/studio

RapidMiner

RapidMiner Radoop

Predictive analytics on Hadoop and Spark
with R and Python support

Available by request

rapidminer.com/products/radoop

Salesforce

Einstein

CRM automation and predictive analytics

N/A

salesforce.com/products/einstein/
overview

Samsung

Veles

Distributed machine learning platform

Open source

github.com/Samsung/veles

Scikit Learn

Machine learning libraries for Python

Open source

scikit-learn.org/stable

Predictive analytics

Open source

shogun-toolbox.org

deeplearning4j.org

Scikit Learn

Shogun

Skymind

Deeplearning4j

Deep learning software for Java and Scala

Open source

Skytree

ML model builder and predictive analytics

Available by request

Open source

spacy.io

Natural language processing toolkit

Open source

stanfordnlp.github.io/CoreNLP

Torch

Machine learning framework for use with
GPUs

Open source

torch.ch

Umass Amherst

MALLET

Java library for NLP and machine learning

Open source

mallet.cs.umass.edu

University of
Montreal

Theano

Deep learning library for Python

Open source

deeplearning.net/software/
theano/

Machine learning and data mining for Java

Open source

cs.waikato.ac.nz/ml/weka

Data stream mining, machine learning

Open source

moa.cms.waikato.ac.nz

Stanford University

Torch

spaCy

skytree.net

Python natural language processing platform

spaCy

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Stanford CoreNLP

University of Waikato

Weka

University of Waikato

Massive Online Analysis

Unravel

Predictive analytics and machine learning
performance monitoring

Wipro

HOLMES

AI development platform

Wit.ai

Natural language interface for apps

Available by request

N/A
Open source

unraveldata.com/product

wipro.com/holmes
wit.ai

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

DZONE.COM/GUIDES

G
L
O
S
S
A
R
Y

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

ALGORITHMS (CLUSTERING, CLASSIFICATION,

DEEP LEARNING

REGRESSION, AND RECOMMENDATION)

The ability for machines to autonomously mimic

A set of rules or instructions given to an AI,

human thought patterns through artificial

neural network, or other machine to help it learn

neural networks composed of cascading layers

on its own.

of information.

ARTIFICIAL INTELLIGENCE

FLUENT

A machine’s ability to make decisions and per-

A condition that can change over time.

form tasks that simulate human intelligence
and behavior.

MACHINE LEARNING
A facet of AI that focuses on algorithms, allow-

ARTIFICIAL NEURAL NETWORK (ANN)

ing machines to learn and change without being

A learning model created to act like a human

programmed when exposed to new data.

brain that solves tasks that are too difficult for
traditional computer systems to solve.

MACHINE PERCEPTION
The ability for a system to receive and interpret

CHATBOTS

data from the outside world similarly to how hu-

A chat robot (chatbot for short) that is designed

mans use their senses. This is typically done with

to simulate a conversation with human users

attached hardware, such as sensors.

by communicating through text chats, voice
commands, or both. They are a commonly used

NATURAL LANGUAGE PROCESSING

interface for computer programs that include

The ability for a program to recognize human

AI capabilities.

communication as it is meant to be understood.

CLASSIFICATION

RECOMMENDATION

Classification algorithms let machines assign a

Recommendation algorithms help machines

category to a data point based on training data.

suggest a choice based on its commonality with
historical data.

CLUSTERING
Clustering algorithms let machines group

RECURRENT NEURAL NETWORK (RNN)

data points or items into groups with

A type of neural network that makes sense of

similar characteristics.

sequential information and recognizes patterns,
and creates outputs based on those calculations

COGNITIVE COMPUTING
A computerized model that mimics the way

REGRESSION

the human brain thinks. It involves self-learning

Regression algorithms help machines predict

through the use of data mining, natural language

future outcomes or items in a continuous data

processing, and pattern recognition.

set by solving for the pattern of past inputs, as in
linear regression in statistics.

CONVOLUTIONAL NEURAL NETWORK (CNN)
A type of neural networks that identifies and

SUPERVISED LEARNING

makes sense of images

A type of Machine Learning in which output datasets train the machine to generate the desired

DATA MINING

algorithms like a teacher supervising a student;

The examination of data sets to discover and

more common than unsupervised learning

‘mine’ patterns from that data that can be of
further use.

SWARM BEHAVIOR
From the perspective of the mathematical

DATA SCIENCE

modeler, it is an emergent behavior arising from

A field of study that combines statistics, com-

simple rules that are followed by individuals and

puter science, and models to analyze sets of

does not involve any central coordination.

structured or unstructured data.

UNSUPERVISED LEARNING

DECISION TREE

A type of machine learning algorithm used to draw

A tree and branch-based model used to map de-

inferences from datasets consisting of input data

cisions and their possible consequences, similar

without labeled responses. The most common un-

to a flow chart.

supervised learning method is cluster analysis.

DZONE’S GUIDE TO ARTIFICIAL INTELLIGENCE: MACHINE LEARNING & PREDICTIVE ANALYTICS

Visit the Zone

MACHINE LEARNING

COGNITIVE COMPUTING

CHATBOTS

DEEP LEARNING

Source Exif Data:

File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.6
Linearized                      : No
Create Date                     : 2018:03:07 15:08:50-05:00
Creator                         : Adobe InDesign CC 13.0 (Macintosh)
Modify Date                     : 2018:03:07 15:12:45-05:00
Has XFA                         : No
Language                        : en-US
XMP Toolkit                     : Adobe XMP Core 5.6-c015 84.159810, 2016/09/10-02:41:30
Metadata Date                   : 2018:03:07 15:12:45-05:00
Creator Tool                    : Adobe InDesign CC 13.0 (Macintosh)
Instance ID                     : uuid:290b4d3e-690c-a942-abc3-16b4a99e6b8a
Original Document ID            : xmp.did:88fe41b6-3c9b-4add-8ced-4c56d94e38f3
Document ID                     : xmp.id:d397dfe1-8d73-4d88-b21b-224862bc05c1
Rendition Class                 : proof:pdf
Derived From Instance ID        : xmp.iid:5ff53e72-8646-44ff-8f77-7db28835215e
Derived From Document ID        : xmp.did:3ef2f79b-23ee-45bc-9613-8228950ba207
Derived From Original Document ID: xmp.did:88fe41b6-3c9b-4add-8ced-4c56d94e38f3
Derived From Rendition Class    : default
History Action                  : converted
History Parameters              : from application/x-indesign to application/pdf
History Software Agent          : Adobe InDesign CC 13.0 (Macintosh)
History Changed                 : /
History When                    : 2018:03:07 15:08:50-05:00
Format                          : application/pdf
Producer                        : Adobe PDF Library 15.0
Trapped                         : False
Page Count                      : 38

EXIF Metadata provided by EXIF.tools

8403229 Dzone Guide Artificialintelligence 2017

Navigation menu

Versions of this User Manual:

Views

Navigation