With New Datawatch Partnership, IBM Continues to Enable the ‘Citizen Analyst’

IBM (IBM) is looking to the future and sees a problem. The Internet of Things (IoT) is already producing data at a rate that’s difficult to manage. Analyst firm Gartner says there’re about 5 billion connected “things” in the world today. By 2020, that number will quintuple to 25 billion. Our world will turn on big data, but who will analyze it for us?

“We’re creating bigger data problems than ever before, and are we keeping up? Is our skillset keeping up with this growth?” asks Robin Grosset, chief architect of IBM Watson Analytics. No, he says. The problem is that “…we’re not producing enough data scientists to crunch the numbers that we’re generating right now, and we need help.”

That help may come in the form of IBM Watson Analytics, which debuted in 2014. One of the goals of the technology is to enable the rise of the “citizen analyst.” These are the everyday line-of-business (LOB) users and their partners who want to mine the ever-increasing amounts of data they produce to support business hypotheses without having to turn to an often slow-moving IT department or expensive, complicated enterprise-grade platforms. Watson Analytics is a cognitive computing system that uses natural language processing and machine learning to help day-to-day knowledge workers glean insights from unstructured data.

One of the big hurdles users face with any analytics platform, even Watson, is the sheer amount of time it takes to input data in order to even begin to crunch the numbers. Last year, Forrester said that analysts spend up to 80 percent of their time just trying to create useable data sets. IBM hopes that its latest partnership with Datawatch, announced last week, will help reduce that time. Datawatch’s Monarch data shaping and preparation tool can now be configured to feed data directly into Watson Analytics and Cognos Analytics. “This partnership is all about making it easy for organizations…to load their data into our offerings,” says Grosset. “It’s a big enabler targeted at everyday people, not the super-technical.”

With Watson and Cognos, IBM is trying to help ordinary people fill the skills gap created by the explosion of data the world is creating today and the lack of trained data scientists to analyze it. Not that long ago, data analytics was the domain of big organizations with deep pockets, vast data warehouses and highly managed infrastructure. What IBM sees is an increase in “ad hoc analysis,” or what Grosset calls “data discovery.” The people using analytics are becoming more consumer-oriented. “Everybody wants a self-service experience,” Grosset says. “That’s the change that we’re leveraging here.”

IBM recognizes that simply providing a tool to crunch data isn’t enough to raise the collective skillset to a level that can navigate the onslaught of information the IoT will generate. Users need to be able to learn, at least to a degree, the basics of analytics. That, Grosset says, is where Watson’s cognitive computing platform and partners step in to help.

“Cognitive systems are computer systems that act at a more human level. They understand, perceive, relate information,” Grosset explains. Using Watson, users can load data, ask questions of the data using natural language, and receive analysis and even visualizations from the system. The Watson architects want the process to be simple, but also transparent. “If we’re applying some sort of advanced analytic on behalf of the user, you can go into all of the detail about what was done, all the way down into the machine learning algorithm that was used,” he says. Giving users access to these deep analytic details is key to teaching data analysis skills.

If cognitive systems make it possible for software to guide users, to explain how and why it made certain decisions, then in essence those systems are educating the people who are using them. The technology becomes the teacher, and the user gains a new skill set that allows them to self-service. “It’s a very human interaction that you have with the software, and it’s making it possible for a completely different group of people to do analytics,” Grosset says. “It gets them started, but then also the tools have the depth for people to grow and to learn more.” The more a person uses Watson, in other words, the more he or she learns. In this way, IBM’s “citizen analyst” is filling the gap of data professionals in our constantly connected world.

So what does this brave new world of democratized IT look like to Grosset? “There used to be this thought that artificial intelligence and machine learning was a particular discipline. There used to be this thought that data warehousing and data processing was a particular discipline. There were these islands. And the same was true of analytics,” Grosset says. But all of these disciplines are joining together now, and the demarcation lines both between disciplines and between the trained and untrained are becoming blurred.

In the near future, when the Internet of Things has gone from a buzzword to simply the reality in which we live, the amount of data we will generate will far outpace our ability to formally train enough data scientists to manage it all. Just as we had to learn, as a society, how to navigate a digital world, everyday people will have to develop new capabilities to handle all of that information. With the Datawatch partnership, Watson hopes to make it easier for LOB workers and service providers to step in and fill the data management gap.