MapR Promises Integration of Apache Spark, Drill Big Data Tools

Big data vendor MapR is ntegrating Apache Drill with Apache Spark, a move that should make it possible to process diverse data sources more quickly.

Drill and Spark are disparate open source big data technologies that so far have not been closely intertwined. Drill is useful for processing data in a wide variety of formats using SQL queries, while Spark allows in-memory data analysis, which can deliver huge performance increases.

MapR will be working with Databricks, the company founded by Spark’s creators to provide services related to the software, for the purposes of the integration.

By combining the two technologies, MapR said it can deliver high-performance data processing that does not require special data preparation or programming skills. “Integrating Apache Drill and Spark simplifies the development of data pipelines and opens up Drill SQL-based ad-hoc queries on in-memory data,” said M.C. Srivas, CTO and cofounder at MapR Technologies. “Joining forces with Databricks to leverage our combined breadth and depth of technical resources to accelerate innovation is a huge win for customers.”

From the channel perspective, this is a sign of the maturation of open source big data technologies beyond Hadoop, the first open source big data tool to become commercially important. Hadoop remains hugely influential, but vendors are now shifting focus to smoothing over the rough edges of big data by building tools to make processing faster and easier.