MongoDB Fills Platform's Gaps with Atlas Data Lake Preview Debuting in AWS

(Pictured above: MongoDB’s Eliot Horowitz on stage at MongoDB World in New York City, June 18.)

MongoDB is filling some key gaps with the addition of rich text-level search, field-level encryption, visualization and a forthcoming data lake that will let partners create modern, serverless, query-based solutions.

The move comes amid soaring growth of MongoDB’s popular general database platform.

At its annual MongoDB World conference Tuesday in New York City, the company revealed that the much-needed data lake offering is now available in preview in the Amazon Web Services (AWS) cloud with support coming to Microsoft Azure and Google Cloud. In a demo of the new MongoDB Atlas Data Lake during the MongoDB World keynote session, co-founder and CTO Eliot Horowitz created a query against data aggregated among distributed Amazon S3 storage buckets without connecting or moving the data into the database.

“It’s completely serverless. You don’t set up servers, you don’t configure anything, you don’t pay for infrastructure, you only pay for what you use — for data transfer from S3 to our compute, from our compute back to you,” Horowitz said. “That’s it. You’re not moving the data from your S3 buckets to our data lake. You’re not paying for storage and doubling up on storage costs. We query the data exactly where it is.”

Data lakes such as Amazon Athena, Azure Data Lake and Google BigQuery recently have ushered in an increasingly popular new approach to allowing modern cloud-native applications to perform rapid and concurrent queries against distributed data.

“The trend is that users want to decouple storage from compute,” said IDC analyst Carl Olofson. “The obvious answer is, you put it in a data lake and then enable people to run their favorite analytic tools against it.”

While it will compete with AWS Athena, that’s not a problem for Amazon or the other players, who are happy to sell the capacity in any way customers choose.

“It has become clear to me in talking with Amazon that they are a retailer, and if they can sell other people’s products on their platform, they’re OK with that,” Olofson said. “It’s like going into the supermarket and they’ll sell you the store brand or anything else.”

MongoDB Atlas Data Lake enables queries of data in any modern format — notably JSON, BSON, CSV, TSV, Parquet and Avro with its MongoDB Query Language (MQL).

T”here’s a lot of data in MongoDB and they had to go there with this,” said James Kobielus, lead analyst for data science, deep learning and application development at Wikibon. “They call themselves an enterprise data platform, but you can’t be a full platform unless you’ve got the data science workbench,” Kobielus said. “Now they’ve got their foot in the door, but it’s only a foot at this point. Hopefully at next year’s MongoDB World, they’ll have a stronger story here.”

Trinh Nguyen, senior director of technology for NBCUniversal’s E! Entertainment business, a MongoDB customer, said he can see potential use for the new data lake offering.

“The data lake is a very interesting concept,” he told Channel Futures. “To be able to use all of the data that you have on S3, without actually having to spin up new infrastructure to actually run those types of queries, is probably really powerful.”

Charleste King, MongoDB practice lead at Datavail, reserved judgement, having just learned of the new offering.

“You just have to make sure that it’s used appropriately and what is designed for,” King said.

Datavail is the partner that helped E! Entertainment shift its managed MongoDB Enterprise implementation to MongoDB Atlas in AWS, which King and Nguyen explained during a session at MongoDB World. Asked about demand for services around MongoDB, King said it is …

… on the rise.

“Our MongoDB practice has tripled in the last six months,” she said.

For its part, MongoDB Atlas revenue grew 340% in the most recent quarter, accounting for 35% of all revenue, which overall surged 74%. The company reported 800 new customers, bringing the total to 14,200, reported earlier this month.

Among the other solutions MongoDB announced: