At Adeptia, we like to solve thorny integration problems for our customers. In fact, the more complex the integration project, the better we shine! We thought we’d devote some time to looking at how people are solving other thorny or complex challenges…and so here’s another addition to our series on examining the novel ways people or companies have found to get across the finish line.

A next-generation computer system is using artificial intelligence (AI) to exponentially increase the speed and effectiveness at which data can be analyzed. The Data Science Machine (DSM), developed at MIT’s Computer Science Artificial Intelligence Lab, can outperform teams of really smart humans. Moreover, the new machine can not only analyze data, but it can also qualitatively parse that data.

“We view the Data Science Machine as a natural complement to human intelligence,”explains Max Kanterwhose MIT master’s thesis in computer science led to the development of this new supercomputer. “There’s so much data out there to be analyzed. And right now it’s just sitting there not doing anything. So maybe we can come up with a solution that will at least get us started on it, at least get us moving.”

Kanter presented his paper in November at the prestigious IEEE International Conference on Data Science and Advanced Analytics. The DSM is remarkable, in part, because of its ability to analyze “big data” for relevant patterns—without human intervention. Choosing which “features” of a data set to analyze usually requires some human analysis. One example used to illustrate this point is a database containing the beginning and end dates of various sales promotions and profits. The data with meaningful predictive power may not be the dates or the profits themselves, but rather how those results trend over time.

To test the effectiveness of the DSM, MIT researchers put it to a head-head test with 906 “teams” of humans via three different science competitions. In two of the three competitions, the DSM’s predictions were 94 percent and 96 percent as accurate as of the winning submissions. Although it was only 87 percent as accurate in the third.

But here’s where it gets really interesting for “real world” applications that analyze Big Data: It took months for the humans to develop their predictive algorithms. By contrast, the Data Science Machine took less than 12 hours to produce each of its entries.

Kalyan Veeramachaneni, Kanter’s thesis adviser and a research scientist at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), underscores the implications for applying machine-learning techniques to practical problems in big-data analysis. “What we observed from our experience solving a number of data science problems for industry is that one of the very critical steps is called feature engineering,” he said. “The first thing you have to do is identify what variables to extract from the database or compose, and for that, you have to come up with a lot of ideas.”

Kanter thinks one immediate benefit of the DSM will be to facilitate Big Data analysis without requiring trained data scientists to interpret the information first. “I think the biggest potential is for increasing the pool of people who are capable of doing data science,” he told IEEE Spectrum. “If you look at the growth in demand for people with data science abilities, it’s far outpacing the number of people who have those skills.”

Moreover, observes Margo Seltzer, a professor of computer science at Harvard: “The Data Science Machine is one of those unbelievable projects where applying cutting-edge research to solve practical problems opens an entirely new way of looking at the problem.”

Seltzer, who was not part of the DSM project, told MIT News: “I think what they’ve done is going to become the standard quickly — very quickly.”

Greg Sandler, a B2B content development expert and freelance writer, has worked on a wide range of business integration and web development projects. He also has written for hundreds of publications, organizations, government agencies, and private sector clients. In addition to editorial experience, Greg has extensive copywriting and scriptwriting experience. He also has both print and online custom publishing and advertorial experience. Check out hisprofile on LinkedInor send him ane-mail.

For further reading on ways, Adeptia is transforming the integration industry, check out: How To Convert XML to CSV In Under 10 Minutes

Adeptia Connect 2.0 Shortens Customer Onboarding From Months To Minutes