An algorithm can predict human behavior better than humans

[jimmy olsen]

The singularity and the inevitible robot rebellion and genocide of the human race that will follow draws ever closer.  

http://qz.com/527008/an-algorithm-can-predict-human-behavior-better-than-humans/

An algorithm can predict human behavior better than humans

Olivia Goldhill
October 18, 2015

You might presume, or at least hope, that humans are better at understanding fellow humans than machines are. But a new MIT study suggests an algorithm can predict someone's behavior faster and more reliably than humans can.


Max Kanter, a master's student in computer science at MIT, and his advisor, Kalyan Veeramachaneni, a research scientist at MIT's computer science and artificial intelligence laboratory, created the Data Science Machine to search for patterns and choose which variables are the most relevant. Their paper on the project results (pdf) will be presented at the IEEE Data Science and Advanced Analytics conference in Paris this week.


It's fairly common for machines to analyze data, but humans are typically required to choose which data points are relevant for analysis. In three competitions with human teams, a machine made more accurate predictions than 615 of 906 human teams. And while humans worked on their predictive algorithms for months, the machine took two to 12 hours to produce each of its competition entries.


For example, when one competition asked teams to predict whether a student would drop out during the next ten days, based on student interactions with resources on an online course, there were many possible factors to consider. Teams might have looked at how late students turned in their problem sets, or whether they spent any time looking at lecture notes. But instead, MIT News reports, the two most important indicators turned out to be how far ahead of a deadline the student began working on their problem set, and how much time the student spent on the course website. These statistics weren't directly collected by MIT's online learning platform, but they could be inferred from data available.


The Data Science Machine performed well in this competition. It was also successful in two other competitions, one in which participants had to predict whether a crowd-funded project would be considered "exciting" and another if a customer would become a repeat buyer.


Kanter told MIT News that there are many possible uses for his Data Science Machine. "There's so much data out there to be analyzed," he said. "And right now it's just sitting there not doing anything."

[DGuller]

Meh.  This sounds a lot more impressive than it really is. 

As a matter of fact, I was one of those 615 human teams that were beaten by that algorithm?  Why?  Because I made one very, very basic submission, and then switched my focus on another competition that I found more interesting and applicable to my day job.  A large share of such "human teams" may have likewise been very unserious entries, so beating them means nothing.  And in the context of predictive modeling, there is a world of difference between an AUC of 0.86 that they achieved, and an AUC of 0.91 that the winning team achieved.  I think my job is safe for now.

[DGuller]

Ugh, is this an actual scientific peer-reviewed paper?  This reads like a marketing spiel, full of metrics that make zero sense, trying to make very mediocre performance look like anything special. 

Auto-tuning machine learning algorithms aren't anything new.  The ones that can compete with a moderately-experienced human would be new, but this is very far from it.