The Off Topic Topic

[DGuller]

It's interesting because it may show a fundamental limitation/advantage of purely numeric systems such as neural nets.

If you go by the description in the article, what happened was not some novel failure but rather the one extreme of what can always happen when you build an autoencoder-type network like this.  One of the extreme outcomes is that you learn features that are too simplistic to be useful.  The other extreme outcome is that you memorize the input without simplifying it into relatively few useful features. 

All this article seems to say is that researchers got into the latter extreme during an early iteration of their model.  It's not that uncommon, probably every practitioner accidentally builds a perfect model from time to time, you know to watch out for it.

[Maximus]

If you go by the description in the article, what happened was not some novel failure but rather the one extreme of what can always happen when you build an autoencoder-type network like this.  One of the extreme outcomes is that you learn features that are too simplistic to be useful.  The other extreme outcome is that you memorize the input without simplifying it into relatively few useful features. 

All this article seems to say is that researchers got into the latter extreme during an early iteration of their model.  It's not that uncommon, probably every practitioner accidentally builds a perfect model from time to time, you know to watch out for it.

Overfitting and underfitting are indeed uninteresting. I don't see that happening here, though. Rather what seem to have happened is a limitation of their data. They didn't have a X->Y ground truth function because that would require manual labeling which is expensive, so they tried to approximate it with a X->Y->X function.

[DGuller]

If you go by the description in the article, what happened was not some novel failure but rather the one extreme of what can always happen when you build an autoencoder-type network like this.  One of the extreme outcomes is that you learn features that are too simplistic to be useful.  The other extreme outcome is that you memorize the input without simplifying it into relatively few useful features. 

All this article seems to say is that researchers got into the latter extreme during an early iteration of their model.  It's not that uncommon, probably every practitioner accidentally builds a perfect model from time to time, you know to watch out for it.

Overfitting and underfitting are indeed uninteresting. I don't see that happening here, though. Rather what seem to have happened is a limitation of their data. They didn't have a X->Y ground truth function because that would require manual labeling which is expensive, so they tried to approximate it with a X->Y->X function.

The X->Y->X model is what's called the autoencoder model.  The point of that model is for it to predict its own inputs.  Obviously that could be a trivially easy task if your model has enough capacity to just memorize and regurgitate its input, but a meaningful autoencoder would leave you with Ys in the middle that would be a useful simplification of your data. 

The trick is to build it so that Ys capture enough information, and the most useful information, but not so much information that it's just an overly elaborate lossless data compression.  It's very hard to build useful autoencoders, and building rote memorizers is almost inevitable at some point during the development.  That's the analog of overfit supervised models.

[Admiral Yi]

What do you guys think about the American Civil War?

[DGuller]

What do you guys think about the American Civil War?

It was okay.

[Valmy]

What do you guys think about the American Civil War?

I think the Army of the Tennessee was an army of bad-asses.

[Syt]

RIP "Mean" Gene Okerlund (age 76)

https://www.ctvnews.ca/entertainment/wrestling-announcer-mean-gene-okerlund-dies-at-age-76-1.4237721

[Caliga]

What do you guys think about the American Civil War?

STATES RAHTS

[Habbaku]

Sharpsburg. Manassas.

[alfred russel]

What do you guys think about the American Civil War?

It was neither American, nor Civil, nor a War.

[alfred russel]

What do you guys think about the American Civil War?

It was neither American, nor Civil, nor a War.

Wait, I'm getting that mixed up with the Holy Roman Empire.

It was the Holy Roman Empire that was neither American, nor Civil, nor a War.

[The Minsky Moment]

What do you guys think about the American Civil War?

Which one?

[FunkMonk]

What do you guys think about the American Civil War?

It was neither American, nor Civil, nor a War.

Wait, I'm getting that mixed up with the Holy Roman Empire.

It was the Holy Roman Empire that was neither American, nor Civil, nor a War.

I'm starting to feel verklempt.

[Eddie Teach]

What do you guys think about the American Civil War?

Which one?

Upcoming.

[Syt]