Languish.org

General Category => Off the Record => Topic started by: Razgovory on March 22, 2019, 02:45:58 PM

Title: statistics question
Post by: Razgovory on March 22, 2019, 02:45:58 PM
If an event has a 30% of happening and there are 100 attempts what is chance that it will happen at least once?
Title: Re: statistics question
Post by: The Minsky Moment on March 22, 2019, 03:37:06 PM
It depends whether each attempt is independent.
If it is, the chance is extremely close to 100%.  1 - (0.7^100)
Title: Re: statistics question
Post by: Liep on March 22, 2019, 03:51:46 PM
In what situations wouldn't it be independent?
Title: Re: statistics question
Post by: Admiral Yi on March 22, 2019, 03:54:39 PM
Quote from: Liep on March 22, 2019, 03:51:46 PM
In what situations wouldn't it be independent?

Non-replacement.  I.e. you don't put the ball back in the jar after you pull it.
Title: Re: statistics question
Post by: Razgovory on March 22, 2019, 04:25:07 PM
Quote from: The Minsky Moment on March 22, 2019, 03:37:06 PM
It depends whether each attempt is independent.
If it is, the chance is extremely close to 100%.  1 - (0.7^100)


How did you calculate it.
Title: Re: statistics question
Post by: Iormlund on March 22, 2019, 04:46:18 PM
You calculate the chance that it never happens instead. 0.7^100 (70% 100 times). Once you know that, your subtract it from 1 (100% probability). So 100% minus the probability of no event will be what you are looking for.
Title: Re: statistics question
Post by: mongers on March 22, 2019, 05:01:30 PM
Quote from: Razgovory on March 22, 2019, 04:25:07 PM
Quote from: The Minsky Moment on March 22, 2019, 03:37:06 PM
It depends whether each attempt is independent.
If it is, the chance is extremely close to 100%.  1 - (0.7^100)

How did you calculate it.

There's about a one in 3 thousand trillion chance of it not happening.

Title: Re: statistics question
Post by: The Minsky Moment on March 27, 2019, 12:10:16 PM
Quote from: Admiral Yi on March 22, 2019, 03:54:39 PM
Quote from: Liep on March 22, 2019, 03:51:46 PM
In what situations wouldn't it be independent?

Non-replacement.  I.e. you don't put the ball back in the jar after you pull it.

This actually came up in the 2018 election.  A lot of election modelers gave Trump a very low chance to win because he was down in the polls in a bunch of different states.  They figured to win he'd have to string together a bunch of 20-30% chances, which is very unlikely if winning one doesn't increase your chance of winning another.  But a few modelers - 538 was one - didn't model those outcomes independently - i.e. they figured if Trump's support was actually stronger in Ohio then the polling suggested, he was probably also actually stronger in Pennsylvania and other places.  Of course that was the right way to look at it but immediately pre-election there was sharp debate on that point.
Title: Re: statistics question
Post by: DGuller on March 27, 2019, 02:26:56 PM
There was?  It seems incredibly stupid for anyone with professional training to ignore correlation of surprises in voting outcomes, I'm surprised there can be a debate about it.  Then again, some professionals also assumed that mortgages default independently, so I guess I shouldn't be too surprised.
Title: Re: statistics question
Post by: FunkMonk on March 27, 2019, 07:07:02 PM
I remember the debate about it back then too.
Title: Re: statistics question
Post by: alfred russel on March 28, 2019, 12:43:37 PM
Quote from: DGuller on March 27, 2019, 02:26:56 PM
There was?  It seems incredibly stupid for anyone with professional training to ignore correlation of surprises in voting outcomes, I'm surprised there can be a debate about it.  Then again, some professionals also assumed that mortgages default independently, so I guess I shouldn't be too surprised.

On the one hand, you had people making sensible statistical arguments. On the other hand, the people making sensible statistical arguments were asserting that an unqualified idiot racist troll had a real chance to be elected president.
Title: Re: statistics question
Post by: The Minsky Moment on March 28, 2019, 01:14:31 PM
Post from the old 2016 election thread

Quote from: The Minsky Moment on November 03, 2016, 03:00:07 PM
Quote from: Jacob on November 03, 2016, 01:53:31 PM
Sam Wang has been giving Hillary 99% for quite a while, with fewer "intangibles" in their analysis than 538.

I took a quick glance at his FAQ.  I may not be understanding his method properly.  But it seems that he assumes away the possibility of correlated errors between states, which IMO will over-state certainty of outcome.  He also seems to assume pollster error is completely uncorrelated, with the same effect.

The eyeball test tells me that 99% vastly understates the degree of uncertainty inherent in the state polling and if I'm understanding the methodology correctly, I don't think its reliable.  Silver's model may be too swingey and fat-tailed but takes into account serial correlation.   
Title: Re: statistics question
Post by: DGuller on March 28, 2019, 02:22:46 PM
Whew.

Quote from: DGuller on November 03, 2016, 05:53:24 PM
Quote from: The Minsky Moment on November 03, 2016, 03:00:07 PM
Quote from: Jacob on November 03, 2016, 01:53:31 PM
Sam Wang has been giving Hillary 99% for quite a while, with fewer "intangibles" in their analysis than 538.

I took a quick glance at his FAQ.  I may not be understanding his method properly.  But it seems that he assumes away the possibility of correlated errors between states, which IMO will over-state certainty of outcome.  He also seems to assume pollster error is completely uncorrelated, with the same effect.

The eyeball test tells me that 99% vastly understates the degree of uncertainty inherent in the state polling and if I'm understanding the methodology correctly, I don't think its reliable.  Silver's model may be too swingey and fat-tailed but takes into account serial correlation.   
If you're right about his methodology, then I definitely agree with you.  Assuming state-by-state uncertainty is uncorrelated is an extremely severe error of analysis.
Title: Re: statistics question
Post by: The Brain on March 28, 2019, 03:40:37 PM
Quote from: The Brain on January 04, 2017, 12:15:49 PM
Today my bathtub wasn't draining very quickly so I used a plunger.
Title: Re: statistics question
Post by: The Minsky Moment on March 28, 2019, 03:47:19 PM
Overall 538 did a pretty good job with the 16 election, arguably as good as in 2012 when they were feted for getting it "right".  A probability in the 30s is something that has a pretty decent likelihood of happening although it is somewhat more likely not to happen. 
Title: Re: statistics question
Post by: crazy canuck on March 28, 2019, 04:17:37 PM
Quote from: The Brain on March 28, 2019, 03:40:37 PM
Quote from: The Brain on January 04, 2017, 12:15:49 PM
Today my bathtub wasn't draining very quickly so I used a plunger.

Did you sober up in time to realize that was not your bathtub?
Title: Re: statistics question
Post by: The Brain on March 28, 2019, 04:19:44 PM
Quote from: crazy canuck on March 28, 2019, 04:17:37 PM
Quote from: The Brain on March 28, 2019, 03:40:37 PM
Quote from: The Brain on January 04, 2017, 12:15:49 PM
Today my bathtub wasn't draining very quickly so I used a plunger.

Did you sober up in time to realize that was not your bathtub?

I don't remember.