What moves swing voters

[The Minsky Moment]

It's not that cut and dried in practice.  Despite popular belief, you are allowed to use judgment in reaching conclusions from statistical data.  If the authors on some level felt that the "bars not overlapping" standard was too strict for practical considerations, they would not be automatically wrong in going "we believe that the differences are significant despite wide confidence intervals due to low sample size".  It would be a case of their judgment making up for the overly strict implicit assumption that all categories are perfectly negatively correlated with each other.

But they did not exercise their judgment in that way, they exercised it in the the opposite way.

And I don't see how in this case judgment could be used in any other way.  The standard I gave is "meaningful and statistically significant".  Putting aside the sample size and the error bars, the raw % of approval for the two messages is 53 vs 47.  Again this study is in the nature of a survey where there was no testing of sensitivity to different formulations of the soundbite.  (For example, and purely hypothetically, imagine changing the woke prog message from "transformative change" to "real change" - the message is substantively the same but the language is simpler and more direct.  Does that change the result?)

Any political analyst that based messaging solely on the basis of this finding in this study would IMO be guilty of malpractice.  To be clear, I think the conclusion is broadly correct.  I just don't think these particular findings of this particular study is the killer piece of evidence supporting the conclusion and the breathlessness of the NYT piece is misplaced.

[DGuller]

I e-mailed the author for the link to GitHub.  Let's see what happens. 

I finally got a response.  They won't be releasing the data for the time being due to ongoing research, but hope to release it soon.    I don't know about that, I think that when you say that reproducible code is available, it should be available now, not some time later.  I'm definitely not trusting this report until I get my hands on the reproducible code.