The AI dooooooom thread

[viper37]

Silence Where a Story Might Have Been 


JMS on why AI hasn't replaced writers and why it's not happening anytime soon.

[Jacob]

Little interesting detail from Gillian Tett in the FT.

She said she'd met a New York finance guy who said the 2025 summer interns "were the first true AI natives" they'd seen. She asked how it went: "he winced. While those wannabee masters of the universe initially seemed wildly imressive, when senior financiers later probed their ideas they found them alarmingly shallow." Apparently they made fewer offers and are now moving their recruitment strategy to focus less on STEM grads and more on humanities.

It would be ironic indeed if the rise of AI resulted in valuing the humanities again.

Though, I suppose, that is likely contingent on humanities education avoiding falling into students becoming dependent on AI to do their thinking for them as well.

[Sheilbh]

The Pope's long-awaited AI encyclical, Magnifica Humanitas, is out. Some favourite bits:

10. We must, then, avoid the "Babel syndrome," namely the idolatry of profit that sacrifices the weak, a uniformity that neutralizes differences, and the pretense that a single language — even a digital one — can translate everything, including the mystery of the person, into data and performance. The risk of dehumanization — of building a future that excludes God and reduces the other to a means — is an ancient and ever-new temptation that today takes on a technical guise.  [...]
[...]
109. The principles of Social Doctrine offer a framework for understanding this new reality. In a world where data, computational resources and regulatory influence remain in the hands of a few, to speak of the common good means exposing this new form of epistemic, economic and political asymmetry and naming the new monopolies of AI. To speak of the universal destination of goods means finding ways of ensuring universal access to both technologies and the education needed to use them. To speak of subsidiarity calls for protecting the ability of communities to make choices and corrections, rather than confining their role to mere oversight after the standards have been set elsewhere. To speak of solidarity obliges us to recognize the hidden, often exploited workers, who sustain algorithmic systems. To speak of justice requires questioning the global distribution of power that decides who in fact can train these models and who is merely subjected to them. Likewise, it means acknowledging that social justice is not only a goal to be safeguarded after technologies are deployed, but a condition that must shape their very design from the outset.

110. Finally, I would like to employ the expression "to disarm," which is close to my heart. Disarming AI means freeing it from the mentality of "armed" competition, which today is not limited simply to the military context, but is also an economic and cognitive phenomenon. This entails a race for ever more powerful algorithms and larger datasets, driven by the desire to secure geopolitical or commercial dominance. To disarm means discrediting the assumption that technical power automatically confers the right to govern. To disarm does not mean rejecting technology, but preventing it from dominating humanity. It means freeing technology from monopolistic control and opening it to discussion and debate, therefore making it human-friendly and restoring it to the plurality of human cultures and ways of life. Our task today is not only ethical or technical. It is ecological in the deepest sense, for it concerns a new dimension of our common home. AI is already an environment in which we are immersed, as well as a force with which we must engage. For this reason, merely regulating it is insufficient; it must be disarmed, welcoming and accessible.
[...]
172. At the root of these problems lies a technocratic and post-humanist mentality that tends to regard the human person as an object to be manipulated or a resource to be optimized, [172] removing all safeguards against the unchecked pursuit of profit. What prevails is efficiency, rather than respect for freedom and human dignity. Some post-humanist currents even go so far as to envision "second-class" human beings, subordinate to the interests of elites who consider themselves superior. This troubling prospect becomes all the more serious when combined with technological tools that exponentially increase the capacity for control and selection. Even certain forms of structural indebtedness, which keep entire peoples in conditions of dependence, reflect the same mentality, in new forms, that tolerates relationships of subordination akin to slavery.

Breaking the chains of new forms of slavery

173. This distorted view of the human person is reflected today in various forms of servitude directly linked to the digital economy. Nothing in the world of AI is immaterial or magical. Every seemingly immediate and flawless response is the result of a long chain of mediation, involving vast networks of natural resources, energy infrastructure and, above all, people. A significant part of the digital economy's functioning relies on the silent work of millions of people engaged in essential yet largely unseen activities, such as data labeling, model training and content moderation, often involving disturbing material. In many cases, these workers are young people, predominantly women, working under demanding conditions for minimal wages. Added to this invisible labor is the even harsher work of extracting the resources required for the production of the devices and microprocessors on which AI depends. In some regions of the world, children and adolescents work in dangerous conditions, crushing the materials from which rare earth elements are extracted. The bodies of these people are scarred, injured and worn down so that computational flow may continue uninterruptedly. Furthermore, criminal networks use online platforms, messaging systems, anonymous payment methods and profiling techniques in order to recruit, control and transport victims of trafficking — very often minors — reducing men and women to "data" to be tracked and "packages" to be moved around within the same digital circuits that support much of the global economy. This reality deeply challenges the moral conscience of our time. It is not enough to invoke efficiency, nor to celebrate the benefits of innovation, if they are built on a chain of exploitation that remains deliberately hidden. If technology promises emancipation, yet produces new forms of global subordination, it stands in contradiction to the fundamental principle of human dignity.

[mongers]

The Pope's long-awaited AI encyclical, Magnifica Humanitas, is out. Some favourite bits:

Exec.sum.pls.

[The Minsky Moment]

The Pope's long-awaited AI encyclical, Magnifica Humanitas, is out. Some favourite bits:

Exec.sum.pls.

Tech Barons are modern embodiments of Heinrich IV. Time to Canossa the MFers.

[HisMajestyBOB]

If the Pope calls for a crusade against Silicon Valley I might just convert to Catholicism. 

[Syt]

https://futurism.com/artificial-intelligence/academics-meltdown-ai-hallucinations-research

Academics in Meltdown Now That They're Responsible for AI Hallucinations in Their Research Papers

"So this means you expect every author to check every citation and make sure that every citation is real and accurate?"

Even in 2026, there are still plenty of researchers who refuse to use AI to publish their research papers. Others do use the tech for tasks like sourcing journal articles for references, editing copy, or formatting citations — but they face pressure to verify every claim, since AI has a baked-in risk of contaminating their work with hallucinations.

A vocal minority of academics, however, argue they should be able to use AI to write original research while remaining immune from any hallucinated claims or data that make their way into the final product.

Last week, the open-source research repository arXiv announced that it was banning scholarly authors from the platform for up to a year if "hallucinated references" are found in their work. The rationale behind this should be obvious enough for any self-respecting academic: as arXiv computer science chair Thomas Dietterich wrote in his announcement, "if a submission contains incontrovertible evidence that the authors did not check the results of LLM generation, this means we can't trust anything in the paper."

As TechCrunch observed, arXiv isn't banning AI altogether, but simply clarifying that the author is ultimately responsible for any work that goes out under their name. Makes sense, right?

Apparently not. After Dietterich's announcement of X-formerly-Twitter, numerous researchers immediately went on the offensive, trashing the platform for its decision.

"So this means you expect every author to check every citation and make sure that every citation is real and accurate?" economics professor at Smith College James Miller replied in shock. "What if it's beyond the ability of one of the authors to verify one of the citations because that citation is in a language he doesn't know or concerns technical material he doesn't understand but another author on the paper does?"

"This is way too strict. Errors can slip in when using any tools. We aren't perfect," said Luca Ambrogioni, assistant professor in AI at the Donders Institute for brain, cognition and behaviour. "Having a prompt left in is a mistake, it's sloppy but giving permanent answer a one time sloppiness is absurd."

Ambrogioni, who appears to argue that getting reprimanded via arXiv's policy on hallucinated citations will amount to a de facto "lifetime ban" from publishing, continued: "we are not taking just about false citations (more serious), but also more harmless copy pasting editing mistakes. Papers are long, the likelihood of an incorrect copy past in the supplementary isn't zero even in a otherwise good quality work."

Neal Amin, a former neuroscientist and Stanford medical clinician turned biotech startup founder, wrote on X that "this is what overreaction looks like and how gatekeeping starts."

[Oexmelin]

What a fucking moron.

[crazy canuck]

I am seeing this mindset gain same ground, and even before AI.

"What do you mean I am responsible for ensuring all the citations are accurate and appropriate?  I can't be responsible for a glitch in the citation software my lab uses"



Same song different verse.


But I hasten to add that is still a very small minority of academics.

[Sheilbh]

Maybe - I think it maybe was a bit broader even if there's lots of academics reacting to this in a way that is telling on themselves (especially in the STEM world where you have multiple authors - and I do kind of get their concern there). For example, the post-crash intellectual underpinning of austerity and particularly ideas around how it interacts with growth was a paper where there was basically an Excel error. It was repeatedly cited by the IMF, the Commission, think tanks, policy papers for several years (during which it was one of the most cited economics papers) before some researchers actually looked at what they were footnoting.

In that case, the politics still mattered but this is what gave intellectual respectability/credibility to the catastrophic mishandling of the Eurozone crisis and other austerity programs.

[The Minsky Moment]

Reihart-Rogoff!  Yeah that was a memorable one.  That paper already had serious issues before the Excel error came out, and it was compounded by the fact that even taking it on its face, it was cited for broader propositions than it ever really stood for.

[The Minsky Moment]

Ye olde R-R Excel error thread: https://languish.org/forums/index.php/topic,9694.msg565989.html#msg565989

[garbon]

I see Ide's blog is still alive and kicking.

[Syt]

Good for him.

[crazy canuck]

Maybe - I think it maybe was a bit broader even if there's lots of academics reacting to this in a way that is telling on themselves (especially in the STEM world where you have multiple authors - and I do kind of get their concern there). For example, the post-crash intellectual underpinning of austerity and particularly ideas around how it interacts with growth was a paper where there was basically an Excel error. It was repeatedly cited by the IMF, the Commission, think tanks, policy papers for several years (during which it was one of the most cited economics papers) before some researchers actually looked at what they were footnoting.

In that case, the politics still mattered but this is what gave intellectual respectability/credibility to the catastrophic mishandling of the Eurozone crisis and other austerity programs.

Yes, that is exactly the point. The handful of academics who gripe about being held accountable are generally the ones whose body of work  does not meet a high academic standard.