The AI dooooooom thread

[crazy canuck]

I mean, that's not really surprising.  The nature of 'AI' is such that it can never truly be accurate.  All it does is throw in the most probable words one after another, and if pushed back on tells the user exactly what they want to hear. 

That's like saying that all a nuclear bomb does is trigger a runaway chain reaction.  It's a technically accurate thing to say, but it trivializes a very complex thing behind the "all it does".  Getting the conditional probabilities of the next token right depending on the context is kind of where the synthesis of concepts happens in an LLM.

Sure, but the point is that everything a LLM does is worthless, since you can never be sure if it's lying to you or not, and it can never be dependable. 

Lying feels like the wrong word as that feels like attributing human motivations to it.

It can provide inaccurate/false information. Of course, it generally is the case that predictions aren't 100% accurate. After all, when companies buy market research, it isn't because they are going to get a 100% accurate picture of what is currently happening and predictions for the future but that it is better than the knowledge they have if they only relied on their fieldforce, annecdotal customer feedback and sales data.

What does the fact that generative AI fabricates information have to do with the prediction of the future?

The reason the Excel spreadsheet is unuseable is because there can be enough confidence that the numbers input into the spreadsheet are valid.

[Tamas]

transcribing meetings are getting quite overshadowed.

Such a huge quality of life benefit! No longer someone having to devote their brain power to chronicling the meeting. Much more efficient to have AI do it and then review/edit before sending to participants.

Indeed.

On the article, while I agree that AI seems to be seriously overhyped, maybe the other side of that coin is that we overhype how our own brain works. We cannot possibly work on the basis of our cells putting the most likely words one after the other, we have souls from God!

[crazy canuck]

We have had technology to transcribe what people say long before AI was to clean in the current tech titans eyes.

If that is the best that can be done for a use case, use then it's not worth the billions of dollars of investment. 

[Josquius]

Yes. There's clearly a bubble based on marketing of calling this stuff AI rather than LMMs , this brings to mind true AI from sci fi.
But nor is it the completely useless toy some claim.
The reality is in the middle.
If it was called LMMs the investment would be considerably more sensible but it'd still be there and it'd be doing good business (and potentially losing tonnes too).

[DGuller]

Whether you're technical audience or not, the question that I posed at the end of my last reply IS by far the most important question.  I think this article fails both at usefully educating those who know very little, and also those with little knowledge that we all know is dangerous.

Do you care to elaborate? At first thought, I would think that your question would be dangerous for those who know little.

Sure.  Just for clarity, my question was:  "when does that mechanism of predicting the next word results in something that isn't too functionally different from real intelligence."

My question is the fundamental question you need answered for knowing when to use AI, when to use AI very carefully, or when not to use AI at all.  Of course it's a dangerous question to get the answer wrong for, that's what makes it an important question.

This is the question for which it helps understanding how LLMs work.  For example, they work really well in programming because programming is by design something from which you can infer patterns from without actually knowing for sure.  If I were to go into some new language and have to write a conditional statement, I wouldn't know for a fact that it would be an "if" statement, but that would be a pretty good guess.  Knowledge that can be guessed can be effectively compressed and synthesized.

For the same reason, the legal field is where LLMs can be very dangerous, at least general purpose LLMs.  To use an obsolete example, there was a case in 1973 called Roe vs. Wade that legalized abortion in many cases.  Would you be able to use your general knowledge to guess that one party of Roe, the other party was Wade, and that it was decided in 1973?  No, this is something that you just have to know.  A random happening here or there, and it could've been Smith v. Miller decided in 1975 that legalized the abortion.  All that means is that it's very dangerous to generalize about laws, and generalization is what intelligence is about.  Even an intelligent human being who's not educated about the law can be very dangerous if he doesn't understand the limitation of his intelligence when it comes to law and tries to reason out the legal questions based on general knowledge.

[Tamas]

We have had technology to transcribe what people say long before AI was to clean in the current tech titans eyes.

If that is the best that can be done for a use case, use then it's not worth the billions of dollars of investment. 

Transcribing isn't an "AI" feature but then summarising that transcript is.

[crazy canuck]

We have had technology to transcribe what people say long before AI was to clean in the current tech titans eyes.

If that is the best that can be done for a use case, use then it's not worth the billions of dollars of investment. 

Transcribing isn't an "AI" feature but then summarising that transcript is.

I have seen how AI summarizes complex meetings and it is pathetically, inadequate, and downright misleading. It might be OK for simple concepts like where certain tasks to be completed or discussed. Any intern consumer that in more time but less cost. So again I'm not really sure I see the use case.

[Josquius]

Whether you're technical audience or not, the question that I posed at the end of my last reply IS by far the most important question.  I think this article fails both at usefully educating those who know very little, and also those with little knowledge that we all know is dangerous.

Do you care to elaborate? At first thought, I would think that your question would be dangerous for those who know little.

Sure.  Just for clarity, my question was:  "when does that mechanism of predicting the next word results in something that isn't too functionally different from real intelligence."

My question is the fundamental question you need answered for knowing when to use AI, when to use AI very carefully, or when not to use AI at all.  Of course it's a dangerous question to get the answer wrong for, that's what makes it an important question.

This is the question for which it helps understanding how LLMs work.  For example, they work really well in programming because programming is by design something from which you can infer patterns from without actually knowing for sure.  If I were to go into some new language and have to write a conditional statement, I wouldn't know for a fact that it would be an "if" statement, but that would be a pretty good guess.  Knowledge that can be guessed can be effectively compressed and synthesized.

For the same reason, the legal field is where LLMs can be very dangerous, at least general purpose LLMs.  To use an obsolete example, there was a case in 1973 called Roe vs. Wade that legalized abortion in many cases.  Would you be able to use your general knowledge to guess that one party of Roe, the other party was Wade, and that it was decided in 1973?  No, this is something that you just have to know.  A random happening here or there, and it could've been Smith v. Miller decided in 1975 that legalized the abortion.  All that means is that it's very dangerous to generalize about laws, and generalization is what intelligence is about.  Even an intelligent human being who's not educated about the law can be very dangerous if he doesn't understand the limitation of his intelligence when it comes to law and tries to reason out the legal questions based on general knowledge.

The problem here is the AI can't just admit when its wrong.
With the Roe vs. Wade example the AI can probably give you a great writeup- because so much has been written about it, it has munched so much specific material, there's tonnes out there on the web. All sorts.
A bit of a problem maybe it grasps a far right nonsense take on it rather than a good faith one but still, it should do an OK job.

Ask it for something more obscure however then it might well go off telling you all about it, it being a complete fabrication.
IMO AIs really need to start far more strongly always including their sources and maybe sometimes some explanation too of how they got from their source to what they wrote. Maybe some grading of sources could be nice too. Random website written by god knows who vs. the actual court transcript.

We have had technology to transcribe what people say long before AI was to clean in the current tech titans eyes.

If that is the best that can be done for a use case, use then it's not worth the billions of dollars of investment. 

Transcribing isn't an "AI" feature but then summarising that transcript is.

I'd say more important is the quality.
The summarising is often fairly hit and miss, though it is improving. Still can't do sentiment though.
The quality of the transcription is really coming on though from the old rule based pattern matching models that required a lot of user training. Far fewer mistakes in modern setup and they are much better at handling real world discussions with multiple speakers.
They still can't handle some dialects alas. 

[crazy canuck]

An indication of the magnificent the investments being made. From the NYTimes

It's no secret by now, as investors await an earnings report on Wednesday by the chip behemoth Nvidia, that optimism around the windfall that artificial intelligence may generate is pumping up the stock market.

But in recent months, it has also become clear that A.I. spending is lifting the real economy, too.

It's not because of how companies are using the technology, at least not yet. Rather, the sheer amount of investment — in data centers, semiconductor factories and power supply — needed to build the computing power that A.I. demands is creating enough business activity to brighten readings on the entire domestic economy.

Companies will spend $375 billion globally in 2025 on A.I. infrastructure, the investment bank UBS estimates. That is projected to rise to $500 billion next year. Investment in software and computer equipment, not counting the data center buildings, accounted for a quarter of all economic growth this past quarter, data from the Commerce Department shows.

(Even that probably doesn't reflect the whole picture. Government data collectors have long had trouble capturing the economic value of semiconductors and computer equipment that large tech companies like Meta and Alphabet install for their own use, rather than farming out to contractors, so the total impact is likely to be higher.)

[frunk]

IMO AIs really need to start far more strongly always including their sources and maybe sometimes some explanation too of how they got from their source to what they wrote. Maybe some grading of sources could be nice too. Random website written by god knows who vs. the actual court transcript.

The way tokenization works it's difficult if not impossible to include attribution like that within the LLM.  Maybe a next generation LLM might be able to, and it can use regular search or other resources to augment the results.

[Sheilbh]

Lying feels like the wrong word as that feels like attributing human motivations to it.

It can provide inaccurate/false information. Of course, it generally is the case that predictions aren't 100% accurate. After all, when companies buy market research, it isn't because they are going to get a 100% accurate picture of what is currently happening and predictions for the future but that it is better than the knowledge they have if they only relied on their fieldforce, annecdotal customer feedback and sales data.

Yeah I agree it's anthropomorphising to frame as "lying". I also think it misses things that aren't there which can be important in some areas.

We are encouraged to experiment so I've put in some contracts which I've already reviewed and have notes on. A lot of it is good - the hallucination point has not really been an issue. But it doesn't flag that there isn't a cap on liability - and in English law if there isn't a cap on liability then it is uncapped which is really important. But I think this is where sector specific ones will emerge and there are definitely areas of law where I think it will have a big impact - particularly as it's already low value, high volume, highly automated work I can imagine it really taking off in the domestic conveyancing sector.

But also it depends on the tool. It very much depends what you want to do and I think at an economy/business sector wide level it will be the existing enterprise tech providers building in additional functionalities where the impact will be felt first and most strongly - whether that's GitHub or Salesforce or Oracle. I've found ChatGPT useful in my small business thing as I mentioned but I can't think of many uses for it.

The one I like most as a lawyer though is Notebook LM which I know journalists like a lot as well because it cites everything. It doesn't and shouldn't replace the actual work but it's really helpful if a very lengthy judgement or inquiry report in giving a general summary with citations linking to that section of the document you've uploaded. I've also used it with some of the decisions in other European courts because I don't have the languages - I caveat everything and say if we want to properly understand something we'd need to get a proper translation or speak to local counsel. But given that the current situation with a lot of European law is lawyers on LinkedIn sharing, say, Austrian or Polish or Belgian decisions that have been Google Translated this seems like a natural progression (not least because it's a Google product so you get the Google translated OCRed decision anyway - which can in itself be a bit of a hassle). Not something you can rely on but I think enough to be aware of actually there might be a specific risk/rocks in x jurisdiction we need to be mindful of - and if we are uncomfortable with risk in that area/jurisdiction we should speak to local counsel.

The way tokenization works it's difficult if not impossible to include attribution like that within the LLM.  Maybe a next generation LLM might be able to, and it can use regular search or other resources to augment the results.

I mentioned it before Notebook LM is very good at this but it's focused on the documents you upload.

I'd add this is also the big issue for news publishers (and why I think this can't just be "fair use") is there are generative search results which do cite articles and sources. But there is across markets significant click and pageview loss from generative search. I've mentioned before but my suspicion is that the agencies will do very well out of this as people don't normally go to Reuters or AFP anyway while the news publishers like the NYT, the Guardian etc will be hit - possibly quite badly and the impact of that on our democratic society is unclear. But I think the first wave of the internet destroyed the business model of huge chunks of media that have only just recovered spottily but some are gone forever like the local press. I think the effect of that on our society was not great - and I'm not sure generative search will be better.

[Josquius]

IMO AIs really need to start far more strongly always including their sources and maybe sometimes some explanation too of how they got from their source to what they wrote. Maybe some grading of sources could be nice too. Random website written by god knows who vs. the actual court transcript.

The way tokenization works it's difficult if not impossible to include attribution like that within the LLM.  Maybe a next generation LLM might be able to, and it can use regular search or other resources to augment the results.

Chat GPT does use regular search and provide sources- though says nothing about quality of sources and will often go and make something up unsourced.

[frunk]

Chat GPT does use regular search and provide sources- though says nothing about quality of sources and will often go and make something up unsourced.

That's using the LLM itself to generate sources, which as with the rest of the model isn't an authenticated attribution and shouldn't be used as such.

[Josquius]

Chat GPT does use regular search and provide sources- though says nothing about quality of sources and will often go and make something up unsourced.

That's using the LLM itself to generate sources, which as with the rest of the model isn't an authenticated attribution and shouldn't be used as such.

I'm not sure what you mean.
If it says something is a fact then links you off to a journal where you see it says precisely that then isn't that a relevant source?
Its like wikipedia. Anyone can edit it and a rubbish source in itself, but using it as a shortcut to valid sources can be OK.

[Jacob]

That's using the LLM itself to generate sources, which as with the rest of the model isn't an authenticated attribution and shouldn't be used as such.

I'm not sure what you mean.
If it says something is a fact then links you off to a journal where you see it says precisely that then isn't that a relevant source?
Its like wikipedia. Anyone can edit it and a rubbish source in itself, but using it as a shortcut to valid sources can be OK.

I think the issue is the italicized part, and the level of confidence that it "says precisely that".