The AI dooooooom thread

[The Minsky Moment]

It does that if backed into a corner and it can't find a real answer, but it does tend to prefer real answers first.

It doesn't have preferences and doesn't know what reality is. It's just a "stochastic parrot" as per the Bender paper.

For legal purposes I've used AI as a glorified, flexible database tool.  I.e. I can just dump a ton of PDF deposition and hearing transcripts into a walled AI and it can answer questions like: Show me every time someone testifies about Trademark X, or Did any witness testify that they were aware of Fact Y before the contract was executed on Date Z?  It does a pretty good job of that and it's every easy to set up - just drap and drop PDFs, wait a few minutes and start asking away.

Some people also use it to draft relatively simple letters and swear by it, but drafting simple letters is ... simple; I don't need an automated tool for that.  Anything more complex and its more trouble to proof for use.

I would never rely on it to analyze a serious legal issue.  But the worst stakes there is someone gets lousy legal representation.  As applied to something like the FDA, millions of lives are at stake.  It is grotesque irresponsibility.

[Josquius]

Obviously it doesn't have feelings and actual preferences. Why do people keep thinking it's a person?
 I'm describing the way it works. It's coded to prioritise actual facts. Though when none can be found or it gets deep into things....

[HVC]

It's just a "stochastic parrot" as per the Bender paper.

Kill all humans?

[DGuller]

Obviously it doesn't have feelings and actual preferences. Why do people keep thinking it's a person?
 I'm describing the way it works. It's coded to prioritise actual facts. Though when none can be found or it gets deep into things....

The issue is that your description of how it works is wrong.  It doesn't know for a fact that anything it generates is a fact, so it can't prioritize that.  It would be very easy to eliminate hallucination if LLM did know what a fact is.

There is also nothing to find, at least until you connect it to Internet.  There is no database to search, all knowledge is encoded as weights in neural network.

[Josquius]

Obviously it doesn't have feelings and actual preferences. Why do people keep thinking it's a person?
 I'm describing the way it works. It's coded to prioritise actual facts. Though when none can be found or it gets deep into things....

The issue is that your description of how it works is wrong.  It doesn't know for a fact that anything it generates is a fact, so it can't prioritize that.  It would be very easy to eliminate hallucination if LLM did know what a fact is.

There is also nothing to find, at least until you connect it to Internet.  There is no database to search, all knowledge is encoded as weights in neural network.

Most tools I've used were connected to the Internet or some document or other. Can't say I've ever used one that was just free floating.

Chat gpt, Claude, etc... Absolutely can tell you a sourced fact.
If something can easily be found through searching they will do this.
For more complex things... I do find they usually say there's nothing found for it. Though they will sometimes take a best guess which is often not great.

Obviously it's not a real thinking person. It doesn't *know* something is a fact or a hallucination. And this is where the problem lies. But it will aim to get things right via a direct source or the simple logic that correct things get repeated more often.

[DGuller]

ChatGPT is free-floating most of the time.  Even if it can search, it will rarely do so unless you ask it to search.  Just like when I'm writing here, most of the time I will go from my working knowledge and memory, but occasionally if I feel like I need to for some reason, I will google.

[Tamas]

So when it says searching it is not actually searching?

[DGuller]

So when it says searching it is not actually searching?

No, if it says searching, it's searching, but in general conversations it will rarely search unless you ask it to.

[Sheilbh]

Yes - but worth noting that for Google and Microsoft the data they use for scraping for AI is the same as the data they use for indexing the web. So in those cases it's sort of upstream of searching. This has been a tension and one of the big pushes of IP owners/creators to decouple those two - and we're getting there.

Different but slightly similar are the licensing deals the agencies are doing for example with Open AI (and I think news agencies will do very well out of AI). I think those are typically through regularly refreshed APIs (which a publisher would need to make sure corrections, retractions, take downs etc get pulled through). So not quite "searching" because it's basically a wire but again slightly upstream of what appears on search.

[celedhring]

My use case for ChatGPT is polishing my own writing, particularly in English. The suggestions it makes do tend to be  wordy and redundant but a lot of times it will also make good suggestions. So essentially I will dump one of my texts, ask it to edit for stuff like clarity, flow, engagement... and check what it spits out and pick up what I like. That I do find useful.

When doing more creative work is of little use, since it tends to make very bland suggestions. It is also very explicit - it just can't handle subtext or nuance (which is to be expected). Like something I have tried is to feed it one of my scenes, and ask it to add this or that undercurrent to the scene, and it will have one of the characters just say it openly.

Its screenwriting "voice" is also, once more, very generic. I feel I can read all the bad Hollywood scripts it has been fed.

[celedhring]

Also, several people I work with are using it to draft emails - I can tell because of the language - which I find supercringe and slightly rude. Like, "huh, I'm not worth your time so you have chatgpt draft your emails to me?"

[crazy canuck]

Yeah, as I mentioned earlier in this thread, it is becoming very obvious what lawyers are using generative AI tools to do their drafting. It is complete shite.

[Josquius]

It's weird though, as some things people flag as definite signs of AI but they're actually what would have been good English.
—  is a big one. I've never used em dashes as its hard to find on the keyboard, I always used hyphens for the same purpose. Though a few people who write for a living I know are annoyed about doing things right now marking them as AI.
Incidentally I've no idea how ai picked up on that one to begin with considering I don't think I'm alone in just using hyphens.

Beyond this in some contexts you get some words which were the most professional to use but are now kind of cringe markers.

[celedhring]

I use a lot of dashes in regular writing - mostly because they are very prevalent in script-speak and it has kinda bled into my regular writing. Now I feel like people think AI is inserting them into my writing 

EDIT: Fuck, I used one, see?

[garbon]

I use a lot of dashes in regular writing - mostly because they are very prevalent in script-speak and it has kinda bled into my regular writing. Now I feel like people think AI is inserting them into my writing 

EDIT: Fuck, I used one, see?

I am always fighting with chatGPT to remove them.