News:

And we're back!

Main Menu

The Tech Dystopia Thread

Started by Sheilbh, April 13, 2022, 04:58:52 PM

Previous topic - Next topic

Syt

I am, somehow, less interested in the weight and convolutions of Einstein's brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops.
—Stephen Jay Gould

Proud owner of 42 Zoupa Points.

Syt

More seriously, John Oliver had a segment about data brokers last weekend:

I am, somehow, less interested in the weight and convolutions of Einstein's brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops.
—Stephen Jay Gould

Proud owner of 42 Zoupa Points.

Maladict

#17
The AI angle is only part of the story, and didn't get all that much coverage here.
There are actually several issues that created this terrible mess, some dating back a decade before the system was introduced.

First off, lots of errors were made by people claiming benefits, which prompted the investigations as it seemed  mass fraud was taking place. The benefit system is very complicated and the child care providers often misinformed parents as well, sometimes for their own benefit. Although most of these errors were not made intentionally, the tax department held that they were entitled to reclaim all benefits even if a single, small error had been made. The disproportional clamping down on innocent mistakes, and then fighting every appeal tooth and nail is what made it such a scandal. That and the apparent discriminatory practices of selecting double nationalities and low income families as high risk factors for fraud. And finally the downright criminal behavior of trying to cover up the mistake and destroying the evidence.
While the AI situation is problematic those risk analyses were already done well before the system was introduced iirc. The problem is a rotten tax department with a terrible work culture that has been unable to do major reforms for a long time. It's ironic that most IT projects the tax department started were abandoned after spending millions, and the one that actually became operational helped create a disaster.

Jacob

Thanks for the additional context Maladict.

The Brain

Women want me. Men want to be with me.

Darth Wagtaros

I think some areas of the US use predictive AI for crime, and harass people who the algorithm consider criminals.
PDH!

Sheilbh

Reflects as well that what you put into the system (like a secret blacklist) is going to result in those characteristics becoming what it flags. Which I think is normally the issue with these stories.

What have the consequences been because a GDPR fine doesn't seem like enough (or high enough)? Plus I never know how much one bit of government fining another bit of government helps.
Let's bomb Russia!

Josquius

Algorithms to flag risk factors, especially complex stuff where multiple unconnected factors combine intersectionality, should be a good thing.
Spot people to be investigated far more efficiently than through old school methods. Take away the menial sorting through heaps of data part of the job to give civil servants more time to actually do the human stuff of investigating specifics and dealing with people.

Instead it tends to just be taken as an excuse to cut costs. Don't move people away from menial work to more value added stuff. Just downsize them.
And this flag people who have factors behind them that warrant investigation? Guilty until proven innocent. Investigation is as good as guilty.
██████
██████
██████

Sheilbh

I remember working with a company that had an anti-fraud AI product and there was so much work involved in making sure that we never accidentally process data about race or ethnicity or nationality or in any way infer that data (because, in European law, if you infer it you're processing it) - so I just find it mindblowing that the tax authority looked at their racist secret list of dual citizens, especially with "a non-Western appearance" and decided to add that :blink:

I've never read of a secret blacklist that wasn't really dodgy. It feels like if you have to keep what you're doing super-secret maybe you shouldn't be doing it.
Let's bomb Russia!

Maladict

Quote from: Sheilbh on April 14, 2022, 12:57:02 PMReflects as well that what you put into the system (like a secret blacklist) is going to result in those characteristics becoming what it flags. Which I think is normally the issue with these stories.

What have the consequences been because a GDPR fine doesn't seem like enough (or high enough)? Plus I never know how much one bit of government fining another bit of government helps.

I believe they're still looking into criminal charges against a number of tax dept employees. And a new  proportionality principle was introduced, which can be used to appeal against disproportionate fines.


Josephus

I'm becoming more and more luddite each passing day.
Civis Romanus Sum

"My friends, love is better than anger. Hope is better than fear. Optimism is better than despair. So let us be loving, hopeful and optimistic. And we'll change the world." Jack Layton 1950-2011

Jacob

Quote from: Josephus on April 14, 2022, 04:00:53 PMI'm becoming more and more luddite each passing day.

Natural result of getting old while new tech is developing :hug:

PJL

Quote from: Jacob on April 14, 2022, 04:04:45 PM
Quote from: Josephus on April 14, 2022, 04:00:53 PMI'm becoming more and more luddite each passing day.

Natural result of getting old while new tech is developing :hug:

I think I got to that point in 2010 with mobile phones. To be honest, everything since then tech-wise I seem to be lagging behind.

Josephus

Quote from: Jacob on April 14, 2022, 04:04:45 PM
Quote from: Josephus on April 14, 2022, 04:00:53 PMI'm becoming more and more luddite each passing day.

Natural result of getting old while new tech is developing :hug:

Yup.  :(
Civis Romanus Sum

"My friends, love is better than anger. Hope is better than fear. Optimism is better than despair. So let us be loving, hopeful and optimistic. And we'll change the world." Jack Layton 1950-2011

crazy canuck

Quote from: Josephus on April 15, 2022, 06:08:45 AM
Quote from: Jacob on April 14, 2022, 04:04:45 PM
Quote from: Josephus on April 14, 2022, 04:00:53 PMI'm becoming more and more luddite each passing day.

Natural result of getting old while new tech is developing :hug:

Yup.  :(

If it makes you feel any better my older son just mentioned to me that he feels like he starting to get out of touch with technology. Actually that might not make you feel any better.