The Tech Dystopia Thread

[Syt]

 

[Syt]

More seriously, John Oliver had a segment about data brokers last weekend:

[Maladict]

The AI angle is only part of the story, and didn't get all that much coverage here.
There are actually several issues that created this terrible mess, some dating back a decade before the system was introduced.

First off, lots of errors were made by people claiming benefits, which prompted the investigations as it seemed  mass fraud was taking place. The benefit system is very complicated and the child care providers often misinformed parents as well, sometimes for their own benefit. Although most of these errors were not made intentionally, the tax department held that they were entitled to reclaim all benefits even if a single, small error had been made. The disproportional clamping down on innocent mistakes, and then fighting every appeal tooth and nail is what made it such a scandal. That and the apparent discriminatory practices of selecting double nationalities and low income families as high risk factors for fraud. And finally the downright criminal behavior of trying to cover up the mistake and destroying the evidence.
While the AI situation is problematic those risk analyses were already done well before the system was introduced iirc. The problem is a rotten tax department with a terrible work culture that has been unable to do major reforms for a long time. It's ironic that most IT projects the tax department started were abandoned after spending millions, and the one that actually became operational helped create a disaster.

[Jacob]

Thanks for the additional context Maladict.

[The Brain]

Thanks. Makes more sense.

[Darth Wagtaros]

I think some areas of the US use predictive AI for crime, and harass people who the algorithm consider criminals.

[Sheilbh]

Reflects as well that what you put into the system (like a secret blacklist) is going to result in those characteristics becoming what it flags. Which I think is normally the issue with these stories.

What have the consequences been because a GDPR fine doesn't seem like enough (or high enough)? Plus I never know how much one bit of government fining another bit of government helps.

[Josquius]

Algorithms to flag risk factors, especially complex stuff where multiple unconnected factors combine intersectionality, should be a good thing.
Spot people to be investigated far more efficiently than through old school methods. Take away the menial sorting through heaps of data part of the job to give civil servants more time to actually do the human stuff of investigating specifics and dealing with people.

Instead it tends to just be taken as an excuse to cut costs. Don't move people away from menial work to more value added stuff. Just downsize them.
And this flag people who have factors behind them that warrant investigation? Guilty until proven innocent. Investigation is as good as guilty.

[Sheilbh]

I remember working with a company that had an anti-fraud AI product and there was so much work involved in making sure that we never accidentally process data about race or ethnicity or nationality or in any way infer that data (because, in European law, if you infer it you're processing it) - so I just find it mindblowing that the tax authority looked at their racist secret list of dual citizens, especially with "a non-Western appearance" and decided to add that

I've never read of a secret blacklist that wasn't really dodgy. It feels like if you have to keep what you're doing super-secret maybe you shouldn't be doing it.

[Maladict]

Reflects as well that what you put into the system (like a secret blacklist) is going to result in those characteristics becoming what it flags. Which I think is normally the issue with these stories.

What have the consequences been because a GDPR fine doesn't seem like enough (or high enough)? Plus I never know how much one bit of government fining another bit of government helps.

I believe they're still looking into criminal charges against a number of tax dept employees. And a new  proportionality principle was introduced, which can be used to appeal against disproportionate fines.

[Josephus]

I'm becoming more and more luddite each passing day.

[Jacob]

I'm becoming more and more luddite each passing day.

Natural result of getting old while new tech is developing

[PJL]

I'm becoming more and more luddite each passing day.

Natural result of getting old while new tech is developing

I think I got to that point in 2010 with mobile phones. To be honest, everything since then tech-wise I seem to be lagging behind.

[Josephus]

I'm becoming more and more luddite each passing day.

Natural result of getting old while new tech is developing

Yup. 

[crazy canuck]

I'm becoming more and more luddite each passing day.

Natural result of getting old while new tech is developing

Yup. 

If it makes you feel any better my older son just mentioned to me that he feels like he starting to get out of touch with technology. Actually that might not make you feel any better.