![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
The chatbots are not your friends:
April 2023, video: "The Rise and Fall of Replika: A cautionary tale about love, heartbreak, and our AI overlords." I first heard about this through the framing of "making fun of incels for being mad that their bot waifus broke up with them," but this video digs through how predatory and exploitative the company was, and how badly its customers (even if some of them were creepy) got shafted.
You may also remember Replika from the October 2023 story about the man who tried to assassinate the Queen of England, and had extensive chat logs with a Replika bot expressing "her" encouragement and support.
A different service, Character.AI, faces a just-filed lawsuit over the suicide of a 14-year-old boy: "In previous conversations, the chatbot asked Setzer whether he had “been actually considering suicide” and whether he “had a plan” for it, according to the lawsuit. When the boy responded that he did not know whether it would work, the chatbot wrote, “Don’t talk that way. That’s not a good reason not to go through with it,” the lawsuit claims."
Other bad "AI" news, up to and including more deaths:
May 2023 study: Radiologists are more likely to misread mammograms if a bot also reads them and "calculates" the wrong diagnosis. Inexperienced radiologists were affected by it most, but even the moderately- and very-experienced ones were prone to being swayed by false results if "an AI" produced them.
Australian child-abuse case worker caught "using ChatGPT to draft a protection application report in December 2023 — sending a pile of stupendously sensitive information off to OpenAI in the process. The report contained inaccuracies and weirdly unprofessional phrasing — which was how it was spotted as LLM output — and downplayed the risks to the child."
This January: "Parcel delivery firm DPD have replaced their customer service chat with an AI robot thing. It’s utterly useless at answering any queries, and when asked, it happily produced a poem about how terrible they are as a company. It also swore at me."
April: "All six [Israeli intelligence officers] said that Lavender [an "AI" targeting system] had played a central role in the war, processing masses of data to rapidly identify potential “junior” operatives to target. Four of the sources said that, at one stage early in the war, Lavender listed as many as 37,000 Palestinian men who had been linked by the AI system to Hamas or PIJ." This system was approved when the IDF concluded it had a "90% accuracy rate" -- so even if we fully accept that at face value, that means they're happy to use a bot that flagged 3,700 innocent people, and likely everyone they lived with, to be bombed to death.
September: "The Washington Post worked with researchers at the University of California, Riverside to understand how much water and power OpenAI’s ChatGPT, using the GPT-4 language model released in March 2023, consumes to write the average 100-word email." (It's bad.)
"Many developers say AI coding assistants make them more productive, but a recent study set forth to measure their output and found no significant gains. Use of GitHub Copilot also introduced 41% more bugs." (Towards the end of the article, a CEO is quoted claiming his company has doubled their output using chatbots, but there's no explanation of how he got that number.)
"I asked "Which Oscar winners have appeared in episodes of Doctor Who?" Here are the results." Spoiler alert: it correctly ID's several white actors that fit the criteria, but adds several white people who don't...and leaves out a couple of black people who do.
October: "we discovered only a few days before the wedding that our officiant was not legally qualified to marry us because she had followed the incorrect, chatgpt’ed instructions that our planner sent."
April 2023, video: "The Rise and Fall of Replika: A cautionary tale about love, heartbreak, and our AI overlords." I first heard about this through the framing of "making fun of incels for being mad that their bot waifus broke up with them," but this video digs through how predatory and exploitative the company was, and how badly its customers (even if some of them were creepy) got shafted.
You may also remember Replika from the October 2023 story about the man who tried to assassinate the Queen of England, and had extensive chat logs with a Replika bot expressing "her" encouragement and support.
A different service, Character.AI, faces a just-filed lawsuit over the suicide of a 14-year-old boy: "In previous conversations, the chatbot asked Setzer whether he had “been actually considering suicide” and whether he “had a plan” for it, according to the lawsuit. When the boy responded that he did not know whether it would work, the chatbot wrote, “Don’t talk that way. That’s not a good reason not to go through with it,” the lawsuit claims."
Other bad "AI" news, up to and including more deaths:
May 2023 study: Radiologists are more likely to misread mammograms if a bot also reads them and "calculates" the wrong diagnosis. Inexperienced radiologists were affected by it most, but even the moderately- and very-experienced ones were prone to being swayed by false results if "an AI" produced them.
Australian child-abuse case worker caught "using ChatGPT to draft a protection application report in December 2023 — sending a pile of stupendously sensitive information off to OpenAI in the process. The report contained inaccuracies and weirdly unprofessional phrasing — which was how it was spotted as LLM output — and downplayed the risks to the child."
This January: "Parcel delivery firm DPD have replaced their customer service chat with an AI robot thing. It’s utterly useless at answering any queries, and when asked, it happily produced a poem about how terrible they are as a company. It also swore at me."
April: "All six [Israeli intelligence officers] said that Lavender [an "AI" targeting system] had played a central role in the war, processing masses of data to rapidly identify potential “junior” operatives to target. Four of the sources said that, at one stage early in the war, Lavender listed as many as 37,000 Palestinian men who had been linked by the AI system to Hamas or PIJ." This system was approved when the IDF concluded it had a "90% accuracy rate" -- so even if we fully accept that at face value, that means they're happy to use a bot that flagged 3,700 innocent people, and likely everyone they lived with, to be bombed to death.
September: "The Washington Post worked with researchers at the University of California, Riverside to understand how much water and power OpenAI’s ChatGPT, using the GPT-4 language model released in March 2023, consumes to write the average 100-word email." (It's bad.)
"Many developers say AI coding assistants make them more productive, but a recent study set forth to measure their output and found no significant gains. Use of GitHub Copilot also introduced 41% more bugs." (Towards the end of the article, a CEO is quoted claiming his company has doubled their output using chatbots, but there's no explanation of how he got that number.)
"I asked "Which Oscar winners have appeared in episodes of Doctor Who?" Here are the results." Spoiler alert: it correctly ID's several white actors that fit the criteria, but adds several white people who don't...and leaves out a couple of black people who do.
October: "we discovered only a few days before the wedding that our officiant was not legally qualified to marry us because she had followed the incorrect, chatgpt’ed instructions that our planner sent."
![[community profile]](https://www.dreamwidth.org/img/silk/identity/community.png){kind=small}
