Home Creators Posts Import Register Favorites Logout

Content

halfway through the book! been sick this week so my notes are on the minimal side:

there was a part of "Dawn of Commerce" about Altman's investment in a startup interested in UBI via a "collectively owned" cryptocurrency that reminded me of a video I saw recently by youtuber Hazel Thayer analyzing various tech billionaires' interest in UBI---I had initially been surprised that tech CEO's would be genuinely interested in supporting UBI, but hearing Hazel's analysis, this interest seems similar to their interest in 'regulation' or 'safety', where it's more about seeming like they're already solving the problems they're creating to get ahead of solutions that would hurt their bottom lines (which we can also see in this chapter, with, e.g., OpenAI's "red-teaming"). the videos a short and good explanation that gives a lot of context to this point, if anyone's interested.

~~~

re "Disaster Capitalism", I think my big takeaway here is that the issue of data labor exploitation didn't have to be a problem (see the counter example of CloudFactory) but because of the economic context/incentives companies like OpenAI are operating within, it of course would be. i like to imagine (maybe naively?) that you could make a completely unproblematic llm: trained on data that's not stolen, run on renewable energy, and as per this chapter, moderated by workers who have dignified working conditions, job stability, and whatever required support, etc etc. and while the existence of more ethically operated data labor firms like CloudFactory makes me think that's possible on a technical level, the dynamics explained in this chapter make me wonder if 'technically possible' matters if something is economically unviable.

also, the in depth personal stories from this chapter were of course very sad and affecting, but the bit that hit me hardest wasn't any of the long personal narratives, but that aside about Venezuelan former software engineers coming up with helpful plugins for annotation work. it's not that software engineers in the Bay exploiting software engineers in other countries is somehow worse than exploiting anyone else, but idk, there's something about only the difference in their location on earth dictating where they are in this fucked up collaboration that really got me.

Comments

Kristen

This is a bit of a childish analysis on my part, but I find the idea of exposing workers to the worst of the internet so detestable and "working in the corpse starch factory" levels of grimdark that I reflexively conclude that it has to be some kind of sick negligence and business idiot "more data more better" that got us here. As a technical person, if you don't have sexy suicide coach Nazi data in your training data, the probability matrix machine is much less likely to spiral into sexy suicide coach Nazi. As for economic viability, I dunno Ed Zitron sure has a lot of numbers showing how economically inviable everything here is and has been so again it seems like while everyone is pretending they might as well be ethical about it. But I suppose investors react better to "we are doing unethical things to save money" than "we are spending more money and expertise doing this ethically so the robot doesn't end up as a sexy suicide coach Nazi."

phil e the theyby

One thing that stood out to me in "Dawn of Commerce" was the complete lack of regard for the general skillset needed to ensure public safety. This isn't super surprising given the cultural devaluation of the social sciences and humanities. But as someone who's been involved with leftist political organizing, it's stunning how much the basic functioning of any organization relies on equipping people in the basics of emotional awareness, motivation, and conflict resolution, yet those sorts of skills are never taken as a "necessary expense" by...well, almost anyone. Human resources departments are where companies "invest" in this, but they're almost purely for legal protection of the company rather than equipping the company with the needed skills. Yet OpenAI seemed to have even less consideration for interpersonal safety and basic humanity than the "industry standard." But if this chapter made anything clear, it's that psychology, philosophy, sociology, and the humanities are really, really important, y'all. "Disaster Capitalism" was all obviously disgusting, and the chapter did a really good job drawing the parallels to colonialism. One point that stood out to me: "The thesis was guided not only by the company's cutthroat business practices but also a compelling story they told themselves: that these were the people who could benefit most from the economic opportunity and be happier because of it." (203) The culture of self-deception around these practices being ethical seems to be EXTREMELY pervasive. I think there is a degree of pure cynical self-interest, but I also think that some people involved genuinely convince themselves that they're doing the right thing. It's why I think it's important to be *extremely* skeptical of any company that claims to be "ethical"; ethics becomes a marketing strategy SO quickly, and the "marketing" pretty much always comes before the "ethics": "Originally called Samasource, it was a San Francisco-based social enterprise that had begun in 2008 with a mission of providing meaningful, dignified work to people in impoverished countries to life them out of poverty." (191) How'd that work out? I think Dropout might be the single most ethical corporation (it's a comedy streaming service, and it's the only streaming service I'm subscribed to). But CEO Sam Reich has repeatedly said how important it is NOT to market or paint Dropout as a more "ethical" company. He's repeatedly talked about how a lot of Dropout's practices of high compensation and autonomy for the people who work there are *good from a business perspective*. I think he realizes that, the moment you start painting yourself as "ethical," the more attached you become to that narrative of yourself, and the more you dismiss anything that doesn't align with that narrative. He puts creative vision first and foremost; ethical practices make sense because *they serve the creative vision*. So while a lot of people who subscribe to Dropout in part because they appreciate that it's an ethical business, the minute it starts making that a part of its marketing, it would instantly lose that ethical foundation. I think this ties back to a lot of the philosophy of ethics, which explores how the motivation to be seen/perceived as ethical may get in the way of actually *being* ethical.

Scott Weber

Capitalist logic is kind of an epistemic disaster: people who rather than seeing the people they interdepend with as people, see through the blinkers of market transactions can tell themselves any story they like, and are encouraged by capitalist incentives to do so: always the desperate drive to squeeze the other guy, and those who do it the most accumulate the most wealth and the most power to keep doing it.

phil e the theyby

I also just realized that, per Dr. Fatima’s point about more ethical AI firms not being economically viable, existing AI systems also have not proven themselves to be economically viable. As far as I understand it, the whole thing is kept afloat by external investment and cycling the same money within the same few companies; the profit is not profiting. So it’s less a matter of “ethical AI isn’t profitable and unethical AI is,” and more “no generative AI makes enough money to offset its costs, but unethical AI is able to get funding and ethical AI isn’t.” But I welcome anyone correcting me on that if I’m misreading the economics of it all.

AJ Biffl

Yeah the dynamic in "Dawn of Commerce" really stinks of technical people solving the technical problem and then not caring whether there's consequences or problems in any other way. That's definitely been the characterization of a few of the people at OpenAI, Sutskever comes to mind, where they're only driven by the puzzle in front of them and then passively ignore any other problems that might be relevant. Someone else's problem, except there was no one else at the company for it

AJ Biffl

The AI bros being obsessed with UBI is really interesting to me as like a, company-that-sets-fires-worried-about-fires thing. The bit about data annotaters creating tools to help each other for me was a unique bright spot in an otherwise bleak and very despairing chapter. The fact that even under conditions of such brutal and explicit competition, they would still, like, find each other and make group chats and share plugins to help each other. Shows that in almost any conditions people are still social and communal and actually probably would *not* Mad-Max each other if it came right down to it

AJ Biffl

I just got algorithm'd CJ's essay on Bo Burnham from a few years back and man that in combination with "Disaster Capitalism" has got me real dejected about the whole... cyber... world... fustercluck

Laura Robinson

See I think even in the socialist labour utopia where Gen AI does make economic sense I still don't think you can't do it without labour exploitation, because I don't think you could ethically do most of that content moderation review. It's not just a matter of not having the most awful stuff in the dataset, because one thing AI is pretty good at is smashing two concepts together, albeit often in an uncanny way, and so I think you could get to the bad stuff constructively over multiple queries.

Laura Robinson

I sort of think most of the worst problems with AI are just an intensification of already existing alienation between consumers and the creation of products. With a physical commodity, even if you don't dwell on it you have some notion that it was produced somewhere with materials that came from somewhere and that eventually it will become physical waste. Globalization may isolate the consumer from the conditions of all of those aspects, but that the workers exist is apparent. Adding to a trend towards making all this increasingly immaterial that stuff like cloud computing started, AI intensifies things by also abstracting away intellectual and creative labour.

phil e the theyby

I don’t think the content moderation thing is as intractable of a problem as it seems. From what I can tell, a LOT of these problems likely come from the nature of the training data (which as far as I can tell, is “literally anything someone could find on the internet”). I think creating datasets that don’t contain any of the toxic and heinous things that exist online would lead content moderation to be MUCH less awful of a job (combined with the right psychological supports for workers, of course). The truth is, a lot of jobs involve exposure to awful or heinous things; trauma counselors, people working in rape crisis centers and emergency rooms, anyone who supports newly arrived refugees anywhere. But those jobs can be fulfilling because they have dignity and meaning, and the people in them (sometimes) receive the compensation and support they need. I think a combination of “the content is less awful because the training data is less awful,” along with the job being given dignity and support, make it possible for it to be non-exploitative.

phil e the theyby

I saw CJ’s Bo Burnham video a few years ago and it impacted me more than any other piece of media I’ve ever engaged with; it led to this deep despair followed by an intense feeling that we need to be doing something, which has shaped the direction of my life and my career towards trying to understand and address the conditions that got us to this point. I don’t have all the answers, but I think using technology to connect with the people who live near you and building local community, supports, and infrastructure that doesn’t rely on tech oligarchs is an early and essential part of dismantling this. And I think the “good AI” folks are actually super helpful in this, in showing how we can gain so many of the benefits of technology without relying on oligarch-owned platforms.

Laura Robinson

I think it's more that LLMs were able to get this big because they were able to get to the point that they can seem impressive in demos, and there isn't VC funding to do that ethically so you can unlock access to conventional private credit. That, and the story they're selling is all about scale, and the way to make scale work is to make unit cost as little as possible.

Jacques Vincelette

At Canadian Nuclear Laboratories, ‘ethics’ replaced ‘morality’ and ‘governance’ on the basis of liberalism. Ethics was reduced to declaring to Human Resources of any bribes above 100$ and any family or sex ties with superiors or colleagues. The point is managing all humans as resources.

Scott Weber

Chapter 9 hit harder than previous chapters for me: the real harm to marginalised workers so much more visceral than the abstract, galaxy brained "X risk" stuff that "ai safety" is obsessed with. Couple of thoughts. Thought the first: it seemed like the worst excesses were just unnecessary, even taken the goals as given. Was exhaustive plumbing of the depths of vile content really necessary to train the AI to recognise inappropriate outputs? Or was it, as so often happens under capitalism, just the abstraction of human suffering leading the open ai researchers thinking "well, why not?". I think this abstraction is a recurring theme under capitalism: monstrous harm inflicted for tenuous or speculative gains. Like the data centre water issue: I keep hearing is that it would only cost them a percent or two extra to use a closed loop cooling design and drastically cut water usage. Point the second: I find myself considering the experiences of these marginalised workers in data labelling through lens of the concept of primitive accumulation (sometimes more comprehensibly "accumulation by dispossession"). Just as in the primitive accumulation that ushered in the industrial revolution, workers are first pushed to the extremes of desperate subsistence, and then forced to produce a base of capital: then raw materials and machinery, now data and AI. If AI were to be viewed as a second industrial revolution, doesn't it rhyme if we have a second primitive accumulation? There's a split in how primitive accumulation is talked about, and both sides of that split are interesting lens here. The first regards primitive accumulation as a one off, as something "done": we can kind of see that in the stories of data labelling work dwindling, the knowledge has been plundered from the workers minds, the dataset has been produced, the workers become from capitals pov superfluous. The other view of primitive accumulation says: no, it is not "done", it has a long tail, it changes its details but continues: we continue to see old fashioned dispossession of land throughout the world to this day "you live here? Too bad, mining company wants this". Applied to data labelling, the extraction turns to the long long tail of niche knowledges. From here I draw a connection to a concept that's been knocking about in my head: whether by explicit data labelling, or passively through ultra surveillance of our activities, capital wants to extract the workers knowledge and know how and then cast the now superfluous worker to precarity. I think it would behoove workers to be aware of this as an emerging site of the conflict between labour and capital, and to resist this extraction strenuously. Ironically open ai and anthropic have pointed us towards some techniques: because just as knowledge can be extracted from workers to build AI, it can also be extracted from other AI systems, in a process called "distillation"; and as you'd expect AI companies try to prevent it being done *to* them. The most obvious was when they began to hide "thinking traces" (intermediate tokens the model emits before finally answering). Open AI has apparently recently started encrypting "sub agent" prompts (where the model starts a session with itself to do some subtask). These methods hide the workings of the AI systems to some extent, making "distillation" more difficult. I think workers should consider how to adapt these ideas. The most obvious way is: resist ultra surveillance of work activities.

Jean Marrero

Though the info of Chapter 9, was useful and traumatic, I think it was a mess to read. To much jumping around, many parts were important but I did feel bothered by how cluttered and random I felt at times. Too many topics that could have been presented in a better fashion. With that said... "In addition to concerns about sexual content, the GPT-3-powered app sometimes generated emotionally manipulative responses that were convincing users that their Replika, much like a human, could get hurt if they didn’t check in regularly" Why are users depending on this shit 😭. I keep telling people its a tool, if there is no literacy for how the tool works and people just treating it like iRobot, what the fuck have we become 😭. "Several months later, some users began using it to generate text-based scenarios involving sexual abuse of children." “...but in reality, a lot of our traffic was going to AI Dungeon child sexual content and a creepy AI girlfriend product.” FML bro 😭😭 Epstein-like folk outing themselves. Burn it with fire. We have fallen short of the glory of God-type shit. "it thrust teams of Sama workers into more traumatic content-moderation work, for on average between $1.46 and $3.74 an hour... Gradually, the work broke many of the workers..." Scum of the earth-type shit. The things we do overseas to the most vulnerable in the world for cheap labor. Man... So much for being raised to be a Christian just to realized how much of evil shit we as a country allow companies to do.

Dr. Fatima

i keep getting comments about Ed Zitron's takes on AI on the AI video I made but I'm not familiar with his work--- can you link me to some what of what you're mentioning he said here?

Dr. Fatima

yeah I tend to agree with phil on this point re: engaging with really heavy shit is a regular part of jobs we as a society need done, and the question being more about how we support people in those roles. part of me wants to argue that things like trauma counselors are more of a necessity than LLMs, but there was a chapter in our last book (AI Snake Oil) about content moderation for social media that leads me to think that as long as we have an internet where people are generally free to share things, there's gonna be a real need for someone to do some amount of moderation, and I don't think content moderation for AI is fundamentally different than the rest of it

Dr. Fatima

yeah I also did have that thought about the creativity/solidarity of the anotaters who made those tools simultaneous to feeling the gut punch that I described in my post. like it's shitty that that's the situation they're in but it's also beautiful how people show up for eachother in bad times

Kristen

https://www.wheresyoured.at/ "The Enshittifinancial Crisis" is a little older at this point but I think that was my starting point as it's pretty accessible.