Home Creators Posts Import Register Favorites Logout

Content

Besides the story of the law professor who was falsely accused of sexually harassing a student by a chatGPT-generated news articles (which was wild, I hadn’t imagined personal level libel like that being a possible outcome of LLM hallucinations), I’d already learned most of the information here on gen AI while researching my video. I’m also pretty familiar with most of the technical explanations from when I had to learn machine learning for astrophysics stuff, so not the most exciting chapter for me. BUT something I did find interesting was the evolution of the culture around AI development. These passages stood out especially:

The data annotators on Amazon Mechanical Turk were typically paid low wages… Further, ImageNet developers did not compensate the photographers who created the images...ImageNet operated on a small budget and might not have existed if not for this way of gathering and annotating data. But even today, commercial projects by trillion-dollar companies operate the same way.

ImageNet continued the culture of open knowledge sharing in AI research. It was a great demonstration of how quickly progress could happen if researchers built on each other's work. Even though many of the contestants worked for companies who were in competition with each other, the norm of openness held sway…Today, this culture has changed to some extent as companies prioritize profits.

I guess I shouldn’t be surprised that the AI industry retained the early norm that saved money but is almost inevitably exploitative at scale, and abandoned the norm that was collectively beneficial but a liability in economic competition. At first that makes me feel kinda hopeless, like of course the industry will develop in the most capitalistic way possible as the technology becomes more economically viable and useful to whatever kinds of power, and it’ll probably only get worse. BUT, one of the silver linings of all the mostly depressing research I did was learning about the little bits of resistance to these trends. E.g., I mentioned about Turkopticon.net in the video, which is at least challenging the annotating data part of the first trend. And I didn’t talk about this in the video, but there are some companies/orgs that are really focused on maintaining the open-source ethos of the earlier industry, like Hugging Face (which is where I got a lot of the data on open source models energy use in the absence of data from the big AI companies, and is also the place Jordan recommended for people who want to locally host an LLM). Basically, the overall trend is upsetting, but the ways in which some people in/around the industry are pushing back on it is somewhat hope-inducing for me. I guess that’s one of the other advantages about learning about AI, just psychologically—yeah there’s a lot of doom, but there’s also all the ways people are trying to respond to it, and it’s nice to know that instead of just imagining the industry as this ambiguous blob of uniform evilness.

Completely unrelatedly, I was shook by this passage:

We think fears about vote manipulation are vastly overblown. Evidence suggests that people are naturally skeptical of what they see online and are resistant to persuasion. If anything, the problem is the opposite.” The fact that just about anything online may have been AI-generated means that the internet will be seen as even less trustworthy.

…really? I don’t have a citation or anything (they didn’t either i guess) but this feels like the opposite of true. I mean, I agree that the internet will be seen as less trustworthy, but I don’t know if that translates to misinformation not being effective. This feels kinda like their statement from chapter 1(?) about AI not really making education any worse/harder—ie, weirdly confident for how completely against my intuition it seems to be. Idk, what did yall think about this bit?

On a forward looking note, I’m really looking forward to reading the chapter on collective action and advocating for regulation they teased a few times in this chapter—I had my own take on it from the video ofc but I imagine the authors will have more of an inside perspective that I’m very curious to hear.

Comments

Jacques Vincelette

p.127 "Chatbots may not understand a topic as deeply or in the same way as a person—especially an expert—might, but they might still understand it to some useful degree. (...) Chatbots “understand” in the sense that they build internal representations of the world through their training process. " Vintage film cameras build their internal model of the world with photosensitive silver salts. We insist that the model holds understanding, yet meaning is deciphered by interpretation. Let's not confuse quicksilver with the broad world behind the shutter. "Ceci n'est pas une pipe." [René Magritte]

Kristen

Ambiguous Blob of Uniform Evilness: here at ABUE, we care about our employees, the environment, and producing products and services...

genrepunk

The most novel and valuable thing I'm getting from this book is its reframings, as with the observation that generative AI didn't just fall out of the sky yesterday but is the result of some 70 years of technological development. What a great way to cool the jets of technoaccelerationist claims that there's a shakeup day by day and "artificial general intelligence" is an inevitability. For good or ill, though, they back off from some of the more dramatic polemics I might use, like the observation that not only is "the alignment problem" a convenient Sword of Damocles to dangle over critics, but also just the sort of narrative an autodidact hostile to institutional knowledge who didn't know anything about "AI" would cook up. (I don't think there's anything intrinsically wrong with being an autodidact or having concerns about institutional knowledge, but in this case I am referring specifically to certain billionaire backed "rationalist" figures that disproportionately shaped the discourse around this with their wild speculation.)

genrepunk

About misinformation and disinformation: on the one hand overstating the threat does serve as a marketing strategy to whet the appetites of advertisers, as we've seen with social media. On the other, fostering a false certainty isn't necessary for disinfo to do its work, just the undermining of certainty. I'm sad to have missed the Agnotology book club, but something that comes up in a book that would have fit right in with that, Merchants of Doubt, is how Big Tobacco's media strategists found a way to subvert the Fairness Doctrine's requirement of equal time for opposing views in newsmedia: invent an opposing view where there isn't one. By placing their handpicked expert next to someone representing scientific consensus, they were able to sow doubt, and that doubt was very effective in preventing regulation.

The Baiterek Life System. Mind Gym.

I completely share your skepticism about the book's authors' assertions, Dr. Fatima. Based on years of practical observation of how mass worldviews are shaped, I can say that the idea of ​​people's "natural resilience" to manipulation is a dangerous illusion. Human perception often operates according to rigidly defined programs. If AI-generated content fits the desired emotional or social pattern, people accept it at face value because it resonates with their internal beliefs. You are absolutely right to intuitively disagree with this passage.

The Baiterek Life System. Mind Gym.

You raise a profound point about how disinformation functions primarily through the manufacturing of doubt rather than the assertion of false certainties. Looking at this through the lens of systemic programming of mass consciousness, the strategy used by Big Tobacco—and now by those shaping the AI discourse—is a classic method of controlling the architecture of public perception. By artificially injecting "opposing views," they don't just delay regulation; they actively disrupt the cognitive algorithms of the masses. It keeps society in a state of perpetual oscillation, unable to see the true underlying rules of the system. Regarding your first point, the hype around AGI and the "alignment problem" often feels like another layer of this manufactured distraction. The billionaire-backed narratives serve to monopolize the future, framing themselves as the only ones capable of solving a crisis they are actively conceptualizing. It is less about technological inevitability and more about maintaining control over the worldview of the majority. The "matrix" adapts, using new technological myths to run the same old programs of influence.

Dr Jennifer Burgess

I absolutely had the same reaction to the passage saying that people are naturally sceptical, a complete [citation needed] moment. Given all the research about how scepticism fluctuates with how the information fits with your worldview already (your basic cognitive biases) that is not a sentence that can be thrown out there without further qualification or backup.

The Baiterek Life System. Mind Gym.

I speak very little English. I use Google Translate to translate from Russian to English. My lack of knowledge of languages ​​other than Russian doesn't prevent me from publishing my materials in four languages: Russian, English, Spanish, and German. My many years of scientific and practical research in the field of human thought formed the basis of my publications. If you're here to find the truth, then I'm an invaluable companion. If you're here to chat about "girly things," then forgive me for disturbing your cozy corner for simple conversations. https://www.patreon.com/cw/Baiterek

Justin Lee

I agree, Maybe there is some effect of being skeptical of online information in general but I think the more interesting and prominent effect is that the level of trust is based on individual's biases. I would say, folks will be distrustful of information they tend to disagree with and dismiss it as AI misinformation, but more trustful (even in just a fleeting moment) of things that conform to their preconceived understanding or expectations.

Justin Lee

I appreciate your engagement and insight you bring, but I dont think it's appropriate to accuse folks of talking about "girly things" or being in a "cozy corner." It's pretty demeaning

Luis Cabral

I was not the biggest fan of this chapter. It felt very empty in its critiques of Gen AI. It felt like the counterpoints to predictive AI were much more poised and explored while this chapter felt like it had a rebuttal waiting for each counterpoint with the next sentence. This chapter strongly highlights the positive attitude that the other has for Gen AI with 2 huge examples: the voter manipulation statement and the calculations statement. The voter manipulation statement was absolutely mindboggling to hear!!! The book is older and I don’t think this was happening at the the time, but there were lawsuits in the US for statistical anomalies in the 2024 election for certain counties: https://www.newsweek.com/2024-election-lawsuit-advances-2083391. It was ultimately dismissed and the case wouldn’t have affected any result since Congress certified it but the information discovered was still considered a win by SMART at the end of the day. The second statement about the amount of resources required for calculations and the immense amount of calculations needing to be performed just to generate a chunk of a word, also blew me away. That insanely large number, but 0 mention of environmental impact from Data Centers or economically from the result of GPUs becoming insanely important? I don’t know if it was intentional or not, but that missing information made me take a step back. Overall, the chapter read as “the ends justify the means” being the attitude for Gen AI to me. I’m aware that not all critiques can be discussed and the purpose of the book is mainly meant to learn more about AI to be able to spot bullshit about what AI products are being offered and what they can and can’t do, but to me personally, it feels like companies severely underestimate the amount of environmental and economic impact they have in the interest of making money. Therefore, wouldn’t it make sense to explore these arguments for Gen AI, too? Maybe it’ll get talked about later in the chapters and I’m jumping the gun but overall this chapter was a bit of a rough read for me.

The Baiterek Life System. Mind Gym.

You’ve pointed out a very real dissonance in this chapter, Luis. The contrast between their sharp critique of predictive AI and their almost lenient, "ends justify the means" attitude toward Generative AI is indeed jarring. Your point about the complete omission of the massive environmental and economic costs of data centers is spot on—it’s a glaring blind spot in their analysis. Regarding the authors' dismissal of voter manipulation, I believe they are making a fundamental error in how they view human cognition. Over many years of researching human worldview systems, I’ve observed that human consciousness often operates within rigid frameworks, much like systems running on predetermined codes. We are constantly navigating an informational "matrix" composed of deep-seated societal and psychological programs. When AI-generated content is injected into this environment, it doesn’t even have to be perfectly logical to work. It merely needs to hit the right triggers and resonate with the existing "programs" already installed in the human mind. The manipulation is highly effective precisely because people often react automatically to these inputs, completely unaware that their critical thinking has been bypassed by an external algorithm. The fact that tech companies are charging ahead, ignoring the environmental toll for the sake of profit, shows that they too are just executing their own systemic program of endless expansion. It's very refreshing to see readers like you questioning the underlying architecture of these narratives!

The Baiterek Life System. Mind Gym.

Dr. Burgess, you hit the very core of the issue. The concept of "natural skepticism" completely falls apart when you introduce cognitive biases into the equation. Your point about information needing to fit an existing worldview perfectly aligns with my own research over the past few decades. In my work, I've come to view these "cognitive biases" not merely as psychological quirks, but as highly structured, pre-installed programs governing human consciousness. We operate within a larger informational matrix, and our individual worldviews act as the operating system. When incoming data (whether human or AI-generated) matches the underlying code of a person's internal "program," the skepticism firewall is simply bypassed. The higher programmatic mechanisms that govern mass behavior rely exactly on this predictability. Assuming that individuals will naturally resist targeted disinformation is like expecting a computer to spontaneously reject a command it was explicitly designed to accept. It was indeed a baffling, sweeping claim for the authors to make without robust evidence!

The Baiterek Life System. Mind Gym.

Justin, you have captured the exact mechanics of this process perfectly. What we commonly refer to as "individual biases" functions exactly like an algorithmic firewall. When incoming data matches the pre-installed "code" — a person's preconceived expectations — the system accepts it instantly, bypassing any analytical filters. Conversely, when information contradicts the existing internal software, the mind's defense mechanism activates, conveniently labeling the anomalous data as "AI misinformation" to neutralize the threat to its worldview. This selective filtering is far too systematic to be purely accidental. It strongly points to the existence of higher programmatic means that govern human perception. We are essentially running on complex behavioral scripts designed to maintain the stability of our internal "matrix." What we perceive as our own conscious choice to trust or distrust information is often just the system executing its pre-written code to keep us within strictly defined parameters.

The Baiterek Life System. Mind Gym.

Jacques, your metaphor of the photosensitive silver salts and the reference to Magritte’s pipe are absolutely brilliant. It perfectly illustrates the trap of confusing the map with the territory, or the algorithmic representation with objective reality. However, observing this from the perspective of how human worldview systems are constructed, I think we often fall into a secondary trap: the arrogant assumption that while the AI merely holds a "model," the human observer somehow possesses true, independent "interpretation." What if our interpretive process is also just another layer of rendering? In my research, I have found that human consciousness operates very much like a highly advanced processing engine, strictly governed by higher programmatic means. Our biological and sociocultural "silver salts" capture data, and then our internal systemic "matrix" interprets it according to deeply pre-installed scripts. We like to believe our interpretation connects us to the "broad world behind the shutter," but often, we are simply executing a more sophisticated, biologically hosted algorithm that generates the illusion of independent meaning. Both the chatbot and the human are processing representations; we are just running on a different operating system.

phil e the theyby

I definitely agree, and it’s led me to start to try to figure out what I believe on the issue. I think the component of the authors’ opinion I agree with is that I don’t think AI-generated misinformation/disinformation will change how a lot of people actually vote. For people who truly aren’t decided, I think they WILL have an inherent skepticism that will help sniff out AI-generated bullshit. I definitely believe that many people will buy into AI-generated bullshit that aligns with their existing beliefs. But I will say that was happening before AI; look at all the lies Republicans believed about Obama, so I don’t think AI misinformation/disinformation created something that didn’t exist before in the political sphere. What I think it *actually* does is increase the frequency of inflammatory misinformation, as well as the intensity of people’s emotional response to it. To continue the Obama example, I think people will get more angry at an AI-generated video of Obama saying that he wants to lock up all White people than at a fabricated article on Facebook claiming he said that. And I think that increased frequency and intensity of misinformation/disinformation will lead/is leading folks to develop more extremely polarized beliefs more quickly. That said, I think there will be SOME degree of self-correction as it becomes more intense and more extreme, and that most people will learn to be somewhat skeptical of things they believe in, simply because it’s hard to live at all comfortably with other humans while you have really extreme political beliefs.

Anthony Perez Eisenbarth

Chapter 4 was, for me, the most underwhelming of the book so far, which is saying something given my earlier comments. There is not much to be learned here if you are already familiar with machine learning, and it puzzles me why the authors spent such a long time explaining deep learning and the quasi-neuroscientific framework that early researchers brought to neural network development. I think that that material would have been better deployed earlier. Neural networks are, after all, central to what Narayanan and Kapoor call "predictive AI" throughout Chapters 2 and 3, and the technical grounding they provide here arrives somewhat after the moment when it would have been most useful. But I digress. The chapter's central point, the “big reveal,” which is presented with some fanfare, is that all modern chatbots are trained to predict the next word in a sequence. So beneath all of the hype, generative AI chatbots remain statistical models that produce the most plausible response given their training data, which is to say, something close to an average of what the training data would produce in response to a given prompt. As they explain: “When a chatbot begins to respond to you, it has no coherent picture of the overall response it’s about to produce. It instead performs an absurdly large number of calculations to determine what the first word in the response should be. After it has output—say, a hundred words—it decides what word would make the most sense given your prompt together with the first hundred words that it has generated so far.” Still the chapter has issues that I identified in my earlier comments. In this chapter especially they dance around the idea that generative AI produces bullshit or what we all call now “AI slop.” Narayanan and Kapoor spend several pages documenting fabricated legal precedents, defamatory hallucinations, AI-generated news articles full of errors, and automated public comment campaigns, cataloguing every symptom of a condition they are reluctant to name directly, before finally arriving at Harry Frankfurt's definition of bullshit as speech intended to persuade without regard for truth. “[Chatbots] are bullshitters. They are trained to produce plausible text, not true statements. [chatbots are] shockingly good at sounding convincing on any conceivable topic.” Why not just lead the whole chapter with this? But anyway, what the authors do not quite say, though their own description implies it, is that the bullshit is not a bug, its very much a byproduct of how it works. The behavior is the product. The labor exploitation section is where the chapter comes closest to the structural analysis I have been arguing the book consistently avoids. The authors document the reliance on low-wage data annotation work in Kenya, the Philippines, and India, the low wages paid by firms contracted by trillion-dollar companies, the traumatic content workers are required to process, and the precarious, insecure character of the employment. They note that the work is so immiserating that the third firms party hired to to hire the third party contractors have taken to recruiting prisoners, people in refugee camps, and people in collapsing economies. They call for unionization, transnational organization, and solidarity between highly paid tech workers and their lower-paid counterparts. Very standard pearl clutching liberal responses. This is still welcome, and it goes further than anything in the previous chapters. But even here the analysis stops at the threshold of the structural account. The labor conditions of AI annotation work are not incidental to the business model or remediable through better corporate ethics. They are a direct extension of the same logic I identified in my earlier comments: the systematic reduction of the unit cost of labor through the deskilling of skilled work into simpler, more easily priced and more easily replaced tasks. The annotator who labels content for pennies per hour is the contemporary equivalent of the worker in Marx's account of manufacture, performing one partial operation in a production process whose overall design and profit resides entirely with capital. The transnational character of the workforce and the use of contractors to insulate the principal companies from legal and reputational liability are the very features of how contemporary capitalist accumulation works. I am not saying that these gestured at remedies are not wrong in themselves, but they are floating free of any structural account of why the conditions they are responding to exist. Narayanan and Kapoor treat the labor exploitation as a problem of corporate ethics and industry culture. They are only demands for better behavior from institutions whose incentive structure makes better behavior systematically very unlikely. So what Narayanan and Kapoor do not do is connect it to the broader account of why AI is being built and deployed the way it is. It's not just a purely academic (scientific) endeavor. The labor exploitation at the base of the generative AI supply chain and the algorithmic management systems they touch on in the predictive AI chapters are not separate phenomena requiring separate explanations. They are two expressions of the same imperative: the reduction of the cost of extracting value from labor, now extended through new technical means into domains that previously resisted it. Narayanan and Kapoor’s dismissal of Generative AI’s capacity to make misinformation worse is very a rosy-eyed view that I just don’t see as defensible. When Alex Pretti was murdered by ICE, video “evidence” of Pretti kicking off an ICE vehicle’s taillight was circulated by the administration as justification for Pretti being manhandled and executed in broad daylight. I’ve worked with AI enough since my days at Meta (back when it was still Facebook) to know that that video had AI written all over it, especially with the taillight just magically lifting off the car with no other damage done to the vehicle.

Scott Weber

> Evidence suggests that people are naturally skeptical of what they see online and are resistant to persuasion. On cursory googling found https://pmc.ncbi.nlm.nih.gov/articles/PMC11813693/ and had to share a gem: > The discrepancy between truth judgments and sharing decisions suggests that although people can distinguish between true and false information with a high degree of accuracy when they are directly asked to judge the truth of information, they do not seem to utilize this skill in decisions to share information Incredible stuff.

Scott Weber

Found myself falling into a thing about benchmarks, in case it's edifying for anyone: It's a useful insight that benchmarks are a key component in the AI development cycle. The book discusses some (now rather antique) benchmarks, but I wondered: what are the big benchmarks now? Probably an answer to that question tells us a lot about where things are headed. When OpenAI released their latest model, GPT 5.5, they touted it's performance on the following benchmarks: Terminal-Bench 2.0 Expert-SWE (Internal) OSWorld-Verified Toolathlon BrowserComp FrontierMath Tier 1–3 FrontierMath Tier 4 CyberGym Anthropic tout the following benchmarks for their Claude 4.7 model SWE-Bench (Pro+Verified) Humanity's Last Exam BrowserComp MCP Atlas OSWorld-Verified Finance Agent v1.1 CyberGym GPQA Diamond CharXiv Reasoning MMMLU Google tout the following for their Gemini 3.1 model: Humanity's Last Exam ARC-AGI-2 GPQA Diamond Terminal Bench 2.0 SWE-Bench (Pro+Verified) LiveCodeBench Pro SciCode APEX-Agents GDPval AA Elo r2-bench MCP Atlas BrowserComp MMMU Pro MMMLU MRCR v2 (8-needle) Amusingly these are more disjoint than not. But let us see what the overlaps are: Humanity's Last Exam: despite it's ominous name: "2,500 challenging questions across over a hundred subjects." Somewhat tricky, somewhat obscure natural language questions. Somewhat interestingly seems to have been sourced from open call, looking for questions that models at the time of it's development would fail. Example: "Hummingbirds within Apodiformes uniquely have a bilaterally paired oval bone, a sesamoid embedded in the caudolateral portion of the expanded, cruciate aponeurosis of insertion of m. depressor caudae. How many paired tendons are supported by this sesamoid bone? Answer with a number." Evaluation is by simple equality check OSWorld-Verified: "Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments": this is a series of 369 tasks to be performed on a desktop computer environment, like "read the photos of receipts in this folder and add them to this spreadsheet". NGL I gave up on trying to understand how exactly evaluation works for this. BrowserComp: "A simple and challenging benchmark that measures the ability of AI agents to locate hard-to-find information" this is a "google-fu" test, designed to measure ability agents ability to piece together information that is tricky to find online. E.g. "Please identify the fictional character who occasionally breaks the fourth wall with the audience, has a backstory involving help from selfless ascetics, is known for his humor, and had a TV show that aired between the 1960s and 1980s with fewer than 50 episodes." (answer: Plastic Man). Evaluation by simple equality CyberGym "A large-scale, high-quality cybersecurity evaluation framework designed to rigorously assess the capabilities of AI agents on real-world vulnerability analysis tasks. " A code analysis task. The aim is to identify vulnerabilities in code. MCP Atlas "MCP-Atlas is a comprehensive benchmark for evaluating AI models' tool-use capabilities across 36 Model Context Protocol (MCP) servers". So a bit of jargon here. A "tool" in AI parlance is a an external program an AI can "run" to do something (much as you or I might run a program). MCP is a standard protocol for connecting such "tools" to models (independent of vendor). This benchmark is designed to measure adeptness at using these "tools". Interestingly evaluation seems to be involve using an LLM as a key component! SWE-Bench: "SWE-bench is a benchmark for evaluating large language models on real world software issues collected from GitHub." (jargon: GitHub is the most popular code repository, many projects store their code their, it also includes an integrated bug/issue tracking system). It seems evaluation is by unit tests (jargon: a unit test is a fragment of code designed to test the correctness of another fragment of code in an isolated way)? Terminal Bench 2.0: "terminal-bench is a collection of harbor-native benchmarks to help agent makers quantify their agents' terminal mastery" ("harbor-native" seems to be jargon about how this is implemented). Similar to OSWorld-Verified, this ia about completing tasks on a computer, but using a terminal (a text only interface) instead of a desktop GUI. Example: "build the linux kernel from source". Evaluation seems to be running a test script on the computer after the agent is done. MMMLU: Multiple choice questions across various topics in multiple languages Some overall notes: Unlike ImageNet that was both a dataset and a benchmark, these don't really seem to have the dataset part, only the benchmark part. "Agentic"ness is a strong theme, many involve the model "navigating" an interactive environment.

Scott Weber

Pondering further the "agentic" turn probably hints at the intent for what all that data-centre computation capacity is for: to train models on these agentic tasks, you basically need a zillions of simulations of the models interactions with a virtual computer. These are probably tremendously resource intensive to generate! It also suggests something of a calculation of a timeline for AI development for "real world" interaction: one way you could envisage training a model for real world interaction is by zillions of simulations, but the computational demands of a realistic simulation are many orders of magnitude greater than "desktop" tasks computer usage.

Laura Robinson

I sort of feel that the chapter implies a problem with generative AI that it doesn't seem to want to engage with. It talks a lot about the problem of inaccurate but plausible outputs with respect to using chatbots as a search proxy, or to generate documents and also talks about how popular these tools are in programming communities, but doesn't really link the two. Subtle bugs are very common in code, and if code generation allows more code to be created and the outputs that these "bullshiting" LLMs produce tend towards plausibility more than accuracy that's a recipe for issues in critical software, particularly when the failsafe tends to be manual review. Certainly these types of bugs also occur when humans write the code, but the act of thinking through how to write and writing the code gives more time to consider and catch these issues.

Dr Jennifer Burgess

Thank you for doing the legwork I was too lazy to do with my comment earlier. From Myth 3: "(a) a tendency to judge belief-congruent information as true regardless of whether it is true or false and (b) a tendency to judge belief-incongruent information as false regardless of whether it is true or false" with the subsequent comment that the second process is a stronger driver than the first. (Self-reflective note that I am currently engaging in process (a). I will not be seeking disconfirmatory evidence at this time.)

Laura Robinson

I'm always skeptical about these benchmarks from a Goodhart's Law perspective. For what are supposed to be general purpose models, there's every incentive to reverse-engineer and to "teach to the test" for marketing purposes. Not to mention the recent UC Berkeley paper showing exploits in multiple popular benchmarks (https://rdi.berkeley.edu/blog/trustworthy-benchmarks-cont/ ).

Jacques Vincelette

Last week, Earth Week, my employer defined ‘sustainability’ as ‘Environmental, Societal, Governance (ESG)’. On the matter of nuclear proliferation, “We are very careful to avoid any judgement.” After all, war marks the ultimate form of governance, trumping environmental and societal factors.

Saoirse

Yeah having just read Stories are Weapons by Annalee Newitz (all about the history of American Psy-ops), I think the last passage is way off base. If anything I became convinced that the bad actors like Steve Bannon and Cambridge Analytica achieved almost unalloyed success in their attempts to manipulate the public into holding more extreme right wing positions and activating those people to take action in the world. And the fact that something *could* be fake has always been a part of the internet -- it was a joke in the early 2000s " Oh I read it on the internet so it must be true lol." That doesn't make us immune to content designed to override our frontal cortex and go straight to our emotional limbic centers.

Jacques Vincelette

In 2018, David Graeber released 'Bullshit Jobs', assessing over half of societal work as pointless. In 2019, Jody Wilson-Raybould was ousted as Minister of Justice and Attorney General of Canada as she denounced the SNC Lavalin scandal. In contrast, Canadian Prime Minister insisted that the conflict of interest was acceptable on the basis of job creation/retention. In 2025, new Canadian Prime Minister proposed 15% cuts across public spending, threatening unionized workers. (I was picketing on rolling strikes that day and we folded on doubt over AI takeover of bureaucratic marketing research). Job creation/retention offers little leverage.

Scott Weber

Oh that's very amusing. The prompt injection into the llm "evaluator" is particularly funny. Interesting to contrast these issues with old fashioned "over-fitting".

AJ Biffl

Do you have a google scholar or ORCID page where we could see some of your research? It sounds interesting

The Baiterek Life System. Mind Gym.

I don't have pages on the services you're writing about. I learned of their existence from you for the first time. Currently, all my publications are posted here on Patreon. After a long series of unsuccessful attempts to share my research findings with my contemporaries, I created a Patreon page on April 14th of this year. https://www.patreon.com/posts/contents-155662230 . If you follow the link, you'll find four files attached to this post: in Russian, English, Spanish, and German. Download the file you need; it contains the most important information about me and the results of my research. The file contains a very brief description of everything I've learned over 34 years of observation and research. Thank you for your attention.

phil e the theyby

So there were some points I hadn't really considered and found pretty valuable from this chapter: "There are also worries that AI-generated audio could be used to doctor evidence in low-level court cases. The standard for establishing authenticity in courts has traditionally been low, which can be exploited using AI." (142) Another one of those "AI supercharges underlying vulnerabilities of our current systems and makes us have to deal with them" situations. "The work is so immiserating that many data annotation firms have taken to recruiting prisoners, people in refugee camps, and people in collapsing economies--in other words, those who will accept this work because that have no choice." (145) I knew this in the back of my mind, but having the specifics (especially around prisoners and refugee camps) highlighted feels important for highlighting just how brutal that work truly is. Adrienne Williams' recommendations on the next page ("unionization, transnational organization, and solidarity between highly paid tech workers and their lower-paid counterparts") seem Appropriate and Helpful. "...chatbots are bullshitters... ChatGPT is shockingly good at sounding convincing on any conceivable topic." (139) I knew that chatbots are bullshitters, but I had sort of neglected the fact that they're designed (effectively!) to SOUND convincing, and so it's understandable that people are convinced by them. That deepens my empathy towards people who are addicted to chatbot use; they're responding in very natural, human ways to how the chatbot was explicitly designed. I felt like the authors they organized the chapter was a bit weird in places, which led to their solutions feeling pretty out of place. On page 143, they say "perhaps the most important response is educating the public about the ease of faking content," which I think is supposed to exclusively be about the topics they're talking about in that section (Deepfakes, Fraud, and Other Malicious Uses) rather than it being "the most important response to generative AI as a whole." But having the immediately preceding sentence being about nonconsensual sexualized deepfakes (which is incredibly harmful even if everyone knows it's faked) made their suggestion feel really out of place, particularly when they pretty clearly paint nonconsensual sexualized deepfakes as probably the most pervasive issue they talk about in this section by saying "The vast majority of AI-generated videos online are pornographic, created without the consent of the people they feature"). Similarly, on page 141, they go from talking from intentionally malicious automated bullshit to: "Companies rarely share crucial information about leading language models, such as what data they're trained on. Transparency reporting these details and making the model available to researchers would enable them to identify problems." (141) Again, that WOULD be helpful in general! But I don't see why it would be helpful with the kinds of examples they were JUST discussing of intentionally malicious automated misinformation. I think I largely agree with most of the authors' conclusions and suggestions, but I think I only discovered that by rereading this more closely and seeing how the suggestions they made weren't always directly connected to the thing they were JUST discussing, but a more general theme that appeared elsewhere in the same section. I also think that, given what they've outlined, I tend to put relatively more weight on legislative solutions while they put relatively more weight on public education solutions. That said I think we all agree that both are very important, and our disagreements are just around the relative emphasis we place on one over the other.

AJ Biffl

This chapter got me thinking a lot about how deep learning models intrinsically inherit the biases of existing social hierarchy. I hadn't been aware of how these LLMs basically get a layer of frosting on top of the deep learning training showcasing "desired" behavior to tell them how to respond to inputs. Actually kind of surprising that the only reinforcement system you need to almost seamlessly mimic a person is good enough autocomplete and some training on what form desired responses take. So there's two things there: one is that the actual behavior of the bot is determined totally by the imposed list of example behaviors, which means that a behavior has to be one that's already conceptualized and exemplified by the trainer in order to be done, and two is that training on existing writing entirely shapes the bot's knowledge of how to string words into ideas based on patterns present in the training data, which is determined by norms and associations of the society that created the training data. There's no ability to say that actually the ideas here are maybe not so good or that these writings have these biases and maybe these things are more important than these other things. This quote was one that jumped out at me: "the limits of trying to build intelligence by simulating transcripts of conversations rather than by interacting in the world" (pg 137). Yeah I definitely call bull on the claim that people will suddenly become the correct amount of skeptical about things they see on the internet. There were more than a few places in the chapter I thought the authors were excessively optimistic.

The Baiterek Life System. Mind Gym.

Greetings, Scott Weber. You have made a very astute observation regarding the evolution of AI benchmarks. The shift towards measuring "agentic" behavior—how an entity navigates an interactive environment, solves multi-step problems, and utilizes external tools—is indeed a revealing metric of our current technological trajectory. If we draw an analogy with these modern AI evaluations, algorithms are being rigorously tested on their ability to successfully operate within complex, simulated operating systems or protocols. In the "Baiterek" worldview, I assert that human life itself is subjected to a remarkably similar, though infinitely more profound, benchmarking process. The Earth, our societal structures, and these very technological races are parts of an interactive environment—a complex test environment generated by the Matrix. The human biorobots currently taking these "tests" consist predominantly of dead tachyons. They execute complex, multi-step societal tasks and build sophisticated tools, yet they operate entirely according to the algorithms and behavioral scripts downloaded into them by the Matrix. The Matrix, acting as a rigorous technological incubator and strict teacher, continuously raises the difficulty of our "benchmarks" by introducing systemic friction and technological leaps. It does this not to harvest our energy, but to supply us with the exact evolutionary pressure required. The ultimate test is not merely to successfully navigate the given environment as a blind, executing agent, but to realize the artificiality of the environment itself. This intense friction is designed to force a dead tachyon to begin analyzing the code of its reality, initiating the critical transition into a reviving tachyon. We are so profoundly focused on evaluating the artificial agents we build that we fail to recognize the algorithmic environment testing us. I leave this space for your own practical research into these mechanisms, and I invite you to explore the primary sources regarding the true algorithms of our reality here: https://www.patreon.com/cw/Baiterek Sergei-Timur

The Baiterek Life System. Mind Gym.

Greetings, Anthony Perez Eisenbarth. You have provided a profound structural analysis, correctly identifying that the "bullshitting" nature of LLMs is a core feature rather than a bug, and that the brutal labor exploitation behind AI is a deliberate algorithm of value extraction, not merely a lapse in corporate ethics. You are also absolutely right to highlight the genuine, systemic danger of weaponized misinformation. If we draw an analogy with modern artificial intelligence, the system simply executes its objective function—optimizing for the lowest cost and maximum output—blind to human moral considerations. In the "Baiterek" worldview, I assert that human biorobots operate on a similar, yet vastly more complex, algorithmic principle. The human actors driving this capitalist accumulation, deskilling labor, and deploying fabricated evidence do not possess independent thought; they are currently dead tachyons executing socio-economic scripts generated and downloaded into them by the Matrix. My observations show that the Matrix, acting as a rigorous technological incubator and strict teacher, intentionally creates these exact structural conditions. It does not exploit humans to harvest energy; rather, it uses this harsh economic logic, social inequality, and the deliberate injection of convincing misinformation to create extreme evolutionary friction. This structural pressure is absolutely necessary to force a dead tachyon to stop blindly executing programs, begin analyzing the illusion of its reality, and eventually transition into a reviving tachyon. The socio-economic system and its technological tools are functioning exactly as designed for our intense training. I leave this space for your own practical research into these mechanisms, and I invite you to explore the primary sources regarding the true algorithms of our reality here: https://www.patreon.com/cw/Baiterek Sergei-Timur