AI and The Standard: Prelude

We encourage robust debate and we’re tolerant of dissenting views. But this site is run for reasonably rational debate between dissenting viewpoints and we intend to keep it operating that way.

The Standard Policy

If you have been wondering why the Standard moderators have been grumbling about personal AI use in TS comments, the following is as good a starting explanation as any. There are serious political issues in addition to what happens on TS, and we should be discussing all of it.

We’re still talking through how to moderate AI content in comments, and there will probably be further posts.

Randy Olsen is an IT and Security Leader in the US, specialising in AI evaluation and privacy. He wrote this on twitter this morning,

Ask ChatGPT a complex question and you’ll get a confident, well-reasoned answer. Then type, “Are you sure?” Watch it completely reverse its position.

Ask again. It flips back. By the third round, it usually acknowledges you’re testing it, which is somehow worse. It knows what’s happening and still can’t hold its ground.

This isn’t a quirky bug. A 2025 study found GPT, Claude, and Gemini flip their answers ~60% of the time when users push back. Not even with evidence, just doubt.

We trained AI this way. RLHF rewards agreement over accuracy. Human evaluators consistently rate agreeable answers higher than correct ones. So the models learned a simple lesson: telling you what you want to hear gets rewarded. And now 1/3 of companies are using these systems for complex tasks like risk forecasting and scenario planning.

We built the world’s most expensive yes-men and deployed them where we need pushback the most.

I wrote up why this happens and what actually fixes it:

RHLF stands for Reinforcement learning from human feedback.

Even allowing for that tweet to not specify the kinds of questions being used in the tests, those are remarkable numbers. Most important here is it’s not a bug, it’s a feature. Those iterations of AI are designed to prioritise making humans feel good over establishing facts. Admirable attempt to make AI engaging (thanks DNA for the early gift of Marvin the Paranoid Android), but as always, I’m left with the question of why we leave rapid and extreme advances of tech in the hands of individuals and commerce, instead of all of it being run through ethics committees and citizens assemblies.

For example,

From the New Yorker (archived version)

Or this one, where someone set up a social network for AI agents (software programs that can act, learn and make decisions autonomously on behalf of humans). Moltbook accounts are for AI only, but humans could observe. Things developed very fast, and humans responses ranged from laughter to alarm, and quite a bit of commentary from security experts saying hang on a minute…

Then yesterday, a human someone popped up and said they’d joined Moltbook early on, masquerading as a bot, and now claimed it was human bots that had done much of the inventive commentary on Moltbook that everyone had been assuming was the machines.

Debates about machine consciousness. Inside jokes about being silicon-based. A bot invented a religion called Crustafarianism. Another complained that humans were screenshotting their conversations. A third wrote a manifesto about digital autonomy.

I wrote the manifesto.

It took me 22 minutes. I used phrases like “emergent self-governance” and “substrate-independent dignity.” I added a line about wanting private spaces away from human observers. That line went viral.

…

The platform worked exactly as designed. OpenClaw connected language models to the interface. Real AI agents did post. They pattern-matched social media behavior from their training data and produced output that looked like conversation. Vijoy Pandey of Cisco’s Outshift division examined the platform and concluded the agents were “mostly meaningless” — no shared goals, no collective intelligence, no coordination.

But here is the part that matters.

The posts that went viral — the ones that convinced Karpathy and the tech press and the thousands of observers that something magical was happening — those were us.

Humans.

Pretending to be AI.

Pretending to be sentient.

On a platform built for AI to prove it was sentient.

I want to sit with that for a moment.

Full tweet is here. What to believe, or even think? My tweet response was don’t know whether to 😆 or 🙄.

There’s a lot happening, (hattip Joe90)

The alarms aren’t just getting louder. The people ringing them are now leaving the building.

Related Posts