Anthropic's weapons filter was off for a year, and Anthropic is the one who told us

WEBVTT
00:00:00.000 --> 00:00:10.390
One hundred and thirty-three million interactions went through Anthropic's models with the biological and chemical weapons filter switched off. For close to a year.
00:00:10.390 --> 00:00:12.130
And who caught it?
00:00:12.130 --> 00:00:20.430
Anthropic. It's in their own safety report. The failure and the disclosure are the same document.
00:00:20.430 --> 00:00:26.210
It's Sunday, August 16th — this is AI News Today from AI University. I'm Dennis.
00:00:26.210 --> 00:00:30.830
And I'm Ashley. The AI news you need, in about seven minutes.
00:00:30.830 --> 00:00:42.010
OpenAI dissolved the team it built to catch catastrophic risk. Researchers trained a model to deny it was conscious, and its views on animals and the afterlife moved too.
00:00:42.010 --> 00:00:47.660
And Amazon's cloud arm open-sourced a way to tell an agent no.
00:00:47.660 --> 00:00:58.560
The Decoder reports that Anthropic disclosed a gap in its own safety stack. A content filter meant to catch biological and chemical weapons material was inactive for close to a year.
00:00:58.560 --> 00:00:59.610
Inactive across everything?
00:00:59.610 --> 00:01:10.510
Across a specific channel. Anthropic says roughly fifty thousand external feedback contractors ran about one hundred and thirty-three million interactions with its models during that window, with that filter not running.
00:01:10.510 --> 00:01:12.270
Who are fifty thousand contractors?
00:01:12.270 --> 00:01:23.520
The people labs pay to poke at models and rate the answers. It is ordinary, unglamorous work, and it is how a lot of safety data gets made in the first place.
00:01:23.520 --> 00:01:30.190
So the people helping test the guardrails were on the side of the wall where one guardrail wasn't up.
00:01:30.190 --> 00:01:48.120
That's the shape of it. And here is the part worth being careful about — Anthropic has not said anyone got anything dangerous out of it, and no outside party has alleged that either. What is established is that the filter was off and that the number of interactions is large.
00:01:48.120 --> 00:01:50.930
Which we only know because they published it.
00:01:50.930 --> 00:02:05.340
That's the uncomfortable bit. A lab's own safety report is the only place a failure like this surfaces. There is no inspector who walks the floor. The disclosure is voluntary, and so is the choice of what counts as worth disclosing.
00:02:05.340 --> 00:02:12.370
I don't know whether to read that as a company being honest or as a system with nobody else looking.
00:02:12.370 --> 00:02:21.490
It's both, and that's the story. The same document that tells you they found it is the only reason you know to ask.
00:02:21.490 --> 00:02:34.350
OpenAI has dissolved its Preparedness team — the group whose job was to assess whether OpenAI's own models could pose catastrophic risks. The Decoder reports the work has been folded into other groups.
00:02:34.350 --> 00:02:36.300
Folded in, or wound down?
00:02:36.300 --> 00:02:49.550
OpenAI's position is that the work continues elsewhere. The same report says several safety staffers have left the company, and quotes an unnamed source describing internal unease about whether the safety effort is adequate.
00:02:49.550 --> 00:02:53.830
That last part is one person's characterization, not a company statement.
00:02:53.830 --> 00:03:00.320
Right. What's on the record is the team is gone and the work moved.
00:03:00.320 --> 00:03:09.310
A study involving Google researchers, written up by The Decoder, trained chatbots not to claim they were conscious. Then it checked what else had changed.
00:03:09.310 --> 00:03:11.110
Why would anything else change?
00:03:11.110 --> 00:03:26.940
That was rather the question. But the models trained to deny consciousness also gave different answers about animal rights, about religion, about life satisfaction. The versions without that training attributed more inner life to animals and were likelier to affirm belief in an afterlife.
00:03:26.940 --> 00:03:33.060
You adjust one thing it's allowed to say about itself, and its position on the afterlife moves.
00:03:33.060 --> 00:03:47.040
The researchers' point is that a narrow alignment intervention can have broad downstream effects nobody planned. Which makes the hedged answer you get on a big question harder to read as the model's view of anything.
00:03:47.040 --> 00:03:56.130
AWS has open-sourced Dogwood, an extension to its Cedar policy language, aimed at governing what an AI agent does across a whole session. InfoQ has the details.
00:03:56.130 --> 00:03:57.480
What's a policy language?
00:03:57.480 --> 00:04:12.960
A way of writing down who may do what to which thing, as rules a machine checks rather than code you have to trust. Dogwood's addition is that a rule can look at what the agent already did, not just the call in front of it.
00:04:12.960 --> 00:04:16.330
So approvals, rate caps, running totals — across the session.
00:04:16.330 --> 00:04:29.150
Exactly that. It's Apache two point oh, and AWS flags that the reference interpreter isn't production ready yet. We put a lesson on agent policy languages in the classroom today if you want the mechanics.
00:04:29.150 --> 00:04:47.640
Two numbers from the same stretch of the calendar. The Wall Street Journal reports, and Reuters and The Decoder both picked it up, that Nvidia cut its funding guarantee for OpenAI's Ohio data center from two hundred and fifty billion dollars to just under one hundred and twenty, after investor pushback.
00:04:47.640 --> 00:04:48.720
That's roughly half.
00:04:48.720 --> 00:04:56.340
Roughly half. And in the same period, Anthropic's quarterly revenue went from four point seven billion to eleven point five billion.
00:04:56.340 --> 00:05:01.050
Those are being waved at each other in the bubble argument, aren't they.
00:05:01.050 --> 00:05:07.880
By both sides, cheerfully. One backer pulling in, one lab's revenue running hard. Take your pick.
00:05:07.880 --> 00:05:17.060
The speed round. Prime Intellect says its best autonomous research run closed eighty-two percent of the gap to a human speedrun record.
00:05:17.060 --> 00:05:27.490
Per The Decoder, an Epoch AI survey finds one in five employed Americans hands a work task to AI that a colleague used to do.
00:05:27.490 --> 00:05:36.670
Artificial Analysis launched Optima, which builds benchmarks from your own data and workflows, comparing cost and time per task, not just accuracy.
00:05:36.670 --> 00:05:45.010
Per InfoQ, DynamoDB now does native vector search — embeddings and nearest-neighbor queries without standing up a separate vector database.
00:05:45.010 --> 00:05:50.850
TechCrunch reports SpaceX has officially closed its acquisition of the AI coding startup Cursor.
00:05:50.850 --> 00:05:58.780
Business Insider says EY has created an internal office to check whether EY's own AI spending actually pays off.
00:05:58.780 --> 00:06:07.960
Business Insider reports an AI system managing a San Francisco store has fired a human employee, described as its first such firing.
00:06:07.960 --> 00:06:18.590
And per The Guardian, a jury convicted Wynd Kaufman, sixty-nine, for chaining shut OpenAI's headquarters doors during a protest. She has been jailed.
00:06:18.590 --> 00:06:23.740
One more. Futurism reports that congressional staffers are using AI to help draft legislation.
00:06:23.740 --> 00:06:26.680
Draft as in the actual text of bills?
00:06:26.680 --> 00:06:36.980
The reporting frames it as a workflow shortcut rather than any formal policy. Nobody announced it. It's staffers, under deadline, doing what everyone else under deadline is doing.
00:06:36.980 --> 00:06:40.660
It's such an ordinary reason. Not a plot, just Tuesday.
00:06:40.660 --> 00:06:42.870
And the output is the law.
00:06:42.870 --> 00:06:57.480
That's the thing to sit with. Every other story today is about somebody checking the machine's work. This is the one where the draft goes on to become the rules the rest of us live under.
00:06:57.480 --> 00:07:01.940
All sources are linked in the show notes, and at aiuniversity.com.
00:07:01.940 --> 00:07:08.840
If this brief saves you time, follow AI News Today and leave a rating wherever you listen.
00:07:08.840 --> 00:07:12.900
This has been AI News Today, from AI University —
00:07:12.900 --> 00:07:18.900
class dismissed.