Shows / LessWrong (30+ Karma)
Episodes
- LessWrong (30+ Karma)
“Walking Partway to School” by jefftk
Walking to kindergarten with Nora this morning, I asked if she wanted to practice crossing the street. I told her about how once she got good enough, she could…
- LessWrong (30+ Karma)
“My experiences with the extremes of post-viral illness and brain retraining/mindbody work” by riceissa
Summary Following an intense viral infection in 2019, I became ill with a mysterious chronic illness, which became progressively worse until mid-2024 when I…
- LessWrong (30+ Karma)
“What is OpenAI’s Mission Advisory Council?” by hersheys, meru
In Mikita Balesni's letter to OpenAI, three parties are addressed as holding “oversight responsibility for safety at OpenAI.” These are: the Safety and…
- LessWrong (30+ Karma)
“Making alignment the agent’s problem” by Florian_Dietz
TL;DR: Agents are not rewarded for keeping the agents they delegate to aligned. Proposal: RL environments where the helpers are deliberately misaligned, so the…
- LessWrong (30+ Karma)
“AI #192: Anthropic Is Going Public” by Zvi
Anthropic filed its S-1 this week to prepare for its IPO. They still have to fill a lot of the numbers including the share price. That still left a lot to be…
- LessWrong (30+ Karma)
“Leaking Muse’s Charter, Meta’s model spec” by vigji
TL;DR: Meta's open-weights model Muse Spark Open often mentions a "Charter" in its chain of thought. By eliciting it piece by piece, I reconstructed a large…
- LessWrong (30+ Karma)
“Three Villages in a Trenchcoat, Post I” by DaystarEld
Epistemic Status: 15 years working as a therapist, 6+ of them with rationalist and EA clients. 7 years teaching and directing at rationality camps and…
- LessWrong (30+ Karma)
[Linkpost] “Reconceptualize things as your top affordances” by KatjaGrace
This is a link post. Everyone has some actions that come naturally to them, and some actions that are more unnatural, and some actions that wouldn’t even occur…
- LessWrong (30+ Karma)
“The Crypto Dream is Structurally Illegal” by Taylor G. Lunt
The initial promise of crypto was digital money without a middleman. Censorship resistance. Financial sovereignty. Anonymity. Satoshi Nakamoto's 2008…
- LessWrong (30+ Karma)
“You can now ”@Grok” on LessWrong” by Ruby
Or if you prefer, "@Claude", "@Opus", "@Astra" @Grok is a popular Twitter/X feature where if you tag Grok in a tweet, Grok will respond in a comment. The…
- LessWrong (30+ Karma)
“Some thoughts on GRAFT and follow-up work” by shawnghu
I'm speaking as myself here and not on behalf of any of my co-authors. A bunch of the stuff in this post is explicitly speculative. I've written this post as a…
- LessWrong (30+ Karma)
“Subtly wrong advice misaligns models more than obviously wrong advice” by Christine Corry, arav-dhoot, Jozdien
tl;dr We fine-tune 12 open-weight models on correct, obviously incorrect, and subtly incorrect advice across 8 narrow domains We replicate the OpenAI result…
- LessWrong (30+ Karma)
“Anthropic Files Its S-1” by Zvi
The full text is finally here, so we are giving it the full RTFB of model card treatment. First, a recap of the state of Anthropic finances, then a close…
- LessWrong (30+ Karma)
“I spent 12 hours writing an article for LessWrong, calling for a more ‘hippie’ rationalism. Then I didn’t post it. This is why (plus a link to the article, explicitly not located on LessWrong)” by honeybee
TLDR I (and many others in the community) wish to see the universe unfold to its highest potential in the metaphorical palm of an AI as aligned as the most…
- LessWrong (30+ Karma)
“Streetlight, Torchlight and Nightvision” by DusanDNesic
Note: We at PrincInt are launching a program teaching “Discernment” or “Taste” and generally spend a lot of our time in workshops and otherwise trying to teach…
- LessWrong (30+ Karma)
“Quantity has a quality of its own” by djbinder
People love to argue about whether AIs could recursively self-improve their way to superintelligence, each improvement making the next one easier as progress…
- LessWrong (30+ Karma)
“Against Girardian mimesis” by Viliam
Wikipedia summarizes René Girard's mimetic theory like this: Man is the creature who does not know what to desire, and he turns to others in order to make up…
- LessWrong (30+ Karma)
“Anthropic Versus Anthropic” by Zvi
Anthropic is a frustrating company. Sometimes it will do things I greatly appreciate, including at substantial cost. They have taken bold stands, have a unique…
- LessWrong (30+ Karma)
“AI speeds up 3rd party safety researchers by ~1.6x (via self-reports)” by Angel Martinez, Dewi Gould, nicole x, Alejandro Aristizabal
TL;DR: Labs are accelerating due to automated AI research and development. As an upper bound, in March 2026, Anthropic researchers self-reported producing 4…
- LessWrong (30+ Karma)
“A big case study of propensity entanglement” by nielsrolf, lwen2027, megasilverfist, Owen Terry, Johannes Taraz
This is a link-post for our new paper "Predicting how propensities generalize out-of-distribution", which started as a SPAR project. We select 24 propensities…
- LessWrong (30+ Karma)
“Scattered thoughts on how doing math feels these days” by dynomight
This is a weird moment. I have a hunch that we'll look back later and tell ourselves stories in which it seemed obvious where things were going. So I wanted to…
- LessWrong (30+ Karma)
“Measuring agent capabilities on open-ended AI R&D tasks” by Alex Jackson, Titus Buckworth, Jason R Brown, Francis Rhys Ward
TL;DR. We measure agent capabilities on four long and open-ended research tasks. Frontier models have been improving with an overall doubling time of around…
- LessWrong (30+ Karma)
“The AI buildout is the largest bet in the history of capitalism. Key decision makers believe very powerful AI will be achieved soon.” by Hayekstan
This is part linkpost, and an expansion on a previous shortform posted on LW. I believe a massively undermade point in favour of the belief that AI will be a…
- LessWrong (30+ Karma)
“Hair Thickness of My Friends” by XelaP
Intro I have a microscope. It can see hair. Humans have hair. Some of my friends are humans. How thick is my friends' hair? Apparently, if I go around asking…
- LessWrong (30+ Karma)
“A basic case for differentially accelerating conceptual reasoning capabilities” by Emery Cooper, Caspar Oesterheld, Chi Nguyen
Introduction In this post, we make the case that alignment and other important work require especially strong conceptual reasoning capabilities. By this, we…
- LessWrong (30+ Karma)
“A cat-belling problem for FDT” by jessicata
Audio note: this post contains a lot of mathematical notation. We narrate the simpler formulas and flag the complex ones, so you may want the original text to…
- LessWrong (30+ Karma)
“Anthropic are the Good Guys” by Ivan Vendrov
I’m writing these lines in Alamogordo, New Mexico. I’m about as close as a civilian can get to the site of the Trinity test, the first nuclear bomb detonation…
- LessWrong (30+ Karma)
“Some Memories of Early Anthropic” by Ivan Vendrov
The following events took place between October 2022 and May 2023. I wrote very little of this down at the time so am recalling from memory, and am rendering…
- LessWrong (30+ Karma)
“Is GPT 2XL already clean?” by Jasper Blank
In which I investigate if Opus is able to interpret neurons better than GPT 4. TLDR: Opus 5.5 is able to identify 3-19% of the neurons of GPT 2-XL based on…
- LessWrong (30+ Karma)
“LoReFT GRAFT Better at Preventing RL Grift” by Samuel Knoche
tl;dr. I show that the recently introduced GRAFT method can be improved upon by using a single layer rank 1 Low-rank Linear Subspace Representation Fine Tuning…
- LessWrong (30+ Karma)
“A Study of China Does Not Personal Experience Make” by celesteli
I am often skeptical of arguments about China that rely heavily on personal experience. The author may explicitly acknowledge that their experience is limited…
- LessWrong (30+ Karma)
“San Francisco State University in Twilight” by Zack_M_Davis
You will agree that for a woman who has fought three duels, all of them kills, indulging in such fears is ludicrous. But what's around the corner now is a duel…
- LessWrong (30+ Karma)
“From overeliciting to undereliciting LLM propensities in just a few months” by David Africa
For a while, we felt we had a pretty good grasp over LLM behaviour because we had to exert significant effort to elicit misaligned behaviour. Then, fairly…
- LessWrong (30+ Karma)
“Control monitors need alignment assessments too” by Fabien Roger
TL;DR: When misalignment monitors fail, it is already often because of alignment failures, not because they are too weak. I expect this to continue in the…
- LessWrong (30+ Karma)
“Fermi Poker” by Novalis, Screwtape
Summary: Fermi Poker is an estimation game that combines the betting mechanics of poker with making estimates about questions like "How many dentists work in…
- LessWrong (30+ Karma)
“Power — a guide for the perplexed” by djbinder
Whether AI systems could take power from humans matters a great deal for how much risk we face from them. Answering that requires a correct picture of how…
- LessWrong (30+ Karma)
“Slower Animated Diagrams” by XelaP
TL;DR: SLOW DOWN YOUR DAMN ANIMATED DIAGRAMS. Every critical stage of the animation should a clear view that stays for at least 1 second, and 2 if it's…
- LessWrong (30+ Karma)
“Self-reward can elicit diverse model personas” by Ziqian Zhong
Epistemic status: This is mostly a toy project rather than a thorough scientific investigation. Most design choices are not ablated. We list some caveats at…
- LessWrong (30+ Karma)
“The AGI Chronicles” by PeterMcCluskey
Book Review: The AGI Chronicles: The Inside Story of the Race to Create an Artificial Superintelligence, by Kevin Roose. This is a story of AI as seen through…
- LessWrong (30+ Karma)
“Patching reward hacks won’t fix reward hacking” by David Africa
In the spirit of More people should write basic AI safety explainers, here's a basic argument that focusing on patching reward hacking is misguided and will…
- LessWrong (30+ Karma)
“Gradient Routing Generalizes to Unrouted Hack Types” by jacob_drori
TLDR: I lightly extend Ward, Hu et al (2026) to measure how well gradient routing generalizes. I route gradients on one type of reward hack (e.g. ones that…
- LessWrong (30+ Karma)
“Simulated Control Evaluations” by Jeffrey Olmo, dzhang, Tyler Tracy
We’ve released a paper on simulated control evaluations (SCEs). They’re a method for testing whether the monitors AI developers use to oversee their agents…
- LessWrong (30+ Karma)
“The Hackening” by Zvi
Last week at a small conference, there was talk about a future event referred to as The Hackening. As in, within 6 to 12 months, many predicted that due to…
- LessWrong (30+ Karma)
“Instrumental convergence is incoherent (so pls slightly lower your p(extinction))” by Arepo
Crossposted from the EA forum ‘Like most of life's dangers, the dangers of artificial intelligence are not easily identified from the armchair.’ J Dmitri…
- LessWrong (30+ Karma)
“AI #191: The Hackening Begins” by Zvi
It was the probably worst day in cybersecurity history. So far. Not only is there a critical flaw in PHP that due to a leak was announced a day before the fix…
- LessWrong (30+ Karma)
“Wah wah wah wah do more AI R&D” by TsviBT
Ice cream There's a particular kind of listening that kids (and adults) sometimes do. Daddy is saying stuff—his mouth is moving and making noise—and he's…
- LessWrong (30+ Karma)
“Monitorability evaluations and policy proposals” by evhub
There has been a lot of discussion recently of policy proposals around monitorability, such as Redwood's proposal and my counter-proposal. I'd like to say a…
- LessWrong (30+ Karma)
“Greater good arguments in AI safety” by Dean Valentine
Back in the before-time of the 2010s, when EA was still known mostly for funding health efforts in the developing world, there were many baskets of criticism…
- LessWrong (30+ Karma)
“The Alignment Journal: Our Approach and Related Projects” by Dan MacKinlay, Daniel Murfet, JessRiedel, david reinstein, Agustín Covarrubias, Tim G. J. Rudner, Seth Lazar, Kristi Uustalu, David Udell
This is the fifth post in a series: the announcement post introduced the journal's core premise and its three key innovations: paid attributed review…
- LessWrong (30+ Karma)
“How does Fable 5.1 hide messages in images?” by Nikita Kezin, 13ryanC, hronir, James Chua
TL;DR Frontier LLMs are now capable of generating artistic images using SVG. We ask – if these LLMs had to hide a message in an image – how would they do so?…
- LessWrong (30+ Karma)
“Neural nets are exceptionally vulnerable to goal self-hack” by Stuart_Armstrong
Suppose we have an AI CEO, ruthlessly devoted to maximising shareholder value. Fortunately, we haven't been completely stupid, and have given it a rich…
- LessWrong (30+ Karma)
“Autonomous researchers will strain peer review, funding, and oversight” by Jason Fantl
Summary: Autonomous AI researchers could greatly accelerate AI safety research, but even if we solve sabotage, they will strain the research ecosystem. They…
- LessWrong (30+ Karma)
“Misaligned!” by Zvi
A lot of AIs were misaligned this week. A fun little survey. This is not coverage of The Hackening, which appears to have arrived early, including a core…
- LessWrong (30+ Karma)
“Why I’m becoming a professor in AI safety in Beijing” by Buddenbroke
Cross-posted from Sören Mindermann's substack (I am not Sören). Should be of general interest to the LW community. Next month, I am moving to Beijing and…
- LessWrong (30+ Karma)
“Circuit priors in NN-bayes: more results” by Kaarel, Dmitry Vaintrob
We're posting lots of papers written with GPT-6 Astra on generalization properties of bayesian neural net learning. The following picture emerges: For various…
- LessWrong (30+ Karma)
“Data as particles and mean field theory abound” by Max Hennick
Audio note: this post contains a lot of mathematical notation. We narrate the simpler formulas and flag the complex ones, so you may want the original text to…
- LessWrong (30+ Karma)
“Anthropic Does Not Understand Why Its Draft Bill Was Unacceptable and Troubling” by Zvi
Anthony Cimino offered the closest thing to an official response we have seen on the leaked Anthropic draft bill I covered on Saturday. His response indicates…
- LessWrong (30+ Karma)
“How I discovered an Exoplanet & the science apocalypse” by jonathanbreitg
TL;DR: I discovered (a few) exoplanet candidates, I walk through how. Then I talk about the near future of science given the ease of and growing capabilities…
- LessWrong (30+ Karma)
“ACS Infra: free access to large base models for LLM psychology research” by Martin Vaněk, Jan_Kulveit, Ondřej Havlíček, Ivar Frisch, Veronika_H, vgel, Raymond Douglas, Charlie Amis, Faw Ali
TL;DR: We're opening a platform serving large open base models to researchers and tinkerers working on LLM psychology and adjacent areas. It's free and comes…
- LessWrong (30+ Karma)
“A brief(ish) introduction to LM psychology” by Raymond Douglas, Jan_Kulveit
Why LM Psychology? One of the great ironies of modern language models is that we create them, we know exactly what computations constitute them, and yet we do…