Designer babies. Self-improving AI. Are we ready for either?

How do we know when the world has changed?

On June 1, a team of scientists published a preprint scientific paper claiming they had edited human embryonic DNA with more precision than any previous attempt. As a technical achievement, the work is undoubtedly impressive, largely avoiding the errors that had accompanied earlier efforts to gene edit embryos. With further development, such embryonic editing could free future children from fatal or debilitating genetic diseases, but as the veteran science writer Carl Zimmer reported in the New York Times later that week, the real headline news was that the work “could open the way to babies engineered with particular characteristics” — designer children, in other words.

The same day the Times piece published, the AI company Anthropic published a post asserting that AI was already accelerating AI development, which the authors argue may represent an early step toward recursive self-improvement (RSI) — AI systems that design and build their own successors, faster and faster. Already most of the code that runs Anthropic’s Claude was written by Claude itself, which has helped the company’s engineers ship eight times as much code as they were two years ago. While more is not automatically better, and Claude is still far from being able to guide itself, the possibility of self-improving AI is on the horizon — and “it could come sooner than most institutions are prepared for,” as Anthropic co-founder Jack Clark and Anthropic Institute head Marina Favaro wrote.

These two writings were published by academic biologists and the employees of an AI company, in two wildly disparate disciplines, but they nonetheless point to a possible near future that is fundamentally different from the world we live in now.

Both events are potential key steps toward unprecedented powers — not all of which we would have firm control over: newly designed intelligences and newly designed humans. What the two share is not just consequence, but bivalence — the possibility of both the miraculous and the catastrophic. The biological precision that could eradicate an inherited disease like Huntington’s could also pave the way to a genetic caste system. The AI capability that could accelerate decades of scientific progress could also utterly disempower its makers — us.

The world may have walked through a historic door with both of these advances last week. But we can’t yet know which kind.

Take the biology step first. Strip away the headlines — which come from the media, not from the scientists themselves — and the experiment is fairly narrow.

Using so-called base editors, which make a small nick in a gene strand rather than chopping out an entire segment, as CRISPR does, Columbia University geneticist Dieter Egli and his team edited two genes: PCSK9 and HBG. You might have heard of the first one; PCSK9 produces a protein that affects the body’s ability to clear cholesterol from the blood, and certain mutations in the gene can drive LDL cholesterol levels dangerously high. HBG encodes a form of hemoglobin the body relies on before birth and normally switches off afterward. Being able to control these genes could prevent the mutations that increase heart disease risk (PCSK9) and reactivate that fetal hemoglobin in adulthood, easing — though not curing — sickle cell disease and beta-thalassemia (HBG).

The researchers delivered their base editors into fertilized eggs and into two-cell human embryos, and in some cases they managed to make the edits without the chromosomal damage that had been associated with earlier attempts to edit using CRISPR.

The paper — which has yet to be peer-reviewed — is an impressive step forward in the effort to use gene editing technology on human embryo genes with greater precision. But impressive is still far from perfect, or even safe — some edits landed at the wrong spot in the genome, and relatively few of the embryos went on to develop normally. (The embryos, which had been donated by IVF patients, were developed no further than very early stages and none were implanted.) Egli and his colleagues were clear in the paper that any notion of using the base editing technique as it is now for treatment is “premature.” But the paper does show such editing can now apparently be done without shredding chromosomes.

When the Chinese scientist He Jiankui used conventional CRISPR to edit human embryos in 2018, producing three children, his work was widely rejected not just for moral reasons, but technical ones, as his clumsy gene editing did real genetic damage. Should the new paper’s results bear out, the technical obstacles to embryo engineering begin to vanish.

No one knows what comes next. Certain genetic disorders like sickle-cell anemia can be fixed with a single gene edit, but preventing more complex health problems — or engineering the traits some people might dream about, like height or intelligence — would require editing hundreds or even thousands of genes in combinations we don’t fully understand yet. But if the technical barriers keep falling, that will only leave the moral ones — and the moral ones have rarely held back a technology for long.

As revolutionary as the ability to truly engineer human beings would be, biology still moves slowly. The same can’t be said for the subject of the other document released last week.

Anthropic’s post uses over 5,000 words and plenty of (I’m guessing) Claude-produced graphics to make a single point: The proportion of human work that goes into building AI is shrinking at every stage. Engineers who once wrote the code now mostly review what Claude itself writes. Experiments once designed manually are now increasingly proposed and run by the model. While humans still make the judgement call about what is worth building, Anthropic argues even that has started to change, as employees increasingly defer to what the model proposes to do next.

A research loop that is increasingly dominated by AI itself is one that could move ever faster. Technology has always changed at the rate of human beings — how fast they can think, plan, and act. An AI capable of improving itself eliminates that speed limit, allowing for the very real possibility of it moving faster than any human or any human-run institution charged with governing it can follow. Intelligence itself goes critical — each smarter model building a smarter one, the reaction sustaining itself.

That might seem like a lot to put on a few months of internal coding data from an AI company that has a vested interest in making its models look as strong and as smart as possible. (Especially if that AI company happens to have a potentially record-breaking IPO on the horizon.) In the post, Anthropic itself concedes that simply counting lines of code only goes so far, and that speed is only at best a partial metric of success. But independent research has shown that AI models are able to spend longer and longer on a single task, which allows them to work not just quicker but deeper. We can quibble over the speed, but not on the idea that AI is moving forward, and fast.

Powerful and blindingly quick AI could lead to rapid economic, scientific, and medical progress — all the dreams Anthropic CEO Dario Amodei has laid out in his own writing.

But it also threatens to be existentially dangerous as well as profoundly disempowering for most of us, not unlike genetic human enhancement could be for those left out. And the potential speed of such change is so great that Anthropic makes the unusual proposal of calling for AI companies to consider collectively slowing down or even temporarily pausing frontier AI development, to enable societal structures and AI alignment research to keep up. The authors of the Anthropic post specifically cite the international regimes built to control past dangerous technology like nuclear weapons, which, for all their problems, have so far kept the world from annihilating itself. But those institutions, like the International Atomic Energy Agency, took decades of white-knuckling to build, and as the Anthropic leaders note, when it comes to self-improving AI: “We don’t have that long.”

How do we know when the world has changed?

Sometimes it’s immediate. When Otto Hahn and Fritz Strassmann achieved nuclear fission in December 1938, experts understood the implications almost immediately: a nuclear bomb would be possible. Sometimes the scientists see it and the rest of the world doesn’t. When Jennifer Doudna and Emmanuelle Charpentier published the seminal paper detailing CRISPR in 2012, initial press attention was all but nonexistent and the institutions that would eventually need to govern it had no idea what had just happened.

The hardest cases of all are the ones where even the experts can only see half of it. Fission pointed one way, toward a weapon, and the people who understood it could do little to stop it. Each of the two advances of last week point two ways at once. The same editing technology that could spare a child from a fatal disease is one that could eventually sort children into genetic castes. The same intelligence could give us “a country of geniuses in a data center,” as Amodei once put it, could also leave us as little more than spectators in the world.

So we are left where we began, at a threshold we cannot see past. The danger is not just that we may have walked through the wrong door. It is that we’ve walked through without noticing there was one.

A version of this story originally appeared in the Future Perfect newsletter. Sign up here!