Why Security Comes First

Preventing AI technology from being stolen and tampered with is the highest AI policy priority.

I’ve been talking a lot about the need for security improvements at frontier AI companies lately. Since this is a bit of a departure from what I’ve worked on and emphasized historically, I wanted to more fully why I’ve come to focus on this as one of my top few policy priorities.

I’ve covered some of these points in previous blog posts and tweets but wanted to put them all in one place.

For the purpose of this blog post, I define security simply as the inability to be stolen or tampered with, and safety as the absence of accidents, though that’s a bit of a simplification.

Timing disconnect

We are arguably already way behind in terms of security relative to the importance of AI as a technology. Independent experts, drawing on interviews with staff at frontier AI companies, think that such companies are years from being able to defend AI technologies against well-resourced attacks by countries like China and Russia. And that’s if we try hard.

I hope that, as with people saying in 2020 that it’d take many years to make COVID vaccines, the experts are wrong here and we can move faster than current estimates with a concerted effort. But in any case we need to get started ASAP, and I don’t have a very solid basis for my hope.

It’s important to bear in mind that AI systems a few years from now, or even one year from now, will be much more capable. AI companies already have AI systems that are better at competitive coding tasks than all but a few hundred humans, and much more progress is expected soon. I don’t want stuff like that being stolen and abused by whoever happens to good at hacking, espionage, etc.

One calendar year is a long time in “AI years,” so the current situation is just unreasonable at a societal level (which, to be clear, I think most involved would agree with, but it hasn’t been solved yet for the incentive reasons I discuss below).

Interstellar (Paramount Pictures)

Conceptual precedence

In some sense, security takes priority for a technology that can be copied and that is very general-purpose (and thus amenable to all sorts of misuse).

You can’t govern a technology if it won’t sit in one place, and you can’t make that technology controllable if random bad actors are tampering with it, poisoning its data, etc.

Nor can you ensure good safety practices, that may require a lot of care and discussion to get right, if you don’t know who you should even be talking to.

Sometimes it’s good to give everyone access to a technology. But that will not always be a good idea, and we shouldn’t be happy with a situation in which we don’t even have the option of fully protecting any AI IP.

“Don’t Copy that Floppy,” Software Publishers Association

Incentive problems

Unless and until governments, civil society, and/or the public put strong pressure on frontier AI companies to protect their technology rigorously, they will not really pay the full costs of not doing so, and they will have a disincentive to get far ahead of everyone else on security if doing so might slow them down.

Consider theft of American AI IP by Russian or Chinese hackers, for example. A company with lax security may not even find out about this, and if they did, it may not be made public, making it difficult for them to really be penalized for it. And while the costs of investing in security are largely borne by the company, the costs of under-investing are borne by a much larger set of actors, e.g. perhaps the country or world as a whole.

Lastly, acting unilaterally to push super hard on security could slow down research and product progress and give others a leg up in the market. So we should not be expecting this to get solved automatically.

While sorting these incentive issues out is tricky, I think it’s clearly in the national and global interest to do so. While in some cases there will be delays and costs as a result of taking security more seriously, doing so would, overall, help sustain the US’s leadership position in AI and ultimately improve safety and governance outcomes.

(There are incentives to cut corners on safety sometimes, as well as incentives not to do so, which is also true of security. I can’t really do the comparison justice here but the bottom line is that in my view, security is the less well-incentivized one, at least at the standard we should be aiming for.)

Conclusion

Increasing the incentive and technical ability of AI companies to have good security is a very high priority, and in my view even more urgent than safety (though that’s also very important).

While I don’t love the idea of leaving safety until later and in reality we should just be multi-tasking, it’s worth noting that AI safety research can more plausibly be automated at later stages than (some aspects of) security can. AI system N, if safe-ish, can be used to help make AI system N+1 safer through (e.g.) better designing its training process. But building secure chips and datacenters, sorting out security clearances, etc. will inherently take a fair amount of time — more time than the gap between AI model generations. Many parts of security involve atoms (e.g., requiring different chip features and datacenter designs), whereas safety is more of a bits-related problem, and it’s easier to speed up improvements to bits than to atoms.

Security comes first, and should be pursued in a coordinated way so that no one has to put themselves at a disadvantage by making the first move. I hope that this will be one of the highest priorities of the incoming Trump administration.

For further reading on this topic, you might want to check out this long RAND report cited above, or this short version of it, and this report on technical challenges related to security (and other topics) that could benefit from more research.