On Doing Nothing: Security in the AI Age

The history of humankind is the history of doing ‘something.’ As humans, we may not be very good at knowing the right thing to do at any given time, but we’re ever eager to do something. From the earliest days of pre-history, when Homo Sapiens fought and fucked Neanderthals into extinction, to the present, as we race toward the creation of a true Artificial Intelligence that will either destroy humankind or raise it to a new plane of existence, humans have always done something.

This drive to do something was, undoubtedly, a significant evolutionary advantage. It drove us to discover new, high calorie foods that enabled us to develop larger brains. It drove us to experiment with new tools and construction methods that enabled us to avoid predators by building larger, stable communities. It drove us to expand: first across regions, then continents. It drove us to conquer the sea and then the air. We are, in a word, an unsatisfied species. 

Why, then, if the strategy of doing something is so successful, are we going to talk about doing nothing? The answer to this is two-fold. The first answer, the big-picture answer, is that doing something is often only successful on a macro level. Anyone who has ever raised a baby knows that babies are, to put it bluntly, suicidally curious. This is a ‘macro-successful-strategy’ - being suicidally curious will help the species as a whole, but it will not help the babies throughout history who succeeded in being too curious for their own good.

The second answer is that it seems we may be entering a truly unprecedented period in history: a period when the cost of doing something is incredibly low and available to nearly every person. To be clear, we aren’t there yet, many endeavours still require physical ‘things’ and so require working capital. But a large class of things, by far the largest portion of the US economy, now have a very low cost of doing something. That class, as I’m sure you’ve guessed by now, is digital products or, more broadly, intellectual labour.

In fact, intellectual labour often doesn’t require much labour anymore. The newest LLMs can produce full video, audio, and write complete pieces of software with only the most incomplete suggestions. They require very little suggestion at all. You can tell Claude to “make me an app” and it will produce, well, something. If you have any domain knowledge at all, you can guide an LLM into producing a serviceable product for just about anything: chat apps, marketplace apps, dating apps, crypto apps. Practically speaking, the only bar to making something is a hundred dollars in tokens and a vague idea of what you want.

Now, this doesn’t mean what you make is good or that it will be successful. And anyone who uses LLMs will tell you this is something that they are terrible at: the Internet abounds with stories about Chat GPT telling people their idea to sell “shit on a stick” is a wonderful idea. Will this always be true? I don’t profess to know, but it is true now. The most important thing a human provides in modern software development is discernment: is the thing being proposed to do actually a good idea?

Now, before we embark on the next leg of the journey, it’s necessary to engage in some planning. My career and expertise, such as it is, are in security. I may be wrong on certain points, but I have a great deal of experience across a broad range of interrelated security disciplines: software architecture, network defense, penetration testing, analytics. It seems likely to me that the principles I am about to explore do generalize, but I can’t attest with certainty that they do. 

Now, to ‘doing nothing’ when everyone is doing something. 

One of the most important skills in one’s career is ‘triage’. It doesn’t matter whether you’re a soldier, a doctor, a teacher, or a security analyst; you cannot do everything, so you must decide what takes priority. While, at least to some of us, it would be nice to read every line of code in every library we use, and then read every line of code in every library in those libraries. At a certain point, we’ll simply have ‘read all the code’ and we can finally sit back and relax, knowing exactly where our vulnerabilities are.

This is, of course, impossible. And even if it were possible, it still wouldn’t solve the problem. There would still be myriad factors to analyze and edge cases to consider: doing good security is not securing everything against everyone at all times, for this is impossible. Doing good security is practicing discernment: assessing risks, analyzing likely attack vectors, and developing mitigations for the attacks that do get through. 

But this is all doing something. And, most of the time, we should be doing something, we should be following our natural human impulse to do something. But not always. Sometimes, the right choice is to do nothing.

If you look at it a certain way, this is obvious. Installing a piece of software usually does a lot of things: makes a registry key, writes some files, opens a port. Each one of those things represents a potential risk. That registry key could introduce a shim for a privilege escalation. That open port could allow a heap spray. Installing software is a calculated risk: with the exception of EDR, and not all of those, the attack surface of a machine always increases when you install a new piece of software. This is the reason installation of software in an enterprise environment is so tightly controlled. Vulnerable software makes not only the machine on which it is installed vulnerable, it makes the entire network vulnerable. 

This is hardly a novel insight, and I have no intention to present it as such. What I argue is that, with the advent of AI, we need to apply the same circumspection to new features in a piece of software that we do to the installation of software. Just as the correct decision is, often, not installing new software, the correct decision is often to not add a new feature.

For this analogy to cohere, we need to establish a few facts. First, we must acknowledge that when nearly every user installs software, they don’t really know what the computer is doing. In fact, unless you download from source and actually read the source, even the most technically savvy user is only ‘pretty sure’ they know what’s happening behind the scenes. And for the average user, they don’t have any idea. They click the icon. Press “next, next, next, finish,” and go. Bear this in mind, we’ll come back to it.  

Now, until very recently this simply wasn’t the case for software features. If you wanted to add a feature, someone who understood what was going on had to integrate it. This is not to say that things didn’t go wrong, as, well, the entire history of software development can attest, but adding a new feature still had a considerable cost in time and attention. Someone knew what was happening. The errors that arose were of a different fundamental category, they arose from errors in understanding, not a lack of understanding. 

The proof of this is fairly simple. No one would, under reasonable circumstances, introduce errors into their code that rendered it vulnerable or inoperable. The errors arise because they believe that such-and-such a thing is safe or correct. They read the docs, but misunderstood how an API call works. They used an out-of-date library. And so on. The point is that they apprehend, but they apprehend incorrectly.

But, now, in a matter of a year or two, software development has changed: it isn’t a matter of understanding incorrectly, it’s a matter of not understanding at all. It is, as far as I’m aware, impossible to get a real number how much code written today falls into each of: fully-human, AI-written & human-reviewed, full AI-written. There is a gulf between CEOs and CTOs touting 70, 80, 90 percent of code AI-written and actual practice, but the push to write more code, faster is undeniable. 

And the instinct isn’t simply managers and C-suites. Developers and coders of all stripes like writing code - for the most part - so when given the opportunity to ‘make more things’ they often take that opportunity and, well, make more things. But developers and coders of all stripes are also, at various paces, finding that they don’t even read the code an LLM produces: they’re doing  ‘next, next, next, finish’, but for writing software. They’re ‘doing something’, but not always thinking about if that ‘something’ should actually be done.

This, I think, is very clearly bad. It’s bad for all the reasons ‘next, next, next, finish’ is bad when installing software. It means we haven’t thought through all the implications. We haven’t thought about whether that new library is really necessary - because we didn’t have to read the docs. We haven’t thought about the downstream effects - because the LLM elided those for us. We didn’t consider other ways to solve the problem - because we didn’t solve the problem at all.

All this adds up to code we don’t understand, and, more importantly, has no level of human discernment applied to it; no one saying, “no wait, maybe we shouldn’t do that.” Many of us aren’t there yet. But if you’re writing software as a small business owner or as a personal project, both the risk and temptation are magnified: you want to produce something quickly and you don’t have anyone pushing back on your idea to slightly change the hue of every button based on the day of the week. An idea that, coincidentally, requires adding forty-nine new dependencies.

The result is that we’re going to need to think differently if we want to keep protecting our infrastructure. Part of that new paradigm is learning that, sometimes, the right choice is not doing something: not adding a new feature, not integrating a new library, not creating a new feature. This is true for the very same reason not installing a piece of software is often the correct choice. The fact that it was written by AI does not automatically make it secure and stable, it just makes it opaque.

Today, the most important skill a developer can have is not technical know-how. It’s not familiarity with any number of languages, or techniques, or compsci principles. It’s not people skills or time-management. It’s judgement. It’s knowing when to do ‘something’ - and when to do nothing.