RE: Everything is About to 'Go Dark'
Thank you, Matthew, for sharing your thoughts on this matter. Indeed, it is a very interesting time keeping up with the latest LLMs from a security perspective. I do not share the same beliefs and I want to take a moment to evaluate.
As a programmer, I have observed that product teams are incentivized to deploy new features or change product scope. They care about the user objective in our sprint and cadence being deployed, and how quickly it can be shipped. The product organization of a business rarely faces repercussions if the product leaks customer data or has vulnerabilities. This is seen as a technical problem and table stakes for an engineering organization. Yet when engineers advocate for more time to fix bugs, review vulnerabilities, and update penetration tests, there is always pushback.
Security comes from the CTO, CISO, or Risk functions of organizations. Without effective standards, policies, and procedures in place, this process is left to engineering managers to sort out the best they can. EMs do not get promoted for secure apps, and they don’t get demoted or fired for insecure apps or even worse, active compromises. On top of this, detecting active compromises is sufficiently challenging. Many companies do not have effective monitoring in place to analyze just how a vulnerability leads to compromise.
The reason these LLMs will not fix all the security vulnerabilities in these applications is because the LLMs were trained on human written code and behaviors. LLMs will fail for the exact same reasons that humans fail to secure their code at scale. They are sycophantic and context-aware. It does not take long for security to fall out of the context window as AI-driven engineers set scopes and goals for the next iterations of code. They will prioritize shipping the new feature over security.
Let’s say for argument’s sake you have an engineer who does prioritize security review as part of their pipeline. They have a fully secure automated build pipeline using SAST, integration testing, regression testing, library and dependency scans, and finally sign off from a CISO. The CISO has instituted an agent as part of the security review that prioritizes looking at the architecture for vulnerabilities. The whole chain is still just one effective phishing campaign away from a peer on the marketing team clicking the wrong email.
We also cannot ignore the incredible rate at which codebases are growing in 2026 due to LLM use. It would not surprise me if the amount of application source code in existence increases tenfold in the next 18 to 24 months. Unsupervised agents compound the challenge and now there are vast amounts of code to secure which may not even be reviewed by humans.
Believe me, I wish for a time on the web where the security posture of the collective space is much more secure than it is today. I desire this because the people of the world deserve private and secure communications. I try to avoid companies unable to secure my data like the plague, but even with my understanding of deploying secure applications, I’m not much better than the average person at doing so.
The reason the world will not “Go Dark” is because the environment in which we deploy applications does not reward anyone outside of red teams for doing so securely. The world will not be going dark because we have not “solved” secure computing. We only mitigate it as much as is affordable in budget and resource time, whether managed by humans or models.