Focus on AI: Vibe Coding - Trust the Vibe but Verify the Code

ArticleLegal TechnologyLegal InnovationsArtificial IntelligenceAITechnology Trends
Written By Don Philmlee

Rise of Vibe Coding

Artificial intelligence is rapidly changing how software is developed. What started as little more than intelligent autocomplete has evolved into AI systems capable of writing applications, creating files, debugging software, running tests, interpreting errors, and correcting their own work. Increasingly, developers can describe what they want and allow an AI system to figure out how to build it.

This new approach is often called vibe coding, and it represents a significant change in the relationship between people and software.

Instead of writing every function and understanding every line of code, developers can increasingly act as managers of AI coding systems. More significantly, people who are not software developers can now create working applications simply by explaining what they want. This dramatically lowers the barrier to creating software. Unfortunately, it also lowers the barrier to creating insecure software.

The cybersecurity challenge may not be that artificial intelligence creates entirely new vulnerabilities. The larger problem is that AI allows us to create software, vulnerabilities and all, at a scale we have never experienced before.

From Coding Assistant to AI Developer

AI-assisted software development has evolved remarkably quickly. The first generation of AI coding tools primarily helped developers write code faster. A developer might begin writing a function and an AI assistant would suggest how to complete it. Developers could also ask an AI system how to use an unfamiliar API or paste an error message into a chatbot and ask what went wrong.

Today's systems can do considerably more. Modern coding assistants can modify multiple files, create new components, run applications, examine console errors and make additional changes based on the results. More autonomous development agents can be given a broader objective and then research, develop, test and modify software with progressively less human involvement.

This creates a fundamental change in the development process. The developer is moving from writing the implementation toward describing the desired outcome.

That can provide an enormous productivity advantage. It also creates an interesting security problem: the person responsible for the software may increasingly have little understanding of the software's underlying code.

What Exactly Is Vibe Coding?

Vibe coding takes AI-assisted development another step. Instead of carefully reviewing each piece of generated code, the user describes what the application should do and allows the AI to build it. The user evaluates whether the application works rather than necessarily evaluating how it works.

For prototypes and temporary applications, this can be extremely useful. A product manager could create a functioning demonstration instead of a PowerPoint mockup. An analyst could create an internal application to automate a repetitive task. A security professional could quickly create a tool to analyze logs or process vulnerability data.

However, what happens when the prototype becomes useful and someone connects it to a database. Another person adds authentication and someone else connects it to a cloud service. Soon the disposable prototype is handling production data. The application may work perfectly, but that does not mean it is secure.

The Human Review Problem

For years, software security has depended partly on knowledgeable developers understanding the code they create and reviewing the code created by others. AI complicates that model.

Reviewing ten lines of AI-generated code is reasonable. Reviewing hundreds or thousands of lines generated across dozens of files becomes considerably more difficult. The problem becomes even greater when the person directing the AI isn't a programmer. A user may be perfectly capable of determining whether an application performs the desired business function while having no ability to recognize SQL injection, insecure authentication, excessive privileges, improper secrets management or a vulnerable software dependency.

Even experienced developers face another problem: omissions can be difficult to review. It is relatively easy to identify something incorrect that appears in code. It is much harder to identify a security control that should have been there but was never created. So many problems could exist. Was authorization implemented everywhere it should have been? Is user input properly validated or are API credentials being stored securely? Are sessions handled correctly? Are database permissions unnecessarily broad? The application can pass its functional tests while still failing these security requirements.

Slopsquatting and the Software Supply Chain

One particularly interesting AI-related software supply-chain risk has become known as slopsquatting. Traditional typosquatting occurs when an attacker creates a malicious software package with a name similar to a legitimate package. A developer mistypes the package name and accidentally installs the attacker's version.

Slopsquatting takes advantage of AI hallucinations instead. Large language models occasionally invent software packages that do not exist. The generated package name may sound completely legitimate, and the AI may confidently generate code that depends upon it. More importantly, some hallucinated package names can be repeated. Similar prompts can cause an AI model to invent the same nonexistent package.

An attacker could potentially identify these predictable names, register the packages in public repositories and populate them with malicious code. The next time an AI coding tool recommends that nonexistent dependency, it now exists and it belongs to the attacker.

Traditional code review provides surprisingly little protection against this type of attack. A developer examining generated code might see a reasonable-looking package name and assume that the AI selected an appropriate dependency.

The better solution is to control the software supply chain itself. Organizations should restrict applications to approved package repositories, maintain dependency inventories, perform automated software composition analysis (SCA), monitor package provenance and vulnerabilities, and establish policies governing which external components can enter the development environment. AI should not be ab le to choose what the organization trusts.

From Cloud Sprawl to App Sprawl

There is an interesting precedent for what may happen next. Cloud computing dramatically reduced the effort required to create infrastructure. Before cloud computing, deploying another server required hardware, purchasing, installation and configuration. Eventually, infrastructure could be created with a few clicks or an API call.

That convenience transformed computing. It also created cloud sprawl. Organizations suddenly had virtual machines, storage repositories, databases and cloud services scattered throughout their environments. Security teams sometimes didn't know what existed, who owned it, why it had been created or whether it was still needed.

AI development may create the application equivalent. Call it app sprawl. When building an application required a development team and weeks or months of effort, there was natural friction limiting how many applications an organization could create.

Vibe coding removes much of that friction. An employee may soon be able to create an application during lunch, connect it to corporate data in the afternoon and share it with the department before going home.

The immediate application may not even be the biggest concern. What happens two years later when the employee has left the organization, the application's dependencies haven't been updated and nobody remembers that the application still has access to a corporate database? Software that is easy to create is also easy to forget.

Old Vulnerabilities at Machine Speed

The interesting thing about AI-generated software is that many of its cybersecurity problems aren't new. Applications have always suffered from insecure dependencies, poor authentication, excessive permissions, vulnerable code, exposed credentials, bad configurations and inadequate maintenance. AI does not necessarily change those vulnerabilities. It changes their volume and velocity.

If an organization previously created 50 applications each year and AI enables employees and developers to create 500, even maintaining the same vulnerability rate produces a dramatically larger attack surface. Traditional security processes cannot simply scale by adding more people to review more code.

Security controls must increasingly become automated components of the development environment. Static application security testing (SAST), dynamic application security testing (DAST), software composition analysis, secrets detection, dependency management, container scanning and infrastructure-as-code analysis should become automated development pipeline controls rather than optional activities performed at the end of a project. Just as importantly, organizations need to know what applications exist. An application nobody knows about cannot be patched, monitored or retired.

Zero Trust for AI-Generated Code

The basic security principle should sound familiar: never trust, always verify. Organizations should treat AI-generated code exactly as they should treat any other untrusted input. The fact that an AI system generated code does not make the code secure. It also does not necessarily make it insecure. What matters is whether the software has passed the organization's security controls. AI-generated code should therefore move through defined security gates before reaching production.

Dependencies should be verified. Code should be scanned. Credentials should be protected. Infrastructure configurations should be evaluated. Access should follow least-privilege principles. Applications should be inventoried, assigned an owner and continuously monitored. This changes the role of security from reviewing how software was written to verifying what software is allowed to do.

When the Code Stops Mattering

There may be an even larger change coming. If AI becomes sufficiently good at generating software, the source code itself may eventually become less important to humans. Instead, the critical artifacts may become the requirements and the tests.

A human defines what an application must do. Security requirements define what it must not do. Automated tests verify that the application behaves correctly. The AI generates an implementation.

If the implementation fails, the AI generates another. In such an environment, code could eventually become disposable. This would fundamentally change software development. Developers would spend less time describing implementation details and more time defining requirements, architecture, security boundaries and verification criteria.

Security requirements would therefore need to become first-class development requirements. "Create a customer portal" would not be sufficient. The specification would also need to define authentication, authorization, encryption, logging, data retention, approved dependencies, input validation, privacy requirements and other security controls. The quality of the specification may eventually matter more than the quality of the prompt.

Where is this all going?

Vibe coding and agentic development are still evolving, but their direction is becoming clear. Software is becoming easier to create, and more people will be able to create it. That is a tremendous opportunity. It is also a cybersecurity problem.

The answer is not to prohibit AI-generated software. Organizations tried similar approaches with cloud computing, mobile devices and other disruptive technologies. When technology provides enough business value, users generally find ways to use it. Instead, organizations need to build security around the assumption that AI-generated software will exist.

Applications need identities and owners. Dependencies need verification. Security testing needs automation. Production deployment needs controlled pipelines. Access needs to follow least privilege. Applications need inventories and lifecycle management. And abandoned applications need to be discovered and removed.

Most importantly, security cannot depend upon someone carefully reading everything an AI produces. We are moving from a world where humans write software to one where humans increasingly define the rules under which software is created.

That distinction matters because AI may soon be able to create software faster than we can possibly review it. The cybersecurity challenge will be making sure it cannot create risk faster than we can control it.

#

References:

Don Philmlee

Entrepreneur, consultant, strategist and technology advocate - Don has years of experience making technology work. He is an author, teacher, and mentor on security trends and technology implementation. Over the past 30-plus years, his clients have included hundreds of law firms, associations, government and other organizations in the United States and overseas. Don is a certified Project Management Professional (PMP) since 2006 and a Certified Information Systems Security Professional (CISSP) since 2003.

Previous

Focus on AI: When AI Breaks the Rules - Can we ever control it?
Next

Team-Building is Critical to a Successful Project