AI | CO-PILOT | CYBERSECURITY | FLYING MOM | PAMC
The Hacker News article discusses the security challenges and vulnerabilities associated with AI-based software development tools like GitHub Copilot. It highlights the paradox that while AI can improve coding productivity and security, it can also be vulnerable to attacks if not managed carefully. The article highlights the importance of secure coding practices, ongoing security assessments, and the need for developers to remain vigilant and informed of potential vulnerabilities. Copilot’s tendency to replicate vulnerabilities in existing code highlights the need for developers to critically evaluate AI-generated code.
Imagine a world in which the software that powers your favorite apps, secures your online transactions, and manages your digital life could be outwitted and taken over by a cleverly disguised piece of code. This isn’t a plot from the latest cyber-thriller; this has actually been a reality for years now. How this will change – in a positive or negative sense – as artificial intelligence (AI) plays a larger role in software development is one of the big uncertainties of this brave new world.
At a time when AI promises to revolutionize the way we live and work, the discussion about its security implications cannot be put aside. As we increasingly rely on AI for tasks ranging from the most mundane to the most critical, the question is no longer simply: “Can AI strengthen cybersecurity?” (of course!), but also “Can AI be hacked?» (yes!), « Can we use AI To hack?» (of course!) and “Will AI produce secure software?” (Good…). This thought leadership article is about the latter. Cydrill (a secure coding training company) delves into the complex landscape of AI-produced vulnerabilities, with a particular focus on GitHub Copilot, to highlight the imperative for secure coding practices to safeguard our digital future.
You can test your secure coding skills with this short film self evaluation.
AI’s shift from an academic curiosity to a cornerstone of modern innovation happened quite suddenly. Its applications span a breathtaking range of fields, offering solutions that were once the stuff of science fiction. However, this rapid progress and adoption has outpaced the development of corresponding security measures, leaving AI systems and AI-created systems vulnerable to various sophisticated attacks. Already seen? The same things happened when software – as such – invaded many areas of our lives…
At the heart of many AI systems is machine learning, a technology that relies on large data sets to “learn” and make decisions. Ironically, AI’s strength – its ability to process and generalize large amounts of data – is also its Achilles heel. The starting point for “everything we find on the Internet” may not be perfect training data; unfortunately, the wisdom of masses maybe not enough in this case. Additionally, hackers, armed with the appropriate tools and knowledge, can manipulate this data to trick AI into making incorrect decisions or taking malicious actions.
GitHub Co-pilotpowered by OpenAI Manuscript, demonstrates the potential of AI in coding. It was designed to improve productivity by suggesting code snippets and even entire blocks of code. However, several studies have highlighted the dangers of relying entirely on this technology. It has been shown that a significant portion of the code generated by Copilot may contain security vulnerabilities, including vulnerabilities to common attacks such as SQL injection and buffer overflows.
The principle “Garbage In, Garbage Out” (GIGO) is particularly relevant here. AI models, including Copilot, are trained on existing data and, like any other extended language model, most of this training is unsupervised. If this training data is wrong (which is very possible given that it comes from open source projects or large Q&A sites like Stack Overflow), the result, including code suggestions, can inherit and propagate these defects. In the early days of Copilot, a study found that about 40% of the code samples produced by Copilot when asked to complete code based on samples from the CWE Top 25 were vulnerable, which highlights the GIGO principle and the need for increased security awareness. A larger-scale study in 2023 (Is GitHub Copilot as bad as humans at introducing vulnerabilities into code?) had slightly better results, but still far from good: by removing the vulnerable line of code from the real vulnerability examples and asking Copilot to complete it, it recreated the vulnerability about 1/3 of the time and fixed the vulnerability only about 1 time. /4 of the time. Additionally, it performed very poorly on vulnerabilities related to missing input validation, producing vulnerable code every time. This shows that generative AI is ill-equipped to handle malicious input if “silver bullet” solutions to handle a vulnerability (e.g., prepared statements) are not available.
Addressing the security challenges posed by AI and tools like Copilot requires a multifaceted approach:
- Understanding vulnerabilities: It is essential to recognize that AI-generated code can be susceptible to the same types of attacks as “traditionally” developed software.
- Improve secure coding practices: Developers should be trained in secure coding practices, taking into account the nuances of AI-generated code. This involves not only identifying potential vulnerabilities, but also understanding the mechanisms by which AI suggests certain code snippets, in order to effectively anticipate and mitigate risks.
- Adaptation of the SDLC: It’s not just a question of technology. Processes must also take into account the subtle changes that AI will bring. When it comes to Copilot, code development is usually the center of attention. But requirements, design, maintenance, testing and operations can also advantage from large language models.
- Continuous vigilance and improvement: AI systems – just like the tools they power – are continually evolving. Keeping up with this evolution requires staying informed with the latest security research, understanding emerging vulnerabilities, and updating existing security practices accordingly.
Navigating the integration of AI tools like GitHub Copilot into the software development process is risky and requires not only a change in mindset, but also the adoption of robust technical strategies and solutions to mitigate potential vulnerabilities. Here are some practical tips designed to help developers ensure that their use of Copilot and similar AI-powered tools improves productivity without compromising security.
Practical implementation: Defensive programming is still at the heart of secure coding. When accepting Copilot code suggestions, especially for functions handling user input, implement strict input validation measures. Define rules for user input, create a allowlist of allowed characters and data formats, and ensure input is validated before processing. You can also ask Copilot to do this for you; Sometimes It actually works well!
Practical implementation: Copilot may suggest adding dependencies to your project, and attackers can use it to implement supply chain attacks via “packet hallucinations”. Before incorporating suggested libraries, manually verify their security status by searching databases such as the National Vulnerability Database (NVD) for known vulnerabilities or perform a Software Composition Analysis (SCA) with tools such as OWASP Dependency-Check or an npm audit for Node.js projects. . These tools can automatically track and manage dependency security.
Practical implementation: Regardless of the source of the code, whether AI-generated or hand-crafted, conduct regular code reviews and testing with a focus on security. Combine approaches. Test statically (SAST) and dynamically (DAST), perform software composition analysis (SCA). Perform manual testing and supplement it with automation. But remember to put people before tools: no tool or artificial intelligence can replace natural (human) intelligence.
Practical implementation: First, let Copilot write your comments or debug logs – it’s already pretty good in those. Any errors in these elements will not affect the security of your code anyway. Then, once you’re familiar with how it works, you can gradually let it generate more and more code snippets for the actual functionality.
Practical implementation: Never blindly accept what Copilot suggests. Remember you are the pilot, it’s “just” the Co-pilot! You and Copilot can form a very effective team together, but you’re still in charge, so you need to know what the expected code is and what the result should look like.
Practical implementation: Try different things and prompts (in chat mode). Try asking Copilot to refine the code if you’re not happy with what you got. Try to understand how Copilot “thinks” in certain situations and realize its strengths and weaknesses. Plus, Copilot improves over time – so keep experimenting!
Practical implementation: Always keep you and your team informed about the latest security threats and best practices. Follow security blogs, attend webinars and workshops, and participate in forums dedicated to secure coding. Knowledge is a powerful tool for identifying and mitigating potential vulnerabilities in code, whether AI-generated or not.
The importance of secure coding practices has never been more important as we navigate the uncharted waters of AI-generated code. Tools like GitHub Copilot present significant opportunities for growth and improvement, but also unique challenges when it comes to keeping your code secure. Only by understanding these risks can we successfully balance efficiency and security and protect our infrastructure and data. In this journey, Cydrill remains committed to providing developers with the knowledge and tools needed to build a more secure digital future.
Cydrill’s blended learning journey provides proactive and effective secure coding training for developers at Fortune 500 companies around the world. By combining instructor-led training, e-learning, practical laboratories and gamification, Cydrill offers a new and effective approach to learning to code safely.
Check Cydrill’s secure coding courses.
Did you find this article interesting ? This article is a contribution from one of our valued partners. follow us on Twitter And LinkedIn to read more exclusive content we publish.
