A study from Stanford University highlighted the capabilities of an AI agent named ARTEMIS, which demonstrated superior performance in identifying security vulnerabilities compared to human hackers. Over a span of 16 hours, ARTEMIS scoured the university's network, which comprises approximately 8,000 devices. The AI managed to uncover nine valid vulnerabilities, achieving an impressive 82% valid submission rate.
Conducted by researchers Justin Lin, Eliot Jones, and Donovan Jasper, this experiment compared ARTEMIS with professional penetration testers, who each worked a minimum of 10 hours. The AI outperformed nine out of ten human participants in the initial ten-hour assessment. While human testers struggled with certain flaws, ARTEMIS was able to bypass obstacles that prevented human access, showcasing its unique operational advantages.
In terms of cost efficiency, running ARTEMIS is approximately $18 per hour, significantly lower than the average annual salary of $125,000 for professional testers. An advanced version of the AI agent is available at $59 per hour, maintaining a competitive edge over human experts. The study reveals the potential for AI to enhance cybersecurity measures effectively and economically.