At the end of last year, David Haynes, security engineer at Internet infrastructure company Cloudflare, faced a strange picture. "It was pure gibberish," he says. "A whole bunch of gray and black pixels created by a machine." He refused to share the image and said it was a security risk.
Haynes & # 39; caution was understandable. The image was created using a tool called Mayhem that examines software for unknown vulnerabilities. It was created by a startup called ForAllSecure, founded by Carnegie Mellon University. Haynes had tested it with cloudflare software that resized images to speed up websites, and supplied several sample photos. Mayhem mutated them into corrupt, cursed images that crashed the photo processing software by triggering an unnoticed error. This weakness could have given customers paying for Cloudflare a headache to keep their websites running smoothly.
Cloudflare has now made Mayhem a standard part of its security tools. The US Air Force, Navy and Army have also used it. Last month, the Pentagon ForAllSecure placed a $ 45 million order to expand Mayhem's deployment across the U.S. military. The department has many bugs to find. A government report from 2018 found that almost all weapon systems that the Department of Defense tested between 2012 and 2017 had serious software vulnerabilities.
Mayhem is not high enough to completely replace the work of human error finders who use knowledge of software design, code reading skills, creativity and intuition to find errors. According to David Brumley, co-founder and CEO of ForAllSecure, the tool can help human experts do more. The world's software has more vulnerabilities than experts can find, and more errors occur every minute. "Safety is not about being safe or insecure – it's about how fast you can move," says Brumley.
Mayhem emerged from an unusual 2016 hacking competition in a casino ballroom in Las Vegas. Hundreds of people came to follow the Cyber Grand Challenge hosted by Pentagon research agency DARPA. But there was hardly a person on stage, just seven brightly lit computer servers. Everyone hosted a bot that tried to find and exploit bugs on the other servers while finding and fixing their own bugs. After eight hours, Mayhem, manufactured by a team from Brumley's Carnegie Mellon security laboratory, won the top prize of $ 2 million. His magenta server ended up in the Smithsonian.
Brumley, who is still a professor at Carnegie Mellon, said experience had convinced him that the creation of his laboratory could be useful in the real world. He set aside the offensive capabilities of his team's bot, arguing with the defense was more important, and set about commercializing it. "The Cyber Grand Challenge has shown that fully autonomous security is possible," he says. "Computers can do a reasonably good job."
The governments of China and Israel thought so too. Both offered contracts, but ForAllSecure signed with Uncle Sam. It received a contract with the Defense Innovation Unit, a Pentagon group that tries to develop new technologies for the U.S. military.
ForAllSecure was asked to prove Mayhem by looking for bugs in the control software of a commercial airliner with a military variant used by the U.S. Armed Forces. The car hacker found a security vulnerability within minutes, which was then checked and corrected by the aircraft manufacturer.
Other bugs found by Mayhem are a bug discovered earlier this year in the OpenWRT software used in millions of network devices. Last fall, two interns at the company received a payout from Netflix’s bug bounty program after finding a bug in the software in Mayhem that allowed users to send videos from their phone to a TV.
According to Brumley, automotive and aviation companies are particularly interested. Cars and planes are increasingly relying on software that has to function reliably for years and is rarely, if ever, updated.
Mayhem only works with programs for Linux-based operating systems and finds errors in two ways: a scattershot and a more targeted one.
The first is a technique called fuzzing, in which the target software is bombarded with randomly generated inputs such as commands or photos and it is checked whether triggerable crashes occur. The second, called symbolic version, involves creating a simplified mathematical representation of the target software. This dilapidated double can be analyzed to identify potential weaknesses in the real target.
Fuzzing has been used increasingly in computer security in recent years. Last year, Google published a fuzzing tool that it claims has found more than 16,000 bugs in its Chrome browser. However, Cloudflare's Haynes says technology is still not widely used in industry because fuzzing tools usually require too much careful customization for each target program. ForAllSecure designed Mayhem to be more adaptable so Cloudflare can use fuzzing more routinely. The symbolic version can find more complex errors and has so far been used mainly in research laboratories, says Haynes.
People still necessary
Ruoyu Wang, a professor at Arizona State University, hopes Mayhem is just the beginning of an automated future for computer security, but he says that troubleshooting bots will be required to do more with people.
Mayhem shows that automation can do a useful job, Wang says, but existing auto-bug finders can't help much with complex Internet services or software packages. The best software is far from smart enough to understand the intent and functionality of programs as people do. Mayhem's ability to try many different things faster than anyone is no substitute. "Many of the difficult problems with automatically finding vulnerabilities are far from being solved," says Wang.
Wang was part of a team called Mechanical Phish that finished third in the 2016 DARPA tournament that allowed Mayhem to start. He is now working on a new research program by the agency called CHESS and is trying to develop a more powerful debugging software that uses people to help with things that machines cannot do. "At the moment, state-of-the-art automation does not know when it will encounter a barrier," said Wang. "It should recognize that and consult a person." Today Mayhem looks for mistakes himself, but his descendants can be team players.
This article first appeared on wired.com.