Code coverage is one of the most dangerous quality metrics in software testing. Many developers fail to realize that code coverage lies by omission: it measures execution, not verification. Test suites with high coverage can obfuscate the fact that critical functionality is untested as software develops over time. We saw this when mutation testing uncovered a high-severity Arkis protocol vulnerability, overlooked by coverage metrics, that would have allowed attackers to drain funds. Today, we’re announcing MuTON and mewt, two new mutation testing tools optimized for agentic use, along with a configuration optimization skill to help agents set up campaigns efficiently. MuTON provides first-class support for TON blockchain languages (FunC, Tolk, and Tact), while mewt is the language-agnostic core that also supports Solidity, Rust, Go, and more. The goal of mutation testing is to systematically introduce bugs (mutants) and check if your tests catch them, flagging hot spots where code is insufficiently tested. However, mutation testing tools have historically been slow and language-specific. MuTON and mewt are built to change that. To understand how, it helps to first understand what they’re replacing. The regex era Mutation testing dates to the 1970s, but for a long time, the technique rarely saw much adoption in the blockchain space as a software quality measurement. Testing frameworks are coupled tightly to target languages, making support for new languages expensive. Universalmutator changed this with its regex engine. After a commit on March 10, 2018 added Solidity support, the tool gained immediate traction in the blockchain space. We collaborated with the universalmutator team to advance smart contract testing and highlighted the tool in our 2019 blog post. Despite (or perhaps because of) its elegant approach and compact codebase, universalmutator generated impressive mutant counts, enabling developers to assess test coverage more thoroughly than simpler tools co

Read Full Article at Trail of Bits Blog →