Codex’s /goal feature amplifies bug hunting, but getting good results requires the right prompt, the right scope, and the right number of outcomes per run. For Patch the Planet, our joint initiative with OpenAI to find and fix bugs in open-source software, we pointed Codex at some of the most widely used, heavily audited codebases in the world, like Rust, curl, and zlib. One tool came up again and again in our internal bug-report channels: /goal, which hands Codex an open-ended objective and lets it work independently toward a success condition. Here are a few highlights: Found every Rust bug we submitted, including a soundness hole and a miscompilation now patched in Rust 1.98, from a single variant-analysis pipeline. Turned every project’s past CVEs into Semgrep rules that had to fire on the vulnerable version and stay silent on the patched one, then flagged 11 variant hits across multiple projects. Uncovered two potential high-severity privilege-escalation bugs in Keycloak’s SAML component during a discovery run. Over the first few weeks of Patch the Planet, our engineers independently converged on three techniques for using /goal. We found that getting the most out of /goal means treating the prompt as a set of specific success criteria, not a set of instructions. (Note that this blog post uses /goal to refer to goal-based prompting in general. Codex can also set goals for itself through a tool call, and that’s how we recommend everyone use it; we rarely type the slash command ourselves.) 1. Let Codex write the goal The art of using /goal is prompt design, and we found that Codex knows Codex the best. Internally, our single most repeated /goal tip was to use Codex to help write each /goal prompt. We hand Codex threat model files and the context about what we’re looking for, and then tell it to write the goal prompt. As mentioned before, /goal is a tool Codex can invoke on itself, and a few engineers stopped typing goals by hand entirely. $goal-

Read Full Article at Trail of Bits Blog →