LLM Agents for security research

I recommend having the LLMs spawn "validator" agents to check whether the vulnerabilities it finds are real and exploitable. This will save you a ton of time running down false positives. My workflow is to have it spawn tons of agents to fan out and document the critical code paths. Then spawn "hunters" to look for specific classes of bugs along the code paths and document them. Validators should run with the hunters to check their claims. Finally, I go through the claims and look for ones that seem most promising from a bug bounty perspective.

If you're essentially looking for a LLM to hack for you, your mileage may vary. You'll have to be very creative with your prompts to get around the guardrails.

If you are in the trusted access program (which is very easy to get into if you are willing to submit your ID), you can get ChatGPT/Codex to generate PoCs. Even if you're in Anthropic's Cyber Verification Program, Claude simply does not like to hack - you can avoid the guardrails, but eventually it will shut things down and cost you a lot of time. I don't need or use anything other than Codex for this (and Ghidra/Binary Ninja if source isn't available). I will caveat I haven't done any AI research since 5.6 Sol was released. I know it has a few extra guardrails vs 5.5, so it's possible that it makes life a little harder. I can't say for certain.

The validator step works best if you force it to prove reachability instead of just re-reviewing the code, since the usual false positive is calling something exploitable when the path to reach it never actually fires. Make the validator produce a PoC or an execution trace that triggers the bug, and drop anything that can't.

I mean this respectfully, but you are asking the wrong questions. “Is there a pipeline to improve the quality of AIs work in cybersecurity?” I mean, yes, and it’s up to you to figure it out. That’s called engineering. There is no one answer. It depends on your unique needs, and you’ll need to use your brain to make the decisions yourself. The people doing this effectively are not going to guide a script kiddie through it step by step in the Reddit comments. They're not going to point you to a public github repo and say "here just run this command and your AI will find all the bugs, no slop!" It’s like asking someone “is there a way to write code to ship faster with fewer bugs?” The answer is obviously yes, and it’s a really complicated answer that separates the people at the top of the field from the people at the bottom. In the case of using AI for cybersecurity, the answer pretty much changes every 30 days. In fewer words, it’s up to you to figure out the answer to the questions you’re asking. Just pick an agent harness and start trying things and see what works for you and what doesn't. Once you have a little more background, start consuming content from other people doing the same thing and see if you get any inspiration for what to do next to improve your workflow.

this is actually the kind of thing LLMs are decent at right now, like orchestration and reconciliation instead of "find 0day in this blob of code". curious how you validated it vs the commercial SAST though, did you run them in parallel for a few releases and diff the findings or just spot check on a few known-bad repos?

If you're asking llm models, there are many llm models you could find on hugging face which have lesser guardrails so you could use them for offensive security, you can also find models specifically trained for these offensive security purposes. You should check out on hugging face there are many local models or open weights model families.

Are you looking for poc development for exploits through patches or available vendor info with no available poc's? Just curious, can you give an example of hallucinations you're facing with the LLM and what's the task given?

did a mcp server that run a bunch of security scans (secrets, sca, sast,..) with some graph analysis. It will do triage by removing false positives, prioritize with a remediation plan, build pull request with regression test. It works with claude but it could work with any harnesses compatible with skills/commands https://github.com/kubeek-sec/deep-scan

It's fable5/mythos and it's not even close but you need to work somewhere that you can have security research access, otherwise you'll get results that you've been seeing.

I had Claude build a SAST that runs multiple scans from different sources and compares the results. Based on the results I think we'll be able to drop our commercial SAST tool. README: A dual-scanner SCA (software composition analysis) workflow for GSMS release builds. It generates a Software Bill of Materials, scans third-party dependencies for known CVEs with two independent engines, reconciles their findings, adjudicates disagreements with a documented reason per rejection, and produces a customer-ready Consolidated Vulnerability Report PDF. ## Tools used Upstream scanners (run separately; these scripts consume their output): - **Syft** — generates the SBOM for each build in CycloneDX (`sbom.cyclonedx.json`) and native Syft JSON (`sbom.syft.json`). The CycloneDX output also satisfies external SBOM requests directly (e.g. the REC IT TPRM questionnaire). - **Grype** — scans the SBOM against GHSA + NVD CPE data, enriched with CISA KEV and EPSS. Output: `grype-results-cpe.json`. Reaches nested/shaded jars via the SBOM. - **OWASP Dependency-Check** — the existing SCA scan. Output: `dependency-check-report.csv` (plus the legacy HTML report). Runs against the same artifacts by CPE. Each scanner has a blind spot the other covers: Grype misses jars whose Maven coordinates can't be recovered from the manifest; Dependency-Check over-attributes CVEs on CPE name collisions. Cross-referencing the two closes both gaps.

Yeah hallucination's still the biggest headache with all of these, even Opus/Claude Code tbh. Stuff that's helped me cut down false positives: Never trust a single pass. Run it twice with different prompts (find vulns vs explain what the function does) and cross check. If the vuln doesn't show up in the second pass, it's probably just hallucinating. Feed it actual tool output instead of asking it to find vulns cold. Give it semgrep/codeql results and have it prioritize/explain those instead. Way fewer false positives that way. For CVE/0-day stuff specifically, models are decent at spotting patterns (bad deserialization, missing auth checks etc) but pretty bad at judging if it's actually exploitable without a running env. Always verify reachability yourself. Kimi/Deepseek giving me similar results to Claude lately too, but honestly same issue everywhere. none of them are reliable enough to skip manual verification yet. basically treat it as a fast first pass triage tool not ground truth. saves time filtering noise but you still need a human for the final call

Ive been using the Microsoft copilot 'researcher' agent recently. Give it a good enough prompt and as long as you are prepared to wait 10-15mins for it to finish, the results are always impressive