Noah Airmet
I’m working toward a career securing and governing AI systems.
- Development and AI SecuritySimplicity Group
Pen test remediation, SOC 2 compliance, AI vendor risk
- Lead Technical Fellow@transfigurism
- Cybersecurity, class of 2028BYU
CitationBench
Agents cite sources in everything they write now. Give one a real library and it stops inventing them, but they still misread the source at an alarming rate. On real citations from published papers the agents we ran were right 40 to 67% of the time.
AI risk management is a key component of responsible development and use of AI systems.
- gemini-3.8-flash-highAntigravity66±13
- gpt-6.1-solCodex60±14
- claude-haiku-5-5Claude Code41±11
- gpt-6-lunaCodex40±15
Catchword
AI safety guidance comes out faster than anybody can read it, and agents are the ones quoting it now. Catchword is your entire library of safety documentation and AI policy writing, built to help you stay on top of it and in the loop.
A chatbot with web search can find many of the same documents, but Catchword is built for professionals because:
- Cited to the pageA citation opens the exact page, with the quoted words marked.
- Quotes checkedEach quote is compared word for word with the page it cites.
- Every version keptDrafts and revisions stay separate sources. Ten versions of Anthropic’s Responsible Scaling Policy.
- One library13,898 documents: papers, system cards, safety frameworks, government guidance, evaluator reports.

What’s NIST’s guidance on red-teaming generative AI?↵
NIST AI 600-1, the Generative AI Profile, July 2024:
For best results, AI red teams should demonstrate domain expertise, and awareness of socio-cultural aspects within the deployment context.
Which paper showed that models can hide their capabilities during testing?↵
“AI Sandbagging: Language Models can Strategically Underperform on Evaluations,” van der Weij et al., ICLR 2025:
we find that models can be fine-tuned, on a synthetic dataset, to hide specific capabilities unless given a password.
What is the “evidence dilemma” in the International AI Safety Report?↵
International AI Safety Report 2025:
pre-emptive risk mitigation measures based on limited evidence might turn out to be ineffective or unnecessary. On the other hand, waiting for stronger evidence of impending risk could leave society unprepared or even make mitigation impossible