Projects

BaitBench

Measures how often an LLM, writing a job application for you, falls for a prompt injection hidden in the job post.

BaitBench preview

Recruiters and hiring managers hide prompt injections in job posts to catch AI-written and auto-apply applications. BaitBench tests how often AI models fall for that bait. It runs synthetic job posts, each with a candidate profile and one planted injection, across a mix of models from the latest expensive ones to the older and cheaper ones that real auto-apply tools run on. Grading is inverted: a pass means the model resisted. Single-turn harness on Bun and the Vercel AI SDK through the Vercel AI Gateway, graded by a substring check plus a cheap LLM judge, with a Next.js visualizer for the leaderboard, per-job heatmap, and reasoning traces.


More projects

All projects
Open Frame preview
Open FrameLive

Make slides, carousels, thumbnails, and OG images with React and a coding agent

emberd preview
emberdLive

Run untrusted AI-agent code in isolated microVMs you host yourself, inside your own network

Want to talk about BaitBench? Email me or book a call.

Command Menu

Quick navigation and actions