About the Role
Cybersecurity Benchmark Engineer
Location: Remote
Company: Pilotcrew AI
Type: Contract (monthly basis)
Experience: 3+ yrs
About Pilotcrew AI
Pilotcrew AI builds infrastructure for AI Agent Evaluation. We benchmark large language models, run automated agent evaluations, power human-in-the-loop assessments, and host AI arenas for competitive testing. Our mission is to make AI agents measurable, reliable, and production-ready through structured, scalable evaluation systems.
Role Overview
We are building a large-scale benchmark for evaluating the cybersecurity capabilities of frontier AI LLMs. To grow this benchmark, we need hands-on security engineers who can craft real-world vulnerability tasks that are genuinely difficult for state-of-the-art LLMs and agentic systems.
Your core output: carefully designed benchmark instances real software vulnerabilities paired with well-formed task speci...
Ready to Join Through a Referral?
Apply now and get connected directly with the hiring team
Apply for this Position