← Back to opportunities
About the Role
Role/Job title :AI Evaluation Engineer
Work Location :London
Type work : Contract
Mode of work : Hybrid
Responsibilities:
* Define and implement end-to-end evaluation strategy for generative AI conversational systems (LLMs, RAG pipelines, agents).
* Establish evaluation metrics
* Test Dataset & Benchmarking
* Benchmark models (GPT variants, open-source LLMs) and prompt strategies
* Implement automated evaluation pipelines
* Conduct human-in-the-loop evaluation for qualitative validation.
* Prompt & Response Quality Optimization
* RAG & Knowledge Grounding Validation
* Safety, Risk & Compliance Testing
Your Profile
* Strong understanding of LLMs, RAG architecture, prompt engineering
* Python – data analysis, evaluation pipelines
* Prompt evaluation tools (PromptTools, DeepEval, etc.)
Experience with:
- Evaluation Fra...
Work Location :London
Type work : Contract
Mode of work : Hybrid
Responsibilities:
* Define and implement end-to-end evaluation strategy for generative AI conversational systems (LLMs, RAG pipelines, agents).
* Establish evaluation metrics
* Test Dataset & Benchmarking
* Benchmark models (GPT variants, open-source LLMs) and prompt strategies
* Implement automated evaluation pipelines
* Conduct human-in-the-loop evaluation for qualitative validation.
* Prompt & Response Quality Optimization
* RAG & Knowledge Grounding Validation
* Safety, Risk & Compliance Testing
Your Profile
* Strong understanding of LLMs, RAG architecture, prompt engineering
* Python – data analysis, evaluation pipelines
* Prompt evaluation tools (PromptTools, DeepEval, etc.)
Experience with:
- Evaluation Fra...
Ready to Join Through a Referral?
Apply now and get connected directly with the hiring team
Apply for this Position