GenAI Test Automation Lead

As a GenAI Test Automation Lead, you will design scalable, Python-driven automation frameworks to validate LLMs, prompt frameworks, and RAG systems across UI and API layers. You will leverage modern CI/CD pipelines and write complex SQL queries to perform deep backend and database data validation.

In this role, you will

• Lead the design, implementation, and maintenance of scalable end-to-end test automation frameworks tailored for GenAI applications and Large Language Models (LLMs).
• Build automation solutions that go beyond standard UI tools (like Selenium), integrating multiple testing tools, API layers, and backend systems into a cohesive automation suite.
• Actively test and evaluate LLMs, prompt engineering frameworks, and Retrieval-Augmented Generation (RAG) systems using specialized AI evaluation tools like LangChain.
• Embed automated GenAI evaluation and regression suites into modern CI/CD pipelines ...