← Back to opportunities
About the Role
About The Role Lead Inference Platform Engineer – specialized experience in machine learning/deep learning domains such as model compression, hardware‑aware model optimizations, hardware accelerators architecture, GPU/ASIC architecture, ML compilers, high‑performance computing, performance optimizations, numerics, or SW/HW co‑design.
Responsibilities
Optimize LLMs and ML models for high‑performance inference using quantization, pruning, distillation, and hardware specific tuning.
Deploy and scale inference workloads on GPUs across AWS, Azure, GCP and internal Kubernetes clusters, ensuring predictable performance during peak traffic.
Implement routing and fail‑over strategies for OpenAI / Anthropic / Vertex AI traffic.
Integrate models into production‑grade APIs supporting TR products and enterprise workflows.
Develop highly optimized environments and eliminate performance bottlenecks to reduce latency.
Collaborate with Plat...
Responsibilities
Optimize LLMs and ML models for high‑performance inference using quantization, pruning, distillation, and hardware specific tuning.
Deploy and scale inference workloads on GPUs across AWS, Azure, GCP and internal Kubernetes clusters, ensuring predictable performance during peak traffic.
Implement routing and fail‑over strategies for OpenAI / Anthropic / Vertex AI traffic.
Integrate models into production‑grade APIs supporting TR products and enterprise workflows.
Develop highly optimized environments and eliminate performance bottlenecks to reduce latency.
Collaborate with Plat...
Ready to Join Through a Referral?
Apply now and get connected directly with the hiring team
Apply for this Position