Machine Learning Engineer II

Zendesk’s people have one goal in mind: to make Customer Experience better. Our products help more than 125,000 global brands (AirBnb, Uber, JetBrains, Slack, among others) make their billions of customers happy, every day.

The AI/ML Platform team is at the forefront of this mission. We build the foundation that powers every AI-driven experience at Zendesk, enabling product teams to build, evaluate, and deploy state-of-the-art Large Language Model (LLM) applications reliably and at scale.

We’re looking for an ML Engineer II to help design, implement, and improve core components of Zendesk’s GenAI infrastructure — from LLM Proxy and evaluation frameworks to orchestration tools for AI agents. You’ll work alongside senior engineers and technical leads, contributing to systems that are safe, scalable, and developer-friendly.

This is a hands-on engineering role with significant growth potential, ideal for someone with solid ML engineering skills who wants to deepen expertise in LLM systems and GenAI infrastructure.

What you get to do every day

• Contribute to the development of Zendesk’s LLM Proxy, enabling secure, cost-optimized access to multiple foundation models.

• Help develop and maintain benchmarking and A/B testing frameworks for measuring LLM performance, latency, and cost.

• Assist in building orchestration systems that enable multi-step, tool-using AI agents.

• Work closely with applied ML, product, and platform teams to ensure infrastructure meets product needs.

• Deliver well-tested, maintainable, and performant code ready for production deployment.

What you bring to the role

• 2–4 years of hands-on experience developing and deploying ML systems or infrastructure.

• Familiarity with LLM applications, vector databases, or ML/AI infrastructure components.

• Exposure to AWS, GCP, or Azure; understanding of Kubernetes, Docker, or similar containerized environments.

•  Proficiency in Python and familiarity with at least one other backend language (Java, Scala, Golang, or Ruby).

• Understanding of CI/CD workflows, testing frameworks, and software design principles.

• Ability to work effectively in cross-functional teams and take feedback constructively.

Preferred Qualifications

• Experience building small-scale ML services or APIs.

• Exposure to ML observability tools and evaluation frameworks.

• Understanding of prompt engineering or fine-tuning workflows for LLMs.

What our tech stack looks like

• Our code is written in Python.

• Our servers live in AWS.

• LLM Vendors: OpenAI, Anthropic, Google, Llama

• Infra: Kubernetes, Docker, Kafka, AWS 

What we offer

• Full ownership of the projects you work on.

• What you will be doing will have a huge impact.

• Team of passionate people who love what they do.

• Exciting projects, ability to implement your own ideas and improvements.

• Opportunity to learn and grow. 

...and everything you need to be effective and maintain work-life balance

• Flexible working hours.

• Professional development funds.

• Comfortable office and a remote setup.

• Choice of your laptop and other equipment.

• Premium Medical Insurance as well as Private Life Assurance.