Senior AI Engineer - Inference Systems Optimization

The Role 

We are seeking a senior AI engineer passionate about performance optimization to join our team and revolutionize our inference systems. You will be at the heart of technical innovation, transforming the latest research advances into ultra-high-performance production solutions. 

Core Mission 

Your mission will be to radically optimize the performance of our systems by leveraging and improving current leading inference technologies. You will work on cutting-edge technical challenges, reducing latency and maximizing throughput to serve millions of users. 

Key Responsibilities 

Analysis and Benchmarking 

• Analyze performance bottlenecks (CPU scheduling, GPU memory, kernel efficiency) - Establish detailed performance metrics (TTFT, tokens/s, P99 latency, GPU utilization) - Design and implement comprehensive benchmarks comparing SOTA inference solutions performance 

• Document trade-offs between different optimization approaches 

Inference Systems Optimization 

• Optimize current inference engines/frameworks performance for our specific use cases - Implement advanced techniques: RadixAttention, PagedAttention, continuous batching - Develop optimized CUDA kernels (Triton, FlashInfer integration) 

• Integrate torch.compile and CUDA graphs to maximize performance 

• Optimize KV cache management and batching strategies 

Research Paper Implementation 

• Transform the latest academic innovations into production code 

• Implement optimization techniques like MLA (Multi-head Latent Attention) - Adapt MoE (Mixture of Experts) architectures for efficient inference 

• Integrate model-specific optimizations (DeepSeek, Llama, etc.) 

Infrastructure and Scalability 

• Architect distributed multi-GPU solutions with tensor parallelism 

• Optimize GPU fleet utilization (H100, H200, ...) 

• Implement advanced monitoring and profiling systems 

• Develop debugging tools to identify performance issues 

Desired Profile

Essential Technical Skills 

• Deep expertise in PyTorch and NVIDIA ecosystem (CUDA, NCCL, cuDNN)

• Mastery of inference frameworks: SGLang, vLLM, Dynamo, or equivalents

• Solid experience (5+ years) in ML systems optimization in production

• Practical knowledge of Transformer architectures and attention techniques

• Skills in GPU programming (CUDA, Triton) and kernel optimization 

Advanced Technical Skills (Strong Plus) 

• Experience with inference optimization techniques: 

• Quantization (INT8, INT4, FP8) 

• KV cache optimization (MQA, GQA, MLA) 

• Speculative decoding, multi-token prediction 

• Structured generation and constrained decoding 

• Knowledge of frameworks: FlashAttention, FlashInfer, xFormers - Experience with high-performance distributed systems 

• Contributions to open-source ML inference projects 

Personal Qualities 

• Passion for optimization and performance 

• Ability to read and implement complex research papers 

• Excellent analytical and problem-solving skills 

• Autonomy and ability to work on unstructured problems 

• Clear communication of technical results 

What We Offer 

Technical Impact 

• Work on systems serving billions of tokens per day 

• Access to latest GPUs (H100, H200) and compute resources - Direct collaboration with research teams 

• Open-source contributions and technical publications 

Work Environment 

• Team of experts passionate about performance 

• Culture of innovation and technical experimentation 

• Flexibility and autonomy in technical approaches 

• Continuous training on latest advances 

Compensation Package 

- Competitive salary aligned with expertise 

- Significant equity 

- Conference and training budget 

- Cutting-edge hardware

Key Technologies 

Inference frameworks: SGLang, vLLM, TensorRT-LLM 

GPU optimization: CUDA, Triton, FlashInfer, FlashAttention 

Deep Learning: PyTorch, torch.compile 

Architectures: Transformers, MoE, MLA, attention variants 

Infrastructure: Multi-GPU, tensor parallelism, distributed systems 

How to Apply 

Send your resume along with examples of optimization projects you have completed. We particularly value: 

• Open-source contributions to inference projects 

• Benchmarks or performance analyses you have conducted 

• Implementations of innovative optimization techniques