AI Research Engineer (Kernel & Inference Optimization)

About the role

This role sits at the intersection of AI research, systems engineering, and high-performance model inference. You will develop and optimize model-serving architectures for advanced AI systems, focusing on latency, throughput, and memory efficiency across diverse hardware environments including mobile and edge devices.

Responsibilities

• Design and deploy advanced model-serving architectures optimized for high throughput, low latency, and efficient memory utilization.
• Develop inference pipelines capable of operating effectively across diverse environments, including resource-constrained mobile devices and edge platforms.
• Build and execute controlled inference benchmarks in simulated and production environments, tracking latency, throughput, memory consumption, and error rates.
• Identify computational and memory bottlenecks across inference pipelines and implement solutions involving batching, networking, memory management, and other system-level optimizations.
• Develop custom GPU kernels and compute shaders for mobile hardware, including solutions written in Metal Shading Language (MSL).
• Apply advanced inference optimization techniques such as pruning, quantization, Flash Attention, KV caching, and speculative decoding.
• Design and optimize distributed inference systems using approaches such as tensor parallelism, pipeline parallelism, and expert parallelism for large-scale GPU workloads.

Required skills

• Metal Shading Language
• Kernel optimization
• Inference optimization
• GPU kernels
• Model-serving architectures
• Distributed inference
• Tensor parallelism
• Pipeline parallelism
• Pruning
• Quantization
• Flash Attention
• KV caching
• Speculative decoding
• Diffusion models
• Vision Transformers

Qualifications

• Degree in Computer Science or a related technical field
• PhD in NLP, Machine Learning, or a related discipline

Benefits

• Remote-first working environment
• International team