Senior AI Researcher - Pre-training

About the role

The Senior AI Researcher will join the Pre-training team to advance the architecture and training of next-generation foundation models. This role involves working at the intersection of research and engineering to optimize training recipes, ensure model stability, and scale training processes on large GPU clusters. The researcher will translate mathematical reasoning and empirical observations into principled training decisions to improve model quality and efficiency.

Responsibilities

• Own core elements of the training recipe including optimizers, schedules, and initialization.
• Develop hyperparameter scaling laws and scale-up methodologies using proxy experiments.
• Investigate and resolve complex convergence issues and distributed system failures.
• Partner with cross-functional teams to align model lifecycles with hardware constraints and communication topologies.

Required skills

• Python
• PyTorch
• Machine Learning
• Transformer training
• Distributed systems
• Experimental design

Nice to have

• Megatron-LM
• DeepSpeed
• torchtitan
• Scaling laws
• Mixture-of-Experts
• Profiling

Qualifications

• Strong track record in machine learning research and software engineering
• Hands-on experience pre-training large models on substantial infrastructure

Benefits

• 30 days of paid vacation
• Wellhub fitness and wellness
• Mental health support
• Subsidized company pension plan
• Subsidized Germany-wide transportation ticket
• Technical equipment budget
• Flexible working hours
• JobRad bike lease

About the Company

Aleph Alpha is a European company focused on foundation model pre-training for customers in finance, manufacturing, and public administration. They prioritize a culture of ownership, autonomy, and flat organizational structure.