Senior AI Researcher - Pre-training

About the role

Aleph Alpha is seeking a Senior AI Researcher to join the Pre-training team in Heidelberg to advance the architecture and training of next-generation foundation models. The role involves working at the intersection of research and engineering to optimize training recipes, ensure model stability, and manage large-scale training runs. You will be responsible for translating mathematical reasoning and empirical observations into principled training decisions to improve model quality and efficiency.

Responsibilities

• Own core elements of the training recipe including optimizers, schedules, and initialization.
• Design PyTorch-based architectural improvements to maximize convergence, stability, and training efficiency.
• Develop hyperparameter scaling laws and scale-up methodologies using proxy experiments.
• Investigate and resolve complex convergence issues and distributed system failures.
• Partner with cross-functional teams to align model lifecycles with hardware constraints and communication topologies.

Required skills

• Python
• PyTorch
• Machine Learning Research
• Software Engineering
• Transformer training
• Distributed training
• Experimental design

Nice to have

• Megatron-LM
• DeepSpeed
• torchtitan
• Scaling laws
• Mixture-of-Experts
• Profiling

Qualifications

• Strong track record in machine learning research and software engineering
• Hands-on experience pre-training large models on substantial infrastructure
• Ability to work in Germany and collaborate on site in Heidelberg

Benefits

• 30 days of paid vacation
• Fitness and wellness offerings via Wellhub
• Mental health support through nilo.health
• Subsidized company pension plan
• Subsidized Germany-wide transportation ticket
• Budget for additional technical equipment
• Flexible working hours
• JobRad bike lease

About the Company

Aleph Alpha is one of the few companies in Europe doing serious foundation model pre-training for customers in finance, manufacturing, and public administration. The company fosters a culture of ownership, autonomy, and empowerment with a flat organizational structure.