Machine Learning Engineer — Distillation

About the Role We’re looking for a Machine Learning Engineer focused on model distillation to help us build smaller, faster, and more efficient models without sacrificing quality. You’ll work at the intersection of research and production—taking cutting-edge techniques and turning them into systems that scale. This is a hands-on role with real ownership: you’ll design distillation pipelines, run large-scale experiments, and ship models used in production.

What You’ll Do

• Design and implement knowledge distillation pipelines (teacher–student, self-distillation, multi-teacher, etc.)

• Distill large foundation models into smaller, faster, and cheaper models for inference

• Run and analyze large-scale training experiments to evaluate quality, latency, and cost tradeoffs

• Collaborate with research to translate new distillation ideas into production-ready code

• Optimize training and inference performance (memory, throughput, latency)

• Contribute to internal tooling, evaluation frameworks, and experiment tracking

• (Optional) Contribute back to open-source models, tooling, or research

What We’re Looking For

• Strong background in machine learning or deep learning

• Hands-on experience with model distillation (LLMs or other neural networks)

• Solid understanding of training dynamics, loss functions, and optimization

• Experience with PyTorch (or JAX) and modern ML tooling

• Comfort running experiments on multi-GPU or distributed setups

• Ability to reason about model quality vs. performance tradeoffs

• Pragmatic mindset: you care about shipping, not just papers

Nice to Have

• Experience distilling LLMs or large sequence models

• Experience with inference optimization (quantization, pruning, kernels, etc.)

• Familiarity with evaluation for language models

• Open-source contributions or research publications

• Experience in early-stage or fast-moving startups

Why Join

• Work on core model quality and cost efficiency —not side projects

• High ownership and direct impact on product and roadmap

• Small, senior team with strong research + engineering culture

• Competitive compensation + meaningful equity

• Remote-friendly, async-first environment

Originally posted on Himalayas