ML Engineer in 2026: You Build the Tools Everyone Else Uses

Ravi's morning standup: the new customer-support model shipped yesterday; eval scores on the blind test set held up (F1 0.87, hallucination rate 2.1%). The prod traffic shows a 12% drop in escalations. After standup, he reviews a failed eval: the model is wrong when customers use Spanish code-switching. He queues a data curation task to label 500 more examples, plans a LoRA fine-tune this afternoon on Modal, and sketches the A/B test to gate the rollout. Every day is 20% research, 30% data, 30% infra, 20% writing.

• Model selection — frontier APIs (Claude, GPT-5.5, Gemini) vs. open weights (Llama, Qwen, DeepSeek).
• Fine-tuning — LoRA, QLoRA, full fine-tune; tools like Axolotl, Unsloth, and Together AI.
• Eval engineering — Promptfoo, LangSmith, and home-grown harnesses for golden datasets.
• Serving — vLLM, TGI, TensorRT-LLM; latency and throughput optimization.
• Distillation and quantization — take a 70B model to a 7B with 90% of the quality at 10% of the cost.
• RLHF / DPO / RLAIF — align a base model to your task with preference data.
• Monitoring — drift, poisoning detection, RAG freshness, hallucination rates in prod.

• PyTorch + Hugging Face Transformers — still the default research stack.
• vLLM — the high-throughput inference server everyone runs.
• Modal, Together AI, Replicate — serverless GPU + inference.
• Weights & Biases — experiment tracking.
• LangSmith, Braintrust, Promptfoo — eval infrastructure.
• Ray — distributed training and inference.
• Mosaic, Anyscale, and Nebius — GPU training platforms.

Choosing the right problem. Deciding whether to build, buy, or skip. Designing an eval that actually measures what matters (most evals do not). Explaining model limits to a product manager who wants magic. Debugging a training run that silently collapsed after 12 hours and $4,000 of GPU. Negotiating compute with infra. Reading a new paper and deciding what to steal. Designing the system that degrades gracefully when the API you depend on goes down.

• Linear algebra, probability, and optimization — the math everyone wishes they had taken more seriously.
• PyTorch fluency — read papers, implement, reproduce.
• GPU and systems — CUDA basics, memory, distributed training.
• Eval design — the #1 differentiator between junior and senior MLEs.
• Software engineering — MLEs who can ship production code earn more.
• Specialty — LLMs, vision, speech, recsys, ranking, robotics. Go deep in one.

If you want to be an ML engineer: In high school, take AP Calculus BC, AP Statistics, and AP CS. In college, major in CS or math with a heavy ML track; take ML theory, not just Coursera. A master's or PhD opens frontier-lab roles, but strong portfolio work beats credentials in industry. Build on Hugging Face. Reproduce a paper. Fine-tune something small and write about what you learned. Compensation is the highest in engineering — frontier labs start new grads at $300k+ in 2026 — but the field moves fast. Expect to relearn the stack every 18 months and love it.