ML Platform Engineer, Distributed Training

2 hours, 34 minutes ago
Full-time
Senior
DevOps and Infrastructure
Egen.ai

Egen.ai

Egen.ai specializes in providing technology services that leverage cloud computing, data analytics, and artificial intelligence to enhance document intelligence and drive productivity and growth for its clients.

IT Services
Founded 2000

Description

  • Partner with the infrastructure team on GKE GPU node pools, job scheduling, quotas, and capacity management.
  • Diagnose and resolve training-platform failures at their root cause.
  • Tune multi-node GPU networking, including RDMA, GPUDirect, and NCCL.
  • Run, scale, and debug distributed PyTorch and Ray Train workloads.
  • Profile training jobs and eliminate compute, memory, input-pipeline, and networking bottlenecks.
  • Design performance benchmarks measuring throughput, scaling efficiency, and cost per run.
  • Build and maintain model evaluation harnesses and end-to-end pipeline tests.
  • Instrument training platforms with telemetry and monitoring.
  • Proactively identify issues, communicate risks, defend technical decisions, and support client scientists.

Requirements

  • 5+ years of engineering experience, including recent hands-on work with GPU training infrastructure and large-scale training.
  • Strong Python and shell scripting skills, including production-quality code, testing, and code review.
  • Experience with Kubernetes on Google Cloud, GKE GPU node pools, job queueing such as Kueue, and quota management.
  • Hands-on experience running and debugging multi-node distributed training with PyTorch, FSDP, DeepSpeed, Megatron or NeMo, and Ray Train.
  • Experience with RDMA, GPUDirect, NCCL tuning, and H200 or GB200 multi-node GPU topologies.
  • Experience designing performance benchmarks and model evaluations for technical decision-making.
  • Strong software engineering fundamentals and ability to explain system behavior without relying on AI coding tools.
  • Track record of visible ownership, communication, and accountability.
  • Experience with Google Cloud Vertex AI, scalable architectures, LLMs, prompt engineering, fine-tuning, vector databases, semantic search, and LangChain or LlamaIndex.
  • Healthcare or life-sciences data, HIPAA, and training-platform observability experience preferred.

Benefits

  • Competitive salary and comprehensive benefits package.
  • Comprehensive health insurance.
  • Paid vacation/PTO, holidays, and sick leave.
  • Parental and bereavement leave.
  • 401(k) employer match.
  • Employee referral bonuses.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Manager, Software Engineering - AI Infrastructure

Figma 1K-5K Internet Software & Services

Figma is hiring an Engineering Manager for its AI Infrastructure team to lead the development of reliable, observable, and scalable inference systems powering products such as Figma Make and Figma Agent.

1 hour, 49 minutes ago

Staff Engineer - AI Platform

HighLevel 251-1K Internet Software & Services

HighLevel is seeking a hands-on Staff Engineer (AI) to architect shared agentic systems and AI platform foundations that enable engineering teams to build and operate reliable AI capabilities at enterprise scale.

AWS GCP Go Grafana Microservices Node.js OpenSearch Prometheus Python
2 hours, 19 minutes ago

Senior / Staff Platform Software Engineer

Clear Street 501-1000 financial services

Clear Street is hiring a senior software engineer to build the AI platform powering a copilot within its cloud-based, multi-asset trading platform, with a focus on secure, reliable, low-latency trading workflows.

PostgreSQL React React Native Rust TypeScript
2 hours, 19 minutes ago

[Job-32284] Principal AI Platform Engineer , Brazil

CI&T 5K-10K Internet Software & Services

A CI&T busca um(a) Principal AI Platform Engineer hands-on para liderar, na plataforma de Agentic SDLC, a arquitetura, implantação e evolução do acesso a modelos de IA e a entrega da solução em produção com um time técnico.

AWS CI/CD Docker Git Java JIRA Kubernetes Microservices Node.js OpenTelemetry Playwright Python Secrets Management
2 hours, 34 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers