RemoteLeaf Logo
Log in Sign up

Tech Lead - AI Inference

10 hours, 37 minutes ago
United States
Full-time
Lead
Technical Lead
Software Development
C++ Kubernetes Python Rust
Apply Now
WEKA

WEKA

WEKA provides a cutting-edge data platform for cloud and on-premises data management, offering WekaFS for high-performance storage solutions in data-intensive environments.

IT Services
201-500
Founded 2014
$294M raised
View All Jobs 24

Description

  • Own AMG’s end-to-end inference infrastructure, including the NVMe Token Warehouse, GDS data paths, and vLLM/LMCache serving stack.
  • Set technical direction and drive the design, implementation, and delivery of high-throughput, low-latency LLM inference systems.
  • Contribute hands-on to production systems across Python, C++, CUDA, vLLM, NIXL/Dynamo, and Kubernetes.
  • Solve inference-scaling challenges involving disaggregated prefill/decode, persistent KV caching, RDMA transport, and multi-tier GPU memory.
  • Lead, unblock, and grow a squad of three developers while driving execution and delivery outcomes.
  • Mentor engineers through 1:1s, career coaching, and sprint reviews.
  • Establish high standards for code quality, architecture, reliability, ownership, and collaboration.
  • Evaluate and benchmark emerging inference tools, including SGLang, TRT-LLM, and NVIDIA Dynamo, to guide adoption decisions.

Requirements

  • 8+ years of professional software engineering experience, including leadership of engineers and ownership of complex production systems.
  • Experience in AI/ML infrastructure or high-performance computing.
  • Hands-on expertise with LLM serving systems, including KV cache reuse, disaggregated prefill/decode, continuous batching, and multi-tier GPU memory hierarchies.
  • Familiarity with vLLM, LMCache, NIXL/NVIDIA Dynamo, or similar inference frameworks.
  • Strong Python and C++ skills; Rust experience is preferred.
  • Knowledge of CUDA, GPU memory management, and high-performance I/O, including GPUDirect Storage, RDMA, and NVMe data paths.
  • Experience deploying and scaling GPU workloads on Kubernetes.
  • Familiarity with RDMA networking, bare-metal H100/A100 GPU clusters, and high-throughput distributed storage.
  • Demonstrated ability to mentor and develop engineers through effective 1:1s and career support.
  • Track record of building reliable, high-throughput systems and improving engineering practices.

Benefits

  • Total compensation based on location, qualifications, certifications, experience, education, and skills.
  • Medical, dental, vision, and life insurance.
  • 401(k) plan.
  • Flexible time off, sick time, and applicable leave of absence, including FMLA leave.
  • Equal opportunity and inclusive workplace.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Flash

Especialista de Engenharia de Software (Staff Engineer)

Flash 501-1000 Professional Services

Atue como Staff Engineer na Flash, plataforma de soluções para RH e financeiro, definindo a direção técnica e escalando sistemas resilientes que sustentam o crescimento do negócio.

Brazil Full-time Lead Technical Lead
AWS Azure CI/CD GCP Grafana GraphQL Kafka Microservices MongoDB NestJS Node.js REST API TypeScript
1 day, 10 hours ago
360Learning

Technical Lead Engineering

360Learning 201-500 Diversified Consumer Services

As Technical Lead for 360Learning’s Analytics Squad, you will guide the squad’s technical direction, coach its engineers, and deliver reliable product improvements for the collaborative learning platform.

France Full-time Lead Technical Lead
Agile
3 days, 9 hours ago
Brillio

Sr. Engineer HCM App Integration - R01572131

Brillio 1001-5000 IT Services

Brillio is seeking a Sr. Engineer, HCM App Integration to provide technical leadership and architectural oversight for Oracle HCM integrations supporting critical HR, benefits, payroll, equity, and enterprise systems.

United States Lead Technical Lead
$95k-$100k
CI/CD HDL JIRA REST API SFTP SOAP
5 days, 10 hours ago
Brillio

Inventory / Workforce Management Lead/Deployment System Lead - AMIR - R01570963

Brillio 1001-5000 IT Services

Brillio is seeking an Inventory/Workforce Management Lead to deliver integrated inventory tracking, workforce management, and deployment routing systems for an AMI Replacement project in Monterey Park, California.

United States Contract Lead Technical Lead
$125k-$146k
Agile
5 days, 10 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers