RemoteLeaf Logo
Log in Sign up

Staff Software Engineer, AI Inference

2 months, 1 week ago
United States
Full-time
Lead
Software Engineer
Software Development
C++ Go LLM Machine Learning Python Rust
Apply Now
TLATech

TLATech

Syllo is the litigation workspace for the AI era. See how our Agentic AI works and request a demo today.

Specialized Consumer Services
View All Jobs 15

Description

  • Lead the design and development of the production inference platform.
  • Define the technical roadmap for inference infrastructure, model serving, and runtime optimization.
  • Build and operate scalable, cost-effective systems for serving large language models in production.
  • Evaluate and integrate modern inference technologies, frameworks, and serving runtimes.
  • Optimize latency, throughput, GPU utilization, memory efficiency, and infrastructure cost.
  • Develop systems for model deployment, traffic routing, autoscaling, scheduling, observability, and operational excellence.
  • Partner with machine learning engineers to productionize new models and inference techniques.
  • Establish benchmarking methodologies to evaluate new models, runtimes, and hardware.
  • Make architectural decisions about when to build internally versus use open-source or commercial solutions.
  • Mentor engineers and help establish engineering best practices for AI infrastructure.

Requirements

  • Significant experience designing and operating production AI inference systems.
  • Experience building or leading production LLM serving infrastructure.
  • Deep experience with modern inference runtimes and frameworks such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, Hugging Face TGI, NVIDIA Dynamo, or similar technologies.
  • Strong background in distributed systems, backend infrastructure, or high-performance platform engineering.
  • Experience optimizing inference performance across GPU workloads, including latency, throughput, batching, memory utilization, and serving efficiency.
  • Experience operating GPU infrastructure in production.
  • Strong proficiency in Python and at least one systems programming language such as Go, Rust, or C++.
  • Proven ability to lead technical architecture for complex infrastructure initiatives.
  • Excellent communication skills and ability to influence technical direction across engineering teams.

Benefits

  • Salary range of $190,000 to $230,000 USD.
  • Health insurance.
  • Equity.
  • Remote work in the United States.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Affirm

Senior Software Engineer, Backend (Identity International)

Affirm 1K-5K Diversified Financial Services

Affirm is hiring a Senior Software Engineer for its remote UK Identity team to build and operate reliable identity, KYC, and user lifecycle systems supporting international expansion and compliant customer checkout experiences.

United Kingdom Full-time Senior Software Engineer
$152k-$213k
AWS Kotlin Kubernetes MySQL Python
19 hours, 58 minutes ago
Northbeam

Staff Software Engineer, Python

Northbeam 11-50 Internet Software & Services

Northbeam is hiring a Staff Backend-leaning Software Engineer to architect scalable data, experimentation, and AI systems for its remote-friendly marketing intelligence platform.

Canada Full-time Lead Software Engineer
$190k-$220k
Machine Learning Microservices Python React TypeScript
19 hours, 58 minutes ago
Northbeam

Software Engineer, Data Systems (Python)

Northbeam 11-50 Internet Software & Services

Northbeam is hiring a data-focused software engineer to build reliable integrations, pipelines, and data products powering its marketing intelligence platform for eCommerce brands.

United States Full-time Mid Level Software Engineer
$140k-$155k
Apache Airflow Dagster Docker GCP GraphQL Kubernetes Prefect Python REST API Secrets Management Shopify Snowflake SQL
20 hours, 13 minutes ago
Northbeam

Staff Software Engineer, Python

Northbeam 11-50 Internet Software & Services

Northbeam is seeking a Staff Software Engineer with a backend focus to architect scalable data, experimentation, and AI systems for its remote-friendly marketing intelligence platform.

United States Full-time Lead Software Engineer
$190k-$220k
Machine Learning Microservices Python React TypeScript
20 hours, 13 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers