RemoteLeaf Logo
Log in Sign up

Staff Software Engineer, AI Inference

7 hours, 39 minutes ago
United States
Full-time
Lead
Software Engineer
Software Development
C++ Go LLM Machine Learning Python Rust
Apply Now
TLATech

TLATech

Syllo is the litigation workspace for the AI era. See how our Agentic AI works and request a demo today.

Specialized Consumer Services
View All Jobs 4

Description

  • Lead the design and development of the production inference platform.
  • Define the technical roadmap for inference infrastructure, model serving, and runtime optimization.
  • Build and operate scalable, cost-effective systems for serving large language models in production.
  • Evaluate and integrate modern inference technologies, frameworks, and serving runtimes.
  • Optimize latency, throughput, GPU utilization, memory efficiency, and infrastructure cost.
  • Develop systems for model deployment, traffic routing, autoscaling, scheduling, observability, and operational excellence.
  • Partner with machine learning engineers to productionize new models and inference techniques.
  • Establish benchmarking methodologies to evaluate new models, runtimes, and hardware.
  • Make architectural decisions about when to build internally versus use open-source or commercial solutions.
  • Mentor engineers and help establish engineering best practices for AI infrastructure.

Requirements

  • Significant experience designing and operating production AI inference systems.
  • Experience building or leading production LLM serving infrastructure.
  • Deep experience with modern inference runtimes and frameworks such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, Hugging Face TGI, NVIDIA Dynamo, or similar technologies.
  • Strong background in distributed systems, backend infrastructure, or high-performance platform engineering.
  • Experience optimizing inference performance across GPU workloads, including latency, throughput, batching, memory utilization, and serving efficiency.
  • Experience operating GPU infrastructure in production.
  • Strong proficiency in Python and at least one systems programming language such as Go, Rust, or C++.
  • Proven ability to lead technical architecture for complex infrastructure initiatives.
  • Excellent communication skills and ability to influence technical direction across engineering teams.

Benefits

  • Salary range of $190,000 to $230,000 USD.
  • Health insurance.
  • Equity.
  • Remote work in the United States.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

9th Way Insignia

Temporary Task-Based Senior Software Developer

9th Way Insignia 51-250 Internet Software & Services

9th Way Insignia is hiring a Temporary Task-Based Senior Software Developer to support the VA Office of Academic Affiliations Data Management Center in modernizing, sustaining, and securing a portfolio of government applications and data systems.

United States Contract Senior Software Engineer
$98k-$135k
Azure C# CSS JavaScript .NET Power BI SQL Server
7 hours, 24 minutes ago
Conga

Senior Software Engineer (AI Platform)

Conga 1K-5K IT Services

Conga is hiring a Senior Software Engineer for its AI Platform team to design, build, and optimize enterprise AI and GenAI products that support scalable, reliable production systems.

Bulgaria Full-time Senior Software Engineer
C# Celery Docker FastAPI GCP Generative AI Java Kafka Kubeflow Kubernetes Machine Learning MLOps MongoDB NLP PostgreSQL Pytest Python Scala SQL SQL Server Transformers Vertex AI
7 hours, 24 minutes ago
Reddit

Senior Staff Software Engineer, Client Architecture

Reddit 1K-5K Internet Software & Services

Reddit is hiring a Senior Staff Engineer to lead the multi-year client architecture overhaul for its post and comment composers across web and mobile, with the goal of reducing creation friction and expanding new contribution formats, especially video.

United States Full-time Lead Software Engineer
$232k-$326k
Android iOS Kotlin React Swift TypeScript
7 hours, 24 minutes ago
9th Way Insignia

Software Developer (Microsoft Power Platform)

9th Way Insignia 51-250 Internet Software & Services

9th Way Insignia is seeking a mid-level Software Developer for the VA Office of Academic Affiliations Data Management Center to build and maintain Microsoft Power Platform solutions that support enterprise modernization and sustainment work in the VA Enterprise Cloud.

United States Full-time Mid Level Software Engineer
$79k-$110k
Azure .NET Power BI SQL SQL Server
7 hours, 24 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers