RemoteLeaf Logo
Log in Sign up

Staff Software Engineer, AI Inference

1 month, 1 week ago
United States
Full-time
Lead
Software Engineer
Software Development
C++ Go LLM Machine Learning Python Rust
Apply Now
TLATech

TLATech

Syllo is the litigation workspace for the AI era. See how our Agentic AI works and request a demo today.

Specialized Consumer Services
View All Jobs 15

Description

  • Lead the design and development of the production inference platform.
  • Define the technical roadmap for inference infrastructure, model serving, and runtime optimization.
  • Build and operate scalable, cost-effective systems for serving large language models in production.
  • Evaluate and integrate modern inference technologies, frameworks, and serving runtimes.
  • Optimize latency, throughput, GPU utilization, memory efficiency, and infrastructure cost.
  • Develop systems for model deployment, traffic routing, autoscaling, scheduling, observability, and operational excellence.
  • Partner with machine learning engineers to productionize new models and inference techniques.
  • Establish benchmarking methodologies to evaluate new models, runtimes, and hardware.
  • Make architectural decisions about when to build internally versus use open-source or commercial solutions.
  • Mentor engineers and help establish engineering best practices for AI infrastructure.

Requirements

  • Significant experience designing and operating production AI inference systems.
  • Experience building or leading production LLM serving infrastructure.
  • Deep experience with modern inference runtimes and frameworks such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, Hugging Face TGI, NVIDIA Dynamo, or similar technologies.
  • Strong background in distributed systems, backend infrastructure, or high-performance platform engineering.
  • Experience optimizing inference performance across GPU workloads, including latency, throughput, batching, memory utilization, and serving efficiency.
  • Experience operating GPU infrastructure in production.
  • Strong proficiency in Python and at least one systems programming language such as Go, Rust, or C++.
  • Proven ability to lead technical architecture for complex infrastructure initiatives.
  • Excellent communication skills and ability to influence technical direction across engineering teams.

Benefits

  • Salary range of $190,000 to $230,000 USD.
  • Health insurance.
  • Equity.
  • Remote work in the United States.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

KoBold Metals

Senior Software Engineer, Scientific Computing (Hardware/Sensors)

KoBold Metals 51-250 Metals & Mining

KoBold Metals is hiring a Senior Software Engineer, Scientific Computing to build the scientific computing and machine learning systems that support mineral exploration across complex geological and sensor data.

United States Canada Full-time Senior Software Engineer
$170k-$215k
Deep Learning Machine Learning NumPy Python
39 minutes ago
Torc

Software Engineer II - Business Intelligence

Torc 251-1K Road & Rail

Torc is seeking a Data Engineer to help power BI reporting and leadership dashboards by building and maintaining the data pipelines, models, and metrics behind the company’s analytics architecture.

United States Full-time Mid Level Business Intelligence Analyst Software Engineer
$139k-$167k
Apache Spark AWS Databricks Machine Learning NumPy Pandas Power BI Python SQL Tableau
54 minutes ago
LightFeather

Senior ServiceNow Engineer

LightFeather 51-250 Internet Software & Services

LightFeather is hiring a Senior ServiceNow Engineer to design and deliver ServiceNow solutions for federal government projects in a fully remote, full-time role.

United States Full-time Senior Software Engineer Solutions Architect
$90k-$115k
Agile Angular Azure CSS GCP HTML JavaScript JIRA Kanban REST API Scrum
54 minutes ago
Anaplan

Engineer IV

Anaplan 1K-5K Internet Software & Services

Anaplan is hiring Engineer IV to provide technical leadership while building scalable web applications and cloud-based solutions for its AI-infused scenario planning platform.

India Full-time Lead Full-stack Engineer Software Engineer
Agile Angular Azure CI/CD Docker Git Grafana GraphQL JSON Kubernetes Microservices .NET PostgreSQL React REST API Scrum SQL Server System Design Vue.js
54 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers