RemoteLeaf Logo
Log in Sign up

Senior/Principal Local LLM & Generative AI Platform Engineer

12 hours, 35 minutes ago
United States
Full-time
Lead
AI Engineer
DevOps and Infrastructure
Active Directory C C++ CI/CD Docker Encryption Generative AI Git Go Java Kubernetes Linux MLOps Python REST API Secrets Management
Apply Now
Parallel Wireless

Parallel Wireless

Parallel Wireless leads the OpenRAN movement with innovative cloud-native architecture, enabling cost-effective cellular network deployment globally.

Wireless Telecommunication Services
501-1000
Founded 2012
$2M raised
View All Jobs 15

Description

  • Own the architecture and technical roadmap for a secure, reliable local LLM platform in company-controlled infrastructure.
  • Partner with engineering, product, support, IT, security, legal, and domain experts to prioritize use cases and define measurable requirements.
  • Build modular model inference and gateway services with stable APIs, routing, streaming, quotas, and interchangeable serving backends.
  • Evaluate and optimize open-weight language, code, embedding, reranking, and multimodal models for company-specific use cases.
  • Design and operate permission-aware RAG and enterprise-search pipelines across approved internal content sources.
  • Integrate identity, SSO, role-based access controls, secrets management, audit logging, and source-system permissions throughout the platform.
  • Establish automated evaluations, regression testing, release gates, canary releases, rollback procedures, and quality metrics.
  • Implement observability and production operations, including CI/CD, capacity planning, backups, disaster recovery, vulnerability management, and incident response.
  • Design secure tool-calling and agent workflows with least-privilege access, sandboxing, validation, approval controls, and traceability.
  • Integrate reusable AI capabilities into developer tools, source control, CI, knowledge systems, ticketing systems, and internal applications.

Requirements

  • BSc or MSc in Computer Science, Computer Engineering, Electrical Engineering, Data Science, or a related field, or equivalent practical experience.
  • Typically 7+ years of hands-on experience in production software, ML platform, search, data, or infrastructure engineering, including recent experience shipping LLM-powered systems.
  • Strong Python skills and experience building maintainable APIs, services, libraries, and data pipelines; Go, Java, or C/C++ experience is advantageous.
  • Strong understanding of transformer models and production inference, including tokenization, context management, batching, KV caching, parallelism, quantization, structured output, and tool calling.
  • Experience building production RAG or enterprise-search systems using embeddings, vector or lexical search, metadata filtering, reranking, attribution, and retrieval evaluation.
  • Experience defining LLM evaluations with representative datasets, baselines, domain-expert review, automated metrics, human feedback, error analysis, and regression thresholds.
  • Experience deploying and operating containerized Linux services with Docker and Kubernetes or equivalent orchestration.
  • Practical experience with GPU-backed model serving, performance profiling, capacity planning, monitoring, reliability engineering, and distributed systems.
  • Knowledge of authentication, authorization, API security, secrets handling, encryption, auditability, data lifecycle controls, Git, CI/CD, infrastructure as code, and incident response.
  • Experience with private, restricted, or air-gapped LLM environments; inference systems such as vLLM, SGLang, TensorRT-LLM, llama.cpp, Ray Serve, KServe, or Triton; and LLM security testing is preferred.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Nscale

People Systems Engineer

Nscale 501-1000 information technology & services

Nscale is seeking a People Systems Engineer to build and operate custom tools, integrations, and AI agent workflows that unify People platforms and support its growing global workforce.

North America Latin America Full-time Senior AI Engineer
$150k-$230k
10 hours, 20 minutes ago
Brillio

Architect - R01571180

Brillio 1001-5000 IT Services

Brillio is hiring a remote AI / Agentic AI Architect in New York to architect and scale secure, production-grade AI agents, autonomous workflows, and LLM-powered enterprise applications for clients.

United States Lead AI Engineer
$0k-$0k
AWS Azure CI/CD CRM DevSecOps Docker GCP Generative AI Go Java Kubernetes LLM Microservices Node.js Python React SQL TypeScript Vertex AI
12 hours, 50 minutes ago
Brillio

AI Builders

Brillio 1001-5000 IT Services

Join Brillio as a client-embedded healthcare AI engineer, identifying high-value operational opportunities and rapidly building governed, production-ready AI solutions for enterprise clients.

India Lead AI Engineer
HIPAA LLM REST API
12 hours, 50 minutes ago
Brillio

AI Builders

Brillio 1001-5000 IT Services

Brillio is seeking a client-embedded healthcare AI engineer to identify high-value operational opportunities and rapidly deliver scalable, governed AI solutions for enterprise clients.

Romania Lead AI Engineer
HIPAA LLM REST API
12 hours, 50 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers