RemoteLeaf Logo
Log in Sign up

Senior ML Infrastructure Engineer

3 weeks, 6 days ago
United States
Senior
Machine Learning Engineer
DevOps and Infrastructure
Active Directory Azure BERT Docker Helm Kubernetes Python PyTorch Terraform
Apply Now
Orion Innovation

Orion Innovation

Orion Innovation is a global technology services provider specializing in digital transformation, offering solutions in data, analytics, enterprise collaboration, risk & compliance, and cloud services to enhance productivity and decision-making.

IT Services
1K-5K
Founded 1993
View All Jobs 15

Description

  • Own the end-to-end infrastructure layer for the document intelligence platform, from GPU cluster configuration to model serving.
  • Design and manage Kubernetes-based workloads on Azure Kubernetes Service, including multi-node-pool architecture and autoscaling policies.
  • Configure and maintain GPU node pools, device plugins, driver compatibility, and resource limits for ML workloads.
  • Orchestrate Kubernetes jobs and event-driven processing using KEDA, queue triggers, and scaled jobs.
  • Manage CUDA and cuDNN runtime behavior for GPU inference workloads, including debugging performance and memory issues.
  • Support model deployment and inference for BERT-class NLP models using PyTorch and Hugging Face Transformers.
  • Implement batching, FP16 optimization, profiling, and memory management for efficient inference.
  • Build and maintain Azure-integrated services such as queue consumers, async workers, Key Vault, private endpoints, and Azure Data Lake Storage Gen2.
  • Author and maintain infrastructure and deployment assets including Docker images, Helm charts, and infrastructure as code.
  • Collaborate across platform engineering and applied ML to deliver a low-latency analyst-facing query interface.

Requirements

  • Strong experience with Kubernetes and Azure Kubernetes Service (AKS).
  • Experience designing multi-node-pool clusters with taints/tolerations, autoscaler configuration, and GPU node pools such as NC/ND series.
  • Hands-on knowledge of GPU workload tooling including device plugins, driver compatibility, resource limits, KEDA, and CUDA/cuDNN.
  • Experience with PyTorch for GPU inference and runtime configuration; raw kernel development is not required.
  • Experience with batching, FP16, memory management, profiling, and Hugging Face Transformers.
  • Experience loading and serving BERT, DistilBERT, or BGE models, including pipeline APIs and tokenization.
  • Strong Python experience in production environments.
  • Experience building async workers and queue consumers with Azure SDKs and Azure infrastructure.
  • Experience with VNet networking, private endpoints, Key Vault, ADLS, Azure AD, Docker, and Helm.
  • Experience authoring multi-stage builds, Helm charts, and infrastructure as code using Terraform or Bicep.
  • Preferred: willingness to learn and grow into adjacent technologies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

x.ai

ML Infrastructure Engineer

x.ai 51-250 Internet Software & Services

SpaceXAI is hiring an ML Infrastructure Engineer to build and optimize the machine learning platform that powers recommendations on X.

United States Full-time Junior Infrastructure Engineer Machine Learning Engineer
$180k-$440k
Ansible C++ Linux Puppet Python PyTorch Rust
5 hours, 9 minutes ago
Reddit

Senior Machine Learning Engineer, Safety

Reddit 1K-5K Internet Software & Services

Reddit is hiring a remote-friendly Machine Learning Engineer to build and improve safety systems that support enforcement of Reddit rules using large language models.

United States Full-time Senior Machine Learning Engineer
$217k-$303k
Deep Learning LLM Machine Learning NLP Python PyTorch TensorFlow
5 hours, 39 minutes ago
Torc

Senior, Machine Learning Engineer - 3D Perception

Torc 251-1K Road & Rail

Torc is hiring a Senior Machine Learning Engineer – 3D Perception to develop and deploy production perception models for autonomous trucks and improve Bird's Eye View understanding across its autonomy stack.

United States Full-time Senior Machine Learning Engineer
$177k-$213k
C++ Computer Vision Deep Learning Machine Learning Python PyTorch
5 hours, 54 minutes ago
U

Applied AI Engineer

Unframe Inc. 51-200 Technology, Information and Internet

Unframe is hiring an Applied AI Engineer to work with enterprise customers and internal teams on deploying AI-driven solutions that connect business problems to production systems.

Israel Mid Level AI Engineer Machine Learning Engineer
Machine Learning Python React TypeScript
1 day, 4 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers