Principal AI Infrastructure Engineer

4 days, 22 hours ago
Full-time
Lead
DevOps and Infrastructure
Highrise AI

Highrise AI

One platform for developing, training, and deploying AI models.

Internet Software & Services
1-10

Description

  • Lead the bring-up of racked, powered, and cabled AI clusters through production readiness and customer handoff.
  • Establish and validate BIOS/BMC, firmware, GPU driver, operating system, and cluster-software baselines.
  • Deploy and optimize automated bare-metal provisioning, imaging, node configuration, and lifecycle workflows.
  • Configure and troubleshoot Linux, GPU, kernel, driver, accelerator, and host-level performance settings.
  • Integrate and validate Kubernetes and Slurm environments with platform, networking, and operations teams.
  • Execute NVIDIA Field Diagnostics, L11/L12 validation, burn-in, stability, and hardware-health qualification procedures.
  • Run and analyze NCCL tests, GPU benchmarks, and distributed workload performance across nodes, racks, and clusters.
  • Own acceptance criteria, test evidence, defect tracking, remediation verification, and commercial-readiness recommendations.
  • Serve as a Tier 3 escalation point for production incidents and lead root-cause analysis, vendor escalations, and corrective actions.
  • Develop diagnostic tooling, automation, runbooks, and operational handoff materials for support teams.

Requirements

  • Bachelor's degree in computer science, engineering, or a related field, or equivalent practical experience.
  • 8+ years of hands-on Linux systems engineering, HPC, cloud, or large-scale infrastructure experience, including senior-level troubleshooting.
  • Experience bringing GPU or HPC clusters into production and qualifying them against performance and reliability targets.
  • Expertise in Linux, Python, Bash, Ansible, bare-metal provisioning, configuration management, and systems diagnostics.
  • Hands-on experience with NVIDIA GPU platforms, NCCL, NVIDIA Field Diagnostics, and multi-node performance testing.
  • Practical experience with Kubernetes and Slurm.
  • Familiarity with high-speed Ethernet, InfiniBand/RDMA, and parallel or distributed storage.
  • Ability to troubleshoot issues across hardware, firmware, operating systems, and distributed workloads.
  • Strong documentation, testing, root-cause analysis, and cross-functional communication skills.
  • Ability to travel to data centers and support critical incidents or maintenance outside regular business hours.

Benefits

  • Hybrid or remote work arrangement, depending on location.
  • Opportunity to work on large-scale AI compute infrastructure alongside experienced engineers.
  • Opportunity to help define technical standards for a growing global AI infrastructure platform.
  • Exposure to cross-functional peers and leaders across technology, energy, and infrastructure.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

AI Infrastructure Deployment Engineer

Fundrise 51-200 Capital Markets

Fundrise is hiring a remote Electrical Engineer to assess and deploy modular GPU-compute infrastructure across its U.S. real estate portfolio, from site diligence through commissioning.

Transformers
22 hours, 34 minutes ago

Maximo Infrastructure Architect/Engineer

Max Accelerate Internet Software & Services

As an IBM Maximo Infrastructure Architect at MaxAccelerate Technology Group, you will lead end-to-end MAS 9 implementations for global clients while building secure infrastructure and integrations for AI-enabled asset management.

Ansible Argo CD AWS Azure Bash CI/CD DNS Docker Git GitHub Actions GitOps Grafana Helm Jenkins Kafka Linux Load Balancing Podman Prometheus Python SQL SQL Server Tekton Terraform TLS YAML
1 day, 10 hours ago

Founding Backend / Infrastructure Engineer

nahc.io 11-50 Professional Services

Join a seed-stage cross-border neobank and retail brokerage as a founding Backend Engineer, building the resilient financial systems that enable U.S. investors to trade and hold Asian equities.

Go Java Node.js Python Rust
1 day, 19 hours ago

Senior DevOps & Infrastructure Engineer

Turing 501-1000 Internet Software & Services

Turing is seeking a Senior DevOps & Infrastructure Engineer in Brazil to build, secure, and operate the multi-cloud infrastructure and developer platform supporting its AI and enterprise systems.

Ansible AWS Azure Chef CI/CD DNS Docker GCP GitHub Actions Go Jenkins Kubernetes Network Security Puppet Python Secrets Management Terraform WAF
2 days, 19 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers