RemoteLeaf Logo
Log in Sign up

Senior Engineering Manager, AI Quality & Governance

1 month, 2 weeks ago
United States
Full-time
Lead
Engineering Manager
Software Development
AWS LLM Python TypeScript
Apply Now
Mitratech

Mitratech

Mitratech specializes in automating processes through innovative solutions and services in compliance, risk management, enterprise HR, and legal operations, empowering organizations to effectively manage risks and scale for future growth.

Professional Services
1K-5K
Founded 1987
$3M raised
View All Jobs 12

Description

  • Own the AI evaluations and observability platform, including tracing, logs, dashboards, quality signals, and operational feedback loops for prompts, model calls, tool use, and user outcomes.
  • Design and operationalize automated and human-in-the-loop evaluation strategies that incorporate regression checks for quality, safety, latency, and cost.
  • Establish organization-wide standards for production-ready AI, including evaluation criteria, release gates, incident playbooks, and long-term quality metrics.
  • Build and operate AI guardrails and policy enforcement capabilities such as content controls, PII detection and redaction, audit logging, and workflow-level policy checks.
  • Translate governance and risk expectations into engineering systems and platform controls rather than static documentation.
  • Own platform-level SLOs and tier-two incident support for AI behavior issues while partnering with product teams as first-line owners of shipped features.
  • Act as an internal authority on AI quality and governance and participate in customer-facing discussions about quality, safety, observability, and governance posture.
  • Hire, mentor, and grow the team as the combined function evolves over time.
  • Write and review production code as a hands-on leader during the team’s early growth stage.

Requirements

  • 8+ years of software engineering experience, including meaningful experience leading engineers as a manager or technical lead.
  • Experience shipping production AI systems and working through the operational complexity they introduce.
  • Strong experience designing and operating production backend systems and APIs.
  • Demonstrated hands-on experience building, shipping, or operating production AI systems, ideally including LLM-powered or agentic workflows.
  • Experience with AI observability, evaluation, or debugging systems, such as LangSmith or LangFuse, or internally built equivalents.
  • Practical experience designing or operating AI guardrails such as content filtering, redaction, access controls, or controls around model or agent behavior.
  • Familiarity with AI governance and risk frameworks such as NIST AI RMF or ISO/IEC 42001.
  • Ability to engage thoughtfully with legal, compliance, and security stakeholders.
  • A strong bias toward hands-on execution, platform thinking, and creating paved roads for other teams.
  • Experience in legal technology, regulated SaaS, or other environments where auditability and defensibility matter (preferred).
  • Experience with privacy-sensitive systems and PII handling (preferred).
  • Experience participating in AI incident reviews, red-team exercises, or internal review boards for production AI systems (preferred).
  • Engineering environment primarily uses Python and TypeScript.
  • Platform operates across AWS and OCI.
  • Observability tooling is being established, and LangFuse experience is relevant but not required.

Benefits

  • Anticipated pay range of $210K–$230K annually USD.
  • Total compensation includes US employee benefits and annual bonus eligibility.
  • 100% remote work anywhere in the US.
  • Health, dental, and vision insurance.
  • 401(k) with employer match.
  • Unlimited PTO, 11 paid holidays, and 4 annual paid global wellness days off.
  • STD, LTD, and group life insurance.
  • Paid parental leave.
  • Pet insurance, FSA and HSA options, and an Employee Assistance Program.
  • Career advancement and professional development opportunities.
  • LinkedIn Learning platform.
  • Employee recognition programs.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Vannevar Labs

Engineering Manager, AI Platform

Vannevar Labs 11-50 Aerospace & Defense

Vannevar is hiring a Senior Engineering Manager to lead its AI Platform organization, building the shared foundation for machine learning and agent systems that power national security workflows.

United States Full-time Lead Engineering Manager
AWS LLM Machine Learning PostgreSQL Python
20 hours, 3 minutes ago
Figma

Manager, Software Engineering - Data Platform

Figma 1K-5K Internet Software & Services

Figma is hiring a Data Platform engineering leader to scale the systems that power core business metrics, product analytics, and applied AI and experimentation across the company.

United States Full-time Lead Data Engineer Engineering Manager
$258k-$376k
Machine Learning
20 hours, 3 minutes ago
Formativ Group

Engineering Manager - Remote

Formativ Group Internet Software & Services

FormativGroup is hiring a hands-on Engineering Manager to lead a financial services technology team building and modernizing internal products for tax professionals while staying directly involved in delivery.

United States Full-time Senior Engineering Manager
$190k-$210k
Angular AWS AWS CDK Azure C# CI/CD GCP GitHub GitHub Actions .NET Node.js Pulumi Python React SonarQube Terraform
20 hours, 3 minutes ago
Eltropy

Engineering Manager – Unified Telephony Platform

Eltropy 51-250 Communications Equipment

Remote Engineering Manager role at a communications platform company leading the buildout of a cloud-native unified telephony platform for financial institutions.

India Full-time Senior Engineering Manager
CI/CD Go Microservices Python Twilio
1 day, 20 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers