RemoteLeaf Logo
Log in Sign up

Senior Engineering Manager, AI Quality & Governance

3 weeks, 2 days ago
United States
Full-time
Lead
Engineering Manager
Software Development
AWS LLM Python TypeScript
Apply Now
Mitratech

Mitratech

Mitratech specializes in automating processes through innovative solutions and services in compliance, risk management, enterprise HR, and legal operations, empowering organizations to effectively manage risks and scale for future growth.

Professional Services
1K-5K
Founded 1987
$3M raised
View All Jobs 16

Description

  • Own the AI evaluations and observability platform, including tracing, logs, dashboards, quality signals, and operational feedback loops for prompts, model calls, tool use, and user outcomes.
  • Design and operationalize automated and human-in-the-loop evaluation strategies that incorporate regression checks for quality, safety, latency, and cost.
  • Establish organization-wide standards for production-ready AI, including evaluation criteria, release gates, incident playbooks, and long-term quality metrics.
  • Build and operate AI guardrails and policy enforcement capabilities such as content controls, PII detection and redaction, audit logging, and workflow-level policy checks.
  • Translate governance and risk expectations into engineering systems and platform controls rather than static documentation.
  • Own platform-level SLOs and tier-two incident support for AI behavior issues while partnering with product teams as first-line owners of shipped features.
  • Act as an internal authority on AI quality and governance and participate in customer-facing discussions about quality, safety, observability, and governance posture.
  • Hire, mentor, and grow the team as the combined function evolves over time.
  • Write and review production code as a hands-on leader during the team’s early growth stage.

Requirements

  • 8+ years of software engineering experience, including meaningful experience leading engineers as a manager or technical lead.
  • Experience shipping production AI systems and working through the operational complexity they introduce.
  • Strong experience designing and operating production backend systems and APIs.
  • Demonstrated hands-on experience building, shipping, or operating production AI systems, ideally including LLM-powered or agentic workflows.
  • Experience with AI observability, evaluation, or debugging systems, such as LangSmith or LangFuse, or internally built equivalents.
  • Practical experience designing or operating AI guardrails such as content filtering, redaction, access controls, or controls around model or agent behavior.
  • Familiarity with AI governance and risk frameworks such as NIST AI RMF or ISO/IEC 42001.
  • Ability to engage thoughtfully with legal, compliance, and security stakeholders.
  • A strong bias toward hands-on execution, platform thinking, and creating paved roads for other teams.
  • Experience in legal technology, regulated SaaS, or other environments where auditability and defensibility matter (preferred).
  • Experience with privacy-sensitive systems and PII handling (preferred).
  • Experience participating in AI incident reviews, red-team exercises, or internal review boards for production AI systems (preferred).
  • Engineering environment primarily uses Python and TypeScript.
  • Platform operates across AWS and OCI.
  • Observability tooling is being established, and LangFuse experience is relevant but not required.

Benefits

  • Anticipated pay range of $210K–$230K annually USD.
  • Total compensation includes US employee benefits and annual bonus eligibility.
  • 100% remote work anywhere in the US.
  • Health, dental, and vision insurance.
  • 401(k) with employer match.
  • Unlimited PTO, 11 paid holidays, and 4 annual paid global wellness days off.
  • STD, LTD, and group life insurance.
  • Paid parental leave.
  • Pet insurance, FSA and HSA options, and an Employee Assistance Program.
  • Career advancement and professional development opportunities.
  • LinkedIn Learning platform.
  • Employee recognition programs.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

GitLab

Manager, Engineering, Nonlinear Productivity (Friction Elimination & Solutions)

GitLab 1K-5K Internet Software & Services

GitLab is hiring a Manager, Engineering for its Nonlinear Productivity team to improve internal engineering output by identifying friction in the software development lifecycle and turning durable fixes into internal adoption or customer-facing products.

India Full-time Lead Engineering Manager Product Manager
CI/CD DevSecOps GitLab Prototyping
2 hours, 12 minutes ago
AHEAD

Manager of Software Engineering

AHEAD 1K-5K IT Services

AHEAD is hiring an engineering leader to run the Hatch and Unity platform team, guiding delivery of enterprise IT lifecycle products that support digital operations across cloud, data, AI, and security.

United States Full-time Lead Engineering Manager
$180k-$220k
Agile CI/CD Elixir GraphQL Phoenix REST API
2 hours, 27 minutes ago
Stripe

Machine Learning Engineering Manager - Fraud Detection

Stripe 5K-10K Diversified Financial Services

Stripe is seeking an engineering leader to lead its Risk Detection team in building merchant-facing risk management experiences that reduce friction, protect users, and support the company’s growth.

Anywhere Full-time Senior Engineering Manager Machine Learning Engineer
Machine Learning
2 hours, 42 minutes ago
SuperSummary

Software Engineering Manager

SuperSummary 11-50 Media

SuperSummary is seeking a Software Engineering Manager to lead a remote engineering team building and improving its literature platform and AI-powered product features.

Mexico Argentina Brazil Full-time Lead Engineering Manager
Flutter React Ruby on Rails System Design
2 hours, 42 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers