RemoteLeaf Logo
Log in Sign up

Senior Engineering Manager, AI Quality & Governance

2 months, 1 week ago
United States
Full-time
Lead
Engineering Manager
Software Development
AWS LLM Python TypeScript
Apply Now
Mitratech

Mitratech

Mitratech specializes in automating processes through innovative solutions and services in compliance, risk management, enterprise HR, and legal operations, empowering organizations to effectively manage risks and scale for future growth.

Professional Services
1K-5K
Founded 1987
$3M raised
View All Jobs 12

Description

  • Own the AI evaluations and observability platform, including tracing, logs, dashboards, quality signals, and operational feedback loops for prompts, model calls, tool use, and user outcomes.
  • Design and operationalize automated and human-in-the-loop evaluation strategies that incorporate regression checks for quality, safety, latency, and cost.
  • Establish organization-wide standards for production-ready AI, including evaluation criteria, release gates, incident playbooks, and long-term quality metrics.
  • Build and operate AI guardrails and policy enforcement capabilities such as content controls, PII detection and redaction, audit logging, and workflow-level policy checks.
  • Translate governance and risk expectations into engineering systems and platform controls rather than static documentation.
  • Own platform-level SLOs and tier-two incident support for AI behavior issues while partnering with product teams as first-line owners of shipped features.
  • Act as an internal authority on AI quality and governance and participate in customer-facing discussions about quality, safety, observability, and governance posture.
  • Hire, mentor, and grow the team as the combined function evolves over time.
  • Write and review production code as a hands-on leader during the team’s early growth stage.

Requirements

  • 8+ years of software engineering experience, including meaningful experience leading engineers as a manager or technical lead.
  • Experience shipping production AI systems and working through the operational complexity they introduce.
  • Strong experience designing and operating production backend systems and APIs.
  • Demonstrated hands-on experience building, shipping, or operating production AI systems, ideally including LLM-powered or agentic workflows.
  • Experience with AI observability, evaluation, or debugging systems, such as LangSmith or LangFuse, or internally built equivalents.
  • Practical experience designing or operating AI guardrails such as content filtering, redaction, access controls, or controls around model or agent behavior.
  • Familiarity with AI governance and risk frameworks such as NIST AI RMF or ISO/IEC 42001.
  • Ability to engage thoughtfully with legal, compliance, and security stakeholders.
  • A strong bias toward hands-on execution, platform thinking, and creating paved roads for other teams.
  • Experience in legal technology, regulated SaaS, or other environments where auditability and defensibility matter (preferred).
  • Experience with privacy-sensitive systems and PII handling (preferred).
  • Experience participating in AI incident reviews, red-team exercises, or internal review boards for production AI systems (preferred).
  • Engineering environment primarily uses Python and TypeScript.
  • Platform operates across AWS and OCI.
  • Observability tooling is being established, and LangFuse experience is relevant but not required.

Benefits

  • Anticipated pay range of $210K–$230K annually USD.
  • Total compensation includes US employee benefits and annual bonus eligibility.
  • 100% remote work anywhere in the US.
  • Health, dental, and vision insurance.
  • 401(k) with employer match.
  • Unlimited PTO, 11 paid holidays, and 4 annual paid global wellness days off.
  • STD, LTD, and group life insurance.
  • Paid parental leave.
  • Pet insurance, FSA and HSA options, and an Employee Assistance Program.
  • Career advancement and professional development opportunities.
  • LinkedIn Learning platform.
  • Employee recognition programs.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Cohere Health

Senior Manager, Data Platform Engineering

Cohere Health 251-1K Insurance

Cohere Health is seeking a Senior Manager, Data Platform Engineering to lead its data platform team, strengthen engineering practices, and deliver a scalable, reliable data foundation supporting healthcare products and business growth.

United States Full-time Lead Data Engineer Engineering Manager
$175k-$195k
Apache Airflow Apache Spark AWS CI/CD Git System Design
7 hours, 8 minutes ago
Motional

Principal Technical Lead Manager

Motional 1K-5K Automotive

Motional is seeking a Principal Technical Lead Manager to lead its Web Visualization Tooling team, building visualization tools and user experiences that improve machine-learning workflows and autonomous-vehicle system development.

United States Full-time Lead Engineering Manager Technical Lead
$176k-$242k
Agile CI/CD Game Development Machine Learning Node.js Python React TypeScript UI Design UX Design Vue.js
7 hours, 53 minutes ago
TRAILD

Senior Engineering Manager - Platform

TRAILD 11-50 Internet Software & Services

Traild is seeking a Senior Engineering Manager to lead its Platform Engineering team, shaping the cloud infrastructure and developer platforms that support the company’s globally scaling finance automation products.

Australia Full-time Lead Engineering Manager Platform Engineer
CI/CD
8 hours, 23 minutes ago
Cologix

Manager, Software Engineering

Cologix 251-1K Diversified Telecommunication Services

Cologix is hiring a Software Engineering Manager in its Corporate Systems group to lead Salesforce and MuleSoft development, integrations, platform support, and modernization initiatives for its enterprise business systems.

United States Full-time Lead Engineering Manager
$120k-$150k
CRM Cybersecurity Salesforce Scrum
1 day, 7 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers