RemoteLeaf Logo
Log in Sign up

Senior AI Engineer, Agentic Evaluation & V&V

5 days, 9 hours ago
United States, United Kingdom
Full-time
Senior
AI Engineer
Software Development
LLM Machine Learning MLflow Python Reinforcement Learning
Apply Now
Slingshot Aerospace

Slingshot Aerospace

Slingshot.space is a cutting-edge technology company that specializes in providing innovative solutions for space exploration, satellite communication, and aerospace engineering. We offer a range of products and services including satellite launch serv...

Diversified Telecommunication Services
51-250
Founded 2017
View All Jobs 14

Description

  • Extend and maintain Slingshot’s V&V SDK and evaluation framework for simulation-backed validation of agentic AI systems.
  • Design and implement agent-level and end-to-end evaluations, including benchmark scenarios, scoring logic, and experiment harnesses.
  • Build benchmark scenarios and tooling that measure planning, reasoning, and operational performance for autonomous mission planning systems.
  • Translate astrodynamics and mission-domain concepts into executable evaluation scenarios and simulation configurations.
  • Develop reusable SDK interfaces, adapters, and evaluation utilities that connect V&V systems, TALOS benchmarks, and agent workflows.
  • Define and apply metrics for capability evaluation, failure analysis, regression detection, and comparative benchmarking.
  • Partner with cross-functional teams to identify evaluation needs and improve coverage of critical capabilities.
  • Contribute to best practices for evaluating complex, autonomous AI systems.
  • Uphold strong engineering standards through testing, documentation, reproducibility, and maintainable system design.

Requirements

  • 6+ years of experience in software engineering, machine learning engineering, applied AI, or equivalent experience.
  • Strong Python engineering skills with experience building SDKs, libraries, or evaluation tooling.
  • Experience designing evaluation frameworks, benchmarks, metrics, or test harnesses for AI/ML systems.
  • Ability to analyze system behavior, identify failure modes, and evaluate performance in complex autonomous or semi-autonomous systems.
  • Familiarity with modern agent frameworks, orchestration patterns, or protocol-based integrations.
  • Experience working in cross-functional, multidisciplinary teams.
  • Strong written and verbal communication skills.
  • Bachelor’s degree in a relevant science or engineering field, or equivalent experience.
  • Must be a U.S. citizen and eligible to obtain and maintain a government security clearance.
  • Preferred: Experience in autonomous systems such as self-driving or ADAS, including perception, planning, simulation, or safety validation.
  • Preferred: Experience developing or evaluating agentic AI systems, including multi-step, tool-using, or autonomous workflows.
  • Preferred: Experience with reinforcement learning systems and simulation-based evaluation.
  • Preferred: Familiarity with benchmark design, experiment tracking, and trace-based evaluation workflows.
  • Preferred: Experience with orchestration frameworks such as LangGraph or similar tools.
  • Preferred: Knowledge of astrodynamics, orbital mechanics, or spacecraft mission planning.
  • Preferred: Experience translating mission or operational concepts into measurable evaluation scenarios.
  • Preferred: Familiarity with physics-based simulation, trajectory analysis, or space-domain modeling.
  • Preferred: Experience with observability and experiment tooling such as MLflow, Opik, or similar platforms.
  • Preferred: Experience transitioning advanced research systems into production environments.

Benefits

  • Remote, US-based work location.
  • Salary range of $150,000 to $250,000.
  • Full-time exempt classification.
  • Opportunity to work on mission-critical space safety and security applications.
  • Equal Opportunity Employer commitment to equity, diversity, and inclusion.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Coupa Software

AI Engineer, Evaluation - 11319

Coupa Software 1K-5K Internet Software & Services

Coupa is hiring a Mid-Level AI Engineer, Evaluation in Bangalore/remote to build the evaluation infrastructure that validates AI model quality across enterprise products.

India Mid Level AI Engineer
Machine Learning Python
52 minutes ago
CI&T

[Job 29147] Mid-level Developer Backend Java (Focus in AI)

CI&T 5K-10K Internet Software & Services

CI&T is hiring a mid-level Backend Java Developer in Brazil to build and modernize backend systems for an AI-powered chatbot platform using Java, Spring Boot, and cloud-native delivery practices.

Brazil Full-time Mid Level AI Engineer Backend Engineer
Angular CI/CD Docker Java JUnit Kafka Kubernetes LLM Microservices Mockito RabbitMQ REST API Spring Boot SQL
2 hours, 38 minutes ago
FutureSight

Product & Technical Co-founder (CPTO) - AI Compliance Platform

FutureSight 11-50 Internet Software & Services

FutureSight is seeking a technical co-founder and CPTO to lead product and engineering for an explainable AI compliance platform serving U.S. wealth management and other regulated firms.

Canada Full-time Lead AI Engineer Technical Product Manager
Encryption LLM
3 hours, 5 minutes ago
360Learning

AI Engineer

360Learning 251-1K Diversified Consumer Services

360Learning is hiring an AI Engineer to join its AI Squad in Paris or remotely and take ownership of building agentic AI features from prototype through production for its collaborative learning platform.

France Full-time Junior AI Engineer
Elasticsearch FastAPI Generative AI LLM Machine Learning Python
3 hours, 6 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers