RemoteLeaf Logo
Log in Sign up

Senior AI Engineer, Agentic Evaluation & V&V

1 hour, 24 minutes ago
United States, United Kingdom
Full-time
Senior
AI Engineer
Software Development
LLM Machine Learning MLflow Python Reinforcement Learning
Apply Now
Slingshot Aerospace

Slingshot Aerospace

Slingshot.space is a cutting-edge technology company that specializes in providing innovative solutions for space exploration, satellite communication, and aerospace engineering. We offer a range of products and services including satellite launch serv...

Diversified Telecommunication Services
51-250
Founded 2017
View All Jobs 14

Description

  • Extend and maintain Slingshot’s V&V SDK and evaluation framework for simulation-backed validation of agentic AI systems.
  • Design and implement agent-level and end-to-end evaluations, including benchmark scenarios, scoring logic, and experiment harnesses.
  • Build benchmark scenarios and tooling that measure planning, reasoning, and operational performance for autonomous mission planning systems.
  • Translate astrodynamics and mission-domain concepts into executable evaluation scenarios and simulation configurations.
  • Develop reusable SDK interfaces, adapters, and evaluation utilities that connect V&V systems, TALOS benchmarks, and agent workflows.
  • Define and apply metrics for capability evaluation, failure analysis, regression detection, and comparative benchmarking.
  • Partner with cross-functional teams to identify evaluation needs and improve coverage of critical capabilities.
  • Contribute to best practices for evaluating complex, autonomous AI systems.
  • Uphold strong engineering standards through testing, documentation, reproducibility, and maintainable system design.

Requirements

  • 6+ years of experience in software engineering, machine learning engineering, applied AI, or equivalent experience.
  • Strong Python engineering skills with experience building SDKs, libraries, or evaluation tooling.
  • Experience designing evaluation frameworks, benchmarks, metrics, or test harnesses for AI/ML systems.
  • Ability to analyze system behavior, identify failure modes, and evaluate performance in complex autonomous or semi-autonomous systems.
  • Familiarity with modern agent frameworks, orchestration patterns, or protocol-based integrations.
  • Experience working in cross-functional, multidisciplinary teams.
  • Strong written and verbal communication skills.
  • Bachelor’s degree in a relevant science or engineering field, or equivalent experience.
  • Must be a U.S. citizen and eligible to obtain and maintain a government security clearance.
  • Preferred: Experience in autonomous systems such as self-driving or ADAS, including perception, planning, simulation, or safety validation.
  • Preferred: Experience developing or evaluating agentic AI systems, including multi-step, tool-using, or autonomous workflows.
  • Preferred: Experience with reinforcement learning systems and simulation-based evaluation.
  • Preferred: Familiarity with benchmark design, experiment tracking, and trace-based evaluation workflows.
  • Preferred: Experience with orchestration frameworks such as LangGraph or similar tools.
  • Preferred: Knowledge of astrodynamics, orbital mechanics, or spacecraft mission planning.
  • Preferred: Experience translating mission or operational concepts into measurable evaluation scenarios.
  • Preferred: Familiarity with physics-based simulation, trajectory analysis, or space-domain modeling.
  • Preferred: Experience with observability and experiment tooling such as MLflow, Opik, or similar platforms.
  • Preferred: Experience transitioning advanced research systems into production environments.

Benefits

  • Remote, US-based work location.
  • Salary range of $150,000 to $250,000.
  • Full-time exempt classification.
  • Opportunity to work on mission-critical space safety and security applications.
  • Equal Opportunity Employer commitment to equity, diversity, and inclusion.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

NEORIS

AI Agents Engineer SR (Orquestación y Automatización)

NEORIS 5K-10K Internet Software & Services

NEORIS, ahora parte de EPAM, busca un/a AI Agents Engineer SR para trabajar 100% remoto desde Argentina en proyectos de transformación digital y automatización inteligente centrados en el SDLC.

Argentina Full-time Senior AI Engineer Prompt Engineer
54 minutes ago
Future

Applied AI Engineer

Future 251-1K Hotels, Restaurants & Leisure

Future is hiring an Applied AI Engineer to build and ship production AI features for its digital personal training platform, improving the product experience and business outcomes.

United States Full-time Senior AI Engineer Software Engineer
$215k-$250k
AWS AWS CDK Datadog LLM OpenTelemetry Python Terraform
1 hour, 25 minutes ago
Mindrift.ai: Be the “I” in AI

Low-Code Automation Engineer - Freelance AI Trainer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is hiring a part-time remote Low-Code Automation Engineer for the Tendem project to design and maintain scalable automation workflows in a hybrid AI and human environment.

Poland Romania Austria Czech Republic Greece Hungary Part-time Mid Level AI (Artificial Intelligence) AI Engineer
CRM JavaScript JSON Notion
1 hour, 41 minutes ago
Elastic

Lead GenAI Cloud Developer

Elastic 1K-5K Internet Software & Services

Elastic is hiring a Lead GenAI Cloud Developer to evolve ElasticGPT from a chat assistant into an agentic, task-executing platform that improves internal productivity through enterprise-grade AI systems.

Canada Full-time Lead AI Engineer Cloud Engineer
$85k-$126k
AWS Azure Computer Vision Confluence Docker Elasticsearch GCP Generative AI GitHub Hugging Face JIRA Kibana Kubernetes Logstash Microservices NLP OpenTelemetry Python PyTorch TensorFlow TypeScript Vertex AI
1 hour, 54 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers