RemoteLeaf Logo
Log in Sign up

Senior AI Engineer, Agentic Evaluation & V&V

1 month, 4 weeks ago
United States, United Kingdom
Full-time
Senior
AI Engineer
Software Development
LLM Machine Learning MLflow Python Reinforcement Learning
Apply Now
Slingshot Aerospace

Slingshot Aerospace

Slingshot.space is a cutting-edge technology company that specializes in providing innovative solutions for space exploration, satellite communication, and aerospace engineering. We offer a range of products and services including satellite launch serv...

Diversified Telecommunication Services
51-250
Founded 2017
View All Jobs 20

Description

  • Extend and maintain Slingshot’s V&V SDK and evaluation framework for simulation-backed validation of agentic AI systems.
  • Design and implement agent-level and end-to-end evaluations, including benchmark scenarios, scoring logic, and experiment harnesses.
  • Build benchmark scenarios and tooling that measure planning, reasoning, and operational performance for autonomous mission planning systems.
  • Translate astrodynamics and mission-domain concepts into executable evaluation scenarios and simulation configurations.
  • Develop reusable SDK interfaces, adapters, and evaluation utilities that connect V&V systems, TALOS benchmarks, and agent workflows.
  • Define and apply metrics for capability evaluation, failure analysis, regression detection, and comparative benchmarking.
  • Partner with cross-functional teams to identify evaluation needs and improve coverage of critical capabilities.
  • Contribute to best practices for evaluating complex, autonomous AI systems.
  • Uphold strong engineering standards through testing, documentation, reproducibility, and maintainable system design.

Requirements

  • 6+ years of experience in software engineering, machine learning engineering, applied AI, or equivalent experience.
  • Strong Python engineering skills with experience building SDKs, libraries, or evaluation tooling.
  • Experience designing evaluation frameworks, benchmarks, metrics, or test harnesses for AI/ML systems.
  • Ability to analyze system behavior, identify failure modes, and evaluate performance in complex autonomous or semi-autonomous systems.
  • Familiarity with modern agent frameworks, orchestration patterns, or protocol-based integrations.
  • Experience working in cross-functional, multidisciplinary teams.
  • Strong written and verbal communication skills.
  • Bachelor’s degree in a relevant science or engineering field, or equivalent experience.
  • Must be a U.S. citizen and eligible to obtain and maintain a government security clearance.
  • Preferred: Experience in autonomous systems such as self-driving or ADAS, including perception, planning, simulation, or safety validation.
  • Preferred: Experience developing or evaluating agentic AI systems, including multi-step, tool-using, or autonomous workflows.
  • Preferred: Experience with reinforcement learning systems and simulation-based evaluation.
  • Preferred: Familiarity with benchmark design, experiment tracking, and trace-based evaluation workflows.
  • Preferred: Experience with orchestration frameworks such as LangGraph or similar tools.
  • Preferred: Knowledge of astrodynamics, orbital mechanics, or spacecraft mission planning.
  • Preferred: Experience translating mission or operational concepts into measurable evaluation scenarios.
  • Preferred: Familiarity with physics-based simulation, trajectory analysis, or space-domain modeling.
  • Preferred: Experience with observability and experiment tooling such as MLflow, Opik, or similar platforms.
  • Preferred: Experience transitioning advanced research systems into production environments.

Benefits

  • Remote, US-based work location.
  • Salary range of $150,000 to $250,000.
  • Full-time exempt classification.
  • Opportunity to work on mission-critical space safety and security applications.
  • Equal Opportunity Employer commitment to equity, diversity, and inclusion.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Workato

Intern, Forward Deployed Engineering

Workato 251-1K IT Services

Workato is hiring a Forward Deployed Engineering intern to support AI-driven automation initiatives by helping build intelligent agents and enterprise workflow integrations on its Agentic AI platform.

India Internship Entry Level AI Engineer Software Engineer
JavaScript JSON LLM Python REST API Salesforce
14 hours, 45 minutes ago
Jobrack

Downeast Cider - AI Full Stack Developer

Jobrack 11-50 Professional Services

Downeast Cider is hiring an AI Full Stack Developer to become its first technical employee and build production-ready internal tools that improve operations across the business.

South Africa North Macedonia Poland Romania Serbia Georgia Contract Senior AI Engineer Full-stack Engineer
$60k-$72k
CRM GCP JavaScript NetSuite Python Shopify Snowflake SQL TypeScript
15 hours ago
Mindrift.ai: Be the “I” in AI

Freelance Chatbot Developer (WhatsApp / Telegram / Discord)

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is hiring a part-time freelance Bot Developer for the Tendem project to build and refine conversational bots and messaging-platform integrations in a hybrid AI and human workflow.

Costa Rica Jamaica Ecuador Uruguay Part-time Mid Level AI Engineer Backend Engineer
Up to $62k
Docker Node.js OAuth Python REST API Serverless
15 hours, 15 minutes ago
Assistantly

AI Automations Specialist (Systems & Agents Focus)

Assistantly 51-250 Professional Services

Assistantly is hiring an AI Automations Specialist to design and optimize AI-driven workflows and agents for growth-stage client teams, with the goal of reducing manual work and improving business efficiency.

Philippines Full-time Mid Level AI Engineer
JavaScript Python
15 hours, 15 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers