RemoteLeaf Logo
Log in Sign up

AI/ML Research Engineer, LLM Post-Training & Evaluation

2 months ago
United States
Full-time
Mid Level
AI (Artificial Intelligence)
Software Development
CI/CD Hugging Face Machine Learning Python PyTorch TensorFlow
Apply Now
Innodata

Innodata

Innodata Inc. is a global leader in data engineering, offering end-to-end AI solutions and platforms for businesses worldwide, combining AI and human expertise to solve complex data challenges.

IT Services
1K-5K
Founded 1988
View All Jobs 24

Description

  • Lead or co-lead technically complex ML engineering projects from customer discussions through implementation and delivery.
  • Design, build, and improve LLM training and post-training pipelines, including data ingestion, preprocessing, fine-tuning, evaluation, and experiment tracking.
  • Implement and optimize evaluation systems for LLMs and multimodal models, including offline benchmarks and task-specific test harnesses.
  • Integrate human-in-the-loop and AI-augmented evaluation signals into model development workflows.
  • Build robust infrastructure and tooling for reproducible experimentation, metrics logging, and regression monitoring.
  • Diagnose model behavior and pipeline failures, including data issues, training instability, metric inconsistencies, and evaluation drift.
  • Collaborate with Language Data Scientists, Applied Research Scientists, data engineers, and customer technical stakeholders to translate evaluation frameworks into executable systems.
  • Contribute to internal research and platform development, including benchmark frameworks, evaluation tooling, and post-training workflow improvements.
  • Contribute to best practices and standards for LLM training, evaluation, and quality assurance across projects.
  • Mentor junior engineers and contribute to technical design reviews, documentation, and engineering rigor across the team.

Requirements

  • BS/MS/PhD in Computer Science, Machine Learning, AI, Applied Mathematics, or a related quantitative technical field; MS/PhD preferred.
  • 2-3 years of relevant industry or research engineering experience in ML/AI systems.
  • Hands-on experience with LLM training, fine-tuning, or post-training, including supervised fine-tuning, preference optimization, RLHF/RLAIF-style workflows, or task/domain adaptation.
  • Strong programming skills in Python and experience building production-quality ML code.
  • Experience with modern ML frameworks and tooling such as PyTorch, JAX, TensorFlow, Hugging Face, vLLM, or distributed training stacks.
  • Experience designing and implementing evaluation pipelines for LLM/ML systems, including metrics computation, dataset handling, and experiment comparisons.
  • Strong understanding of data pipelines and ML systems engineering, including reproducibility, observability, and debugging.
  • Experience with large-scale distributed ML systems and performance optimization for training/evaluation workloads, preferably in GPU or accelerator environments.
  • Experience with large-scale data processing and workflow orchestration in support of model training and evaluation.
  • Ability to collaborate directly with technical stakeholders, including research scientists, ML engineers, data engineers, and customer technical leads.
  • Strong written and verbal communication skills, including the ability to explain complex technical tradeoffs to technical and non-technical audiences.
  • Experience training, fine-tuning, and evaluating transformer-based models.
  • Understanding of post-training workflows and model iteration loops.
  • Familiarity with inference-time considerations such as latency, throughput, and memory/performance tradeoffs.
  • Experience implementing automated evaluation pipelines and test harnesses.
  • Experience with experiment tracking, versioning, and reproducibility practices.
  • Ability to assess metric quality and ensure consistency across model comparisons.
  • Proficiency in Python and strong software engineering fundamentals.
  • Experience with data processing pipelines, storage formats, and scalable dataset workflows.
  • Familiarity with CI/CD, testing, and engineering quality practices for ML systems.

Benefits

  • Expected salary range of $80,000 to $175,000 USD per year, based on experience, skills, and qualifications.
  • Opportunity to work on LLM training, post-training, and evaluation systems for foundation model builders and leading labs.
  • Work with a cross-functional team spanning language data science, applied research, data engineering, and customer technical stakeholders.
  • Contribute to internal R&D efforts on benchmark datasets, evaluation frameworks, and reusable infrastructure.
  • Help shape best practices and standards for LLM training, evaluation, and quality assurance across projects.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Mindrift.ai: Be the “I” in AI

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking an experienced software development specialist for project-based work creating and evaluating datasets used to test AI coding agents.

France Part-time Lead AI (Artificial Intelligence) Backend Engineer
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
6 hours, 36 minutes ago
Weekday

Voice Actor: CX Agent Voice Cloning (Standard French)

Weekday 11-50 Construction & Engineering

One of our clients is hiring a remote French voice talent to record and evaluate speech samples for an internal CX AI Agent and next-generation text-to-speech technology.

Anywhere United States Contract Mid Level AI (Artificial Intelligence) Customer Support
$0k-$0k
6 hours, 36 minutes ago
Mindrift.ai: Be the “I” in AI

Senior Python Engineer - AI Coding Agent Evaluation (Freelance)

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking a project-based software specialist to design realistic coding tasks and evaluation criteria for AI coding agents in simulated developer environments.

United States Part-time Lead AI (Artificial Intelligence) Backend Engineer
Docker FastAPI JavaScript Kafka PostgreSQL Python React Redis TypeScript
6 hours, 36 minutes ago
FutureSight

Co-Founder & CEO - AI WealthTech

FutureSight 11-50 Internet Software & Services

FutureSight is looking for a Co-Founder & CEO to lead Weli AI, a B2B WealthTech venture that brings embedded AI intelligence into advisors’ existing tools to drive proactive outreach and AUM growth.

United States Full-time Executive AI (Artificial Intelligence) Business Development
CRM LLM
6 hours, 36 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers