RemoteLeaf Logo
Log in Sign up

AI/ML Research Engineer, LLM Post-Training & Evaluation

1 month, 1 week ago
United States
Full-time
Mid Level
AI (Artificial Intelligence)
Software Development
CI/CD Hugging Face Machine Learning Python PyTorch TensorFlow
Apply Now
Innodata

Innodata

Innodata Inc. is a global leader in data engineering, offering end-to-end AI solutions and platforms for businesses worldwide, combining AI and human expertise to solve complex data challenges.

IT Services
1K-5K
Founded 1988
View All Jobs 24

Description

  • Lead or co-lead technically complex ML engineering projects from customer discussions through implementation and delivery.
  • Design, build, and improve LLM training and post-training pipelines, including data ingestion, preprocessing, fine-tuning, evaluation, and experiment tracking.
  • Implement and optimize evaluation systems for LLMs and multimodal models, including offline benchmarks and task-specific test harnesses.
  • Integrate human-in-the-loop and AI-augmented evaluation signals into model development workflows.
  • Build robust infrastructure and tooling for reproducible experimentation, metrics logging, and regression monitoring.
  • Diagnose model behavior and pipeline failures, including data issues, training instability, metric inconsistencies, and evaluation drift.
  • Collaborate with Language Data Scientists, Applied Research Scientists, data engineers, and customer technical stakeholders to translate evaluation frameworks into executable systems.
  • Contribute to internal research and platform development, including benchmark frameworks, evaluation tooling, and post-training workflow improvements.
  • Contribute to best practices and standards for LLM training, evaluation, and quality assurance across projects.
  • Mentor junior engineers and contribute to technical design reviews, documentation, and engineering rigor across the team.

Requirements

  • BS/MS/PhD in Computer Science, Machine Learning, AI, Applied Mathematics, or a related quantitative technical field; MS/PhD preferred.
  • 2-3 years of relevant industry or research engineering experience in ML/AI systems.
  • Hands-on experience with LLM training, fine-tuning, or post-training, including supervised fine-tuning, preference optimization, RLHF/RLAIF-style workflows, or task/domain adaptation.
  • Strong programming skills in Python and experience building production-quality ML code.
  • Experience with modern ML frameworks and tooling such as PyTorch, JAX, TensorFlow, Hugging Face, vLLM, or distributed training stacks.
  • Experience designing and implementing evaluation pipelines for LLM/ML systems, including metrics computation, dataset handling, and experiment comparisons.
  • Strong understanding of data pipelines and ML systems engineering, including reproducibility, observability, and debugging.
  • Experience with large-scale distributed ML systems and performance optimization for training/evaluation workloads, preferably in GPU or accelerator environments.
  • Experience with large-scale data processing and workflow orchestration in support of model training and evaluation.
  • Ability to collaborate directly with technical stakeholders, including research scientists, ML engineers, data engineers, and customer technical leads.
  • Strong written and verbal communication skills, including the ability to explain complex technical tradeoffs to technical and non-technical audiences.
  • Experience training, fine-tuning, and evaluating transformer-based models.
  • Understanding of post-training workflows and model iteration loops.
  • Familiarity with inference-time considerations such as latency, throughput, and memory/performance tradeoffs.
  • Experience implementing automated evaluation pipelines and test harnesses.
  • Experience with experiment tracking, versioning, and reproducibility practices.
  • Ability to assess metric quality and ensure consistency across model comparisons.
  • Proficiency in Python and strong software engineering fundamentals.
  • Experience with data processing pipelines, storage formats, and scalable dataset workflows.
  • Familiarity with CI/CD, testing, and engineering quality practices for ML systems.

Benefits

  • Expected salary range of $80,000 to $175,000 USD per year, based on experience, skills, and qualifications.
  • Opportunity to work on LLM training, post-training, and evaluation systems for foundation model builders and leading labs.
  • Work with a cross-functional team spanning language data science, applied research, data engineering, and customer technical stakeholders.
  • Contribute to internal R&D efforts on benchmark datasets, evaluation frameworks, and reusable infrastructure.
  • Help shape best practices and standards for LLM training, evaluation, and quality assurance across projects.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

S

Director of Customer Service, Enablement and AI

Skylight / Skylight Frame 51-200 Consumer electronics

Skylight is hiring an AI Support Operations lead to own customer-facing and internal AI across its customer service function as the company scales its family-focused products.

United States Full-time Lead AI (Artificial Intelligence)
$150k-$185k
LLM
23 minutes ago
Coconut VA

🌴 AI Automations Specialist - Job Code: 1785267917380

Coconut VA 11-50 Professional Services

Coconut is hiring a part-time AI Automations Specialist to build and support remote automation systems, data workflows, and AI-driven business processes across multiple client priorities.

Philippines Part-time Mid Level AI (Artificial Intelligence) AI Engineer
$11k-$14k
CRM Git LLM NLP Power BI Python REST API SQL
23 minutes ago
Fidus Systems

Chief AI Officer

Fidus Systems 51-250 Professional Services

Fidus is seeking a Chief AI Officer to lead company-wide AI transformation, strategy, and adoption across its engineering services business.

United States Full-time Executive AI (Artificial Intelligence)
LLM Machine Learning MLOps Prototyping
23 minutes ago
Devsu

Revenue Operations Engineer (RevOps) | HubSpot, Automation & AI

Devsu 51-250 Internet Software & Services

RevOps Engineer – Growth at an unspecified company, responsible for reporting, CRM operations, GTM systems, and automation to improve data accuracy and support data-driven growth decisions.

Mexico Colombia Full-time Mid Level AI (Artificial Intelligence)
CRM Google Tag Manager HubSpot JavaScript JSON Power BI Python SEMrush SEO SQL
38 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers