RemoteLeaf Logo
Log in Sign up

Applied Research Scientist, LLM Evaluation & Post-Training

18 minutes ago
Canada
Full-time
Senior
Research Scientist
Software Development
Generative AI Hugging Face LLM Machine Learning Python PyTorch Statistics TensorFlow
Apply Now
Innodata

Innodata

Innodata Inc. is a global leader in data engineering, offering end-to-end AI solutions and platforms for businesses worldwide, combining AI and human expertise to solve complex data challenges.

IT Services
1K-5K
Founded 1988
View All Jobs 24

Description

  • Define and execute a research agenda focused on LLM evaluation and post-training.
  • Design rigorous experiments to study how evaluation methodologies impact fine-tuning and post-training outcomes.
  • Develop and validate evaluation frameworks for LLM and multimodal systems.
  • Design benchmark and task datasets, scoring methods, judge/model-assisted evaluation, human evaluation protocols, and robustness tests.
  • Lead research on advanced evaluation areas including long-context, cross-modal, and dynamic multi-turn evaluations.
  • Analyze model behavior and failure patterns and recommend improvements to models and evaluation design.
  • Collaborate with AI/ML Research Engineers to translate research methods into scalable evaluation and post-training pipelines.
  • Collaborate with Language Data Scientists to integrate human-in-the-loop and synthetic data/evaluation strategies.
  • Engage customer technical stakeholders to review methodologies and provide expert recommendations.
  • Produce technical documentation, research reports, and client-facing materials explaining methods, results, assumptions, and limitations.

Requirements

  • MS/PhD in Computer Science, Machine Learning, Statistics, Applied Mathematics, AI, or a related quantitative scientific field; PhD strongly preferred.
  • 5+ years of relevant applied research or research science experience in ML/AI, with substantial work in LLMs or foundation models.
  • Demonstrated experience with LLM evaluation, benchmarking, alignment, post-training, or model quality research.
  • Strong foundation in experimental design, statistical analysis, and scientific reasoning for ML systems.
  • Strong coding skills in Python for research experimentation and analysis.
  • Experience with modern ML tooling/frameworks such as PyTorch, Hugging Face, JAX, or TensorFlow.
  • Ability to evaluate human and automated evaluation methods and their tradeoffs in cost, reliability, validity, and scalability.
  • Experience designing reproducible evaluation studies and protocols across datasets, model versions, and runs.
  • Ability to collaborate with research scientists, ML engineers, data scientists, and customer technical stakeholders.
  • Strong communication skills with the ability to present nuanced technical conclusions clearly.

Benefits

  • Expected salary range of $245,000–$315,000 CAD per year.
  • Opportunity to work on cutting-edge GenAI research and evaluation methods.
  • Collaborative environment across research, engineering, and data operations.
  • Chance to contribute to internal benchmark datasets, evaluation frameworks, and reusable research assets.
  • Opportunity to influence customer solutions and internal platform innovation.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

xaira.com

Scientist II, Display Technologies

xaira.com Internet Software & Services

Xaira Therapeutics is hiring a protein biochemist to lead end-to-end phage display campaigns that generate high-quality antibody discovery data for AI-driven drug development.

United States Full-time Mid Level Research Scientist
$124k-$155k
Assembly Machine Learning Python R
18 minutes ago
Curai Health

Clinical Informaticist / Clinical AI Specialist

Curai Health 51-250 Health Care Providers & Services

Curai Health is hiring a Clinical Informaticist / Clinical AI Specialist to help shape AI-powered primary care workflows, safety standards, and research at the intersection of medicine and AI.

Anywhere Full-time Junior AI (Artificial Intelligence) Research Scientist
$240k-$300k
LLM
33 minutes ago
CloudLinux

Security Engineer - Node.js Proactive Defense (worldwide remote, work anywhere)

CloudLinux 51-250 IT Services

CloudLinux is hiring an engineer-researcher to build a new runtime protection product for Node.js web apps running on managed hosting, from design through production rollout.

Poland Portugal Romania Bulgaria Spain Czech Republic Full-time Lead Research Scientist Security Engineer
JavaScript Node.js PHP WAF WordPress
2 days, 1 hour ago
Anduril Industries

Senior Numerical Optimization Engineer

Anduril Industries 1K-5K Aerospace & Defense

Anduril Industries is hiring a Senior Numerical Optimization Engineer to build scheduling and optimization systems for its Maritime Digital Production team, supporting shipbuilding operations with rapid production replanning.

United States Full-time Senior Research Scientist Software Engineer
$191k-$253k
C++ CI/CD ERP IoT Java Microservices NumPy Pandas Python Rust SciPy
3 days ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers