RemoteLeaf Logo
Log in Sign up

Technical Solutions Architect, Evals & Fine-Tuning

1 month, 1 week ago
United States
Full-time
Senior
Solutions Architect
Software Development
Hugging Face Machine Learning NLP Python PyTorch
Apply Now
Innodata

Innodata

Innodata Inc. is a global leader in data engineering, offering end-to-end AI solutions and platforms for businesses worldwide, combining AI and human expertise to solve complex data challenges.

IT Services
1K-5K
Founded 1988
View All Jobs 24

Description

  • Lead technical discovery with prospective and existing customers to understand model objectives, gaps, and constraints.
  • Design end-to-end solutions across the post-training stack, including SFT data curation, preference data collection, custom benchmarks, LLM-as-judge pipelines, human-in-the-loop evaluation, red teaming, and multimodal eval.
  • Architect engagements that combine Innodata’s platforms with its global SME workforce across languages and domains.
  • Author technical proposals, statements of work, solution diagrams, and pricing models in partnership with sales, delivery, and finance.
  • Run technical workshops, proofs of concept, and pilot designs to de-risk larger programs and demonstrate value quickly.
  • Serve as an ongoing technical advisor during delivery and keep solutions aligned with customer intent.
  • Feed customer feedback into Innodata’s R&D and product roadmap to influence benchmarks and fine-tuning approaches.
  • Stay current on evolving eval and post-training methods, including dynamic benchmarks, safety evals, SFT, RLHF, DPO, RLAIF, rejection sampling, and distillation.
  • Represent Innodata externally through customer reviews, conferences, and technical content.

Requirements

  • 7+ years of experience in applied ML, ML engineering, ML research, or technical solutions roles.
  • At least 2+ years focused specifically on LLM evaluation and/or post-training.
  • Hands-on experience fine-tuning LLMs, with SFT at minimum.
  • Experience with preference optimization methods such as RLHF, DPO, or KTO is strongly preferred.
  • Experience designing the data pipelines that support LLM fine-tuning.
  • Deep familiarity with LLM evaluation methodology, including benchmark limitations, custom benchmark construction, LLM-as-judge design, inter-annotator agreement, and human eval workflow design.
  • Strong fluency in Python and the modern LLM toolchain, including Hugging Face, PyTorch, vLLM, and evaluation frameworks such as lm-evaluation-harness or lighteval.
  • Excellent technical communication skills with the ability to work across research scientists and non-technical executives.
  • A consultative mindset with the ability to ask sharp questions, challenge assumptions, and own recommendations.
  • Bachelor’s or advanced degree in computer science, machine learning, computational linguistics, or a related field, or equivalent demonstrated experience.

Benefits

  • Expected salary range of $140,000–$160,000 USD per year, based on experience, skills, and qualifications.
  • Opportunity to work on high-impact AI evaluation and fine-tuning programs with leading model labs and enterprise teams.
  • Exposure to a global SME workforce across 85+ languages and domains.
  • Role includes customer-facing external representation at reviews, conferences, and through technical content.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

540

Senior Solutions Architect

540 51-250 Internet Software & Services

540 is hiring a Senior Solutions Architect to lead the modernization of a mission-critical federal health data platform that integrates data across defense and federal systems.

United States Full-time Lead Solutions Architect
Apache Spark AWS Azure Databricks Git
6 hours, 6 minutes ago
Wasabi

Sr. Solutions Architect

Wasabi 251-1K IT Services

Wasabi Technologies is hiring a Sr. Solutions Architect in the EST zone to design and support cloud storage solutions for customers and partners in a large-scale S3-based environment.

United States Full-time Senior Solutions Architect
$140k-$170k
AWS Azure DNS GCP HTTP Linux TCP/IP TLS
6 hours, 21 minutes ago
540

Solutions Architect

540 51-250 Internet Software & Services

540 is hiring a Solutions Architect to help modernize a mission-critical federal health data platform that connects defense and federal systems across hybrid cloud and on-premises environments.

United States Full-time Mid Level Solutions Architect
Apache Spark AWS Azure Databricks Microservices
6 hours, 21 minutes ago
540

Associate Solutions Architect

540 51-250 Internet Software & Services

540 is hiring an Associate Solutions Architect to support a federal health data modernization program by helping design and document modern data platform solutions across cloud, on-premises, and government-hosted environments.

United States Full-time Entry Level Solutions Architect
Apache Spark AWS Azure Databricks Git Java Python SQL Tableau
6 hours, 21 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers