RemoteLeaf Logo
Log in Sign up

Technical Solutions Architect, Evals & Fine-Tuning

2 months, 1 week ago
United States
Full-time
Senior
Solutions Architect
Software Development
Hugging Face Machine Learning NLP Python PyTorch
Apply Now
Innodata

Innodata

Innodata Inc. is a global leader in data engineering, offering end-to-end AI solutions and platforms for businesses worldwide, combining AI and human expertise to solve complex data challenges.

IT Services
1K-5K
Founded 1988
View All Jobs 23

Description

  • Lead technical discovery with prospective and existing customers to understand model objectives, gaps, and constraints.
  • Design end-to-end solutions across the post-training stack, including SFT data curation, preference data collection, custom benchmarks, LLM-as-judge pipelines, human-in-the-loop evaluation, red teaming, and multimodal eval.
  • Architect engagements that combine Innodata’s platforms with its global SME workforce across languages and domains.
  • Author technical proposals, statements of work, solution diagrams, and pricing models in partnership with sales, delivery, and finance.
  • Run technical workshops, proofs of concept, and pilot designs to de-risk larger programs and demonstrate value quickly.
  • Serve as an ongoing technical advisor during delivery and keep solutions aligned with customer intent.
  • Feed customer feedback into Innodata’s R&D and product roadmap to influence benchmarks and fine-tuning approaches.
  • Stay current on evolving eval and post-training methods, including dynamic benchmarks, safety evals, SFT, RLHF, DPO, RLAIF, rejection sampling, and distillation.
  • Represent Innodata externally through customer reviews, conferences, and technical content.

Requirements

  • 7+ years of experience in applied ML, ML engineering, ML research, or technical solutions roles.
  • At least 2+ years focused specifically on LLM evaluation and/or post-training.
  • Hands-on experience fine-tuning LLMs, with SFT at minimum.
  • Experience with preference optimization methods such as RLHF, DPO, or KTO is strongly preferred.
  • Experience designing the data pipelines that support LLM fine-tuning.
  • Deep familiarity with LLM evaluation methodology, including benchmark limitations, custom benchmark construction, LLM-as-judge design, inter-annotator agreement, and human eval workflow design.
  • Strong fluency in Python and the modern LLM toolchain, including Hugging Face, PyTorch, vLLM, and evaluation frameworks such as lm-evaluation-harness or lighteval.
  • Excellent technical communication skills with the ability to work across research scientists and non-technical executives.
  • A consultative mindset with the ability to ask sharp questions, challenge assumptions, and own recommendations.
  • Bachelor’s or advanced degree in computer science, machine learning, computational linguistics, or a related field, or equivalent demonstrated experience.

Benefits

  • Expected salary range of $140,000–$160,000 USD per year, based on experience, skills, and qualifications.
  • Opportunity to work on high-impact AI evaluation and fine-tuning programs with leading model labs and enterprise teams.
  • Exposure to a global SME workforce across 85+ languages and domains.
  • Role includes customer-facing external representation at reviews, conferences, and through technical content.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

GitLab

Solutions Architect, Manager - Public Sector

GitLab 1K-5K Internet Software & Services

GitLab is seeking a Manager of Public Sector Solutions Architects to lead a distributed team supporting public sector customers through technical pre-sales, strategic solution architecture, and customer-focused DevSecOps engagements.

United States Full-time Lead Solutions Architect
$183k-$308k
DevSecOps GitLab
3 hours, 15 minutes ago
Mendix

Solutions Architect - ICX

Mendix 1K-5K Internet Software & Services

Siemens Professional Services is seeking a Solution Architect to lead customer engagements across Intelligence Center X, SLC, Mendix, and other digital solutions, translating business outcomes into secure, scalable, and supportable architectures from discovery through operations.

United States Full-time Senior Solutions Architect
$147k-$293k
DevSecOps Machine Learning
4 hours, 15 minutes ago
Capco

Solution Architect – Financial Services

Capco 5K-10K Internet Software & Services

Capco is hiring a Solution Architect in the Czech Republic to support banks, insurers, and financial services clients by designing and guiding the implementation of future-proof technology architectures.

Czech Republic Full-time Senior Solutions Architect
Java Microservices REST API Spring Boot
1 day, 3 hours ago
Bounteous

Lead Platform Architect

Bounteous 1K-5K Internet Software & Services

Bounteous is seeking a remote Lead Architect to define enterprise solution architectures and technology strategies that enable scalable, secure, high-performing business transformation initiatives.

United States Full-time Lead Solutions Architect
Agile AWS Azure CI/CD CRM Digital Marketing Microservices Scrum SQL
1 day, 3 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers