RemoteLeaf Logo
Log in Sign up

Technical Solutions Architect, Evals & Fine-Tuning

18 hours, 24 minutes ago
United States
Full-time
Senior
Solutions Architect
Software Development
Hugging Face Machine Learning NLP Python PyTorch
Apply Now
Innodata

Innodata

Innodata Inc. is a global leader in data engineering, offering end-to-end AI solutions and platforms for businesses worldwide, combining AI and human expertise to solve complex data challenges.

IT Services
1K-5K
Founded 1988
View All Jobs 14

Description

  • Lead technical discovery with prospective and existing customers to understand model objectives, gaps, and constraints.
  • Design end-to-end solutions across the post-training stack, including SFT data curation, preference data collection, custom benchmarks, LLM-as-judge pipelines, human-in-the-loop evaluation, red teaming, and multimodal eval.
  • Architect engagements that combine Innodata’s platforms with its global SME workforce across languages and domains.
  • Author technical proposals, statements of work, solution diagrams, and pricing models in partnership with sales, delivery, and finance.
  • Run technical workshops, proofs of concept, and pilot designs to de-risk larger programs and demonstrate value quickly.
  • Serve as an ongoing technical advisor during delivery and keep solutions aligned with customer intent.
  • Feed customer feedback into Innodata’s R&D and product roadmap to influence benchmarks and fine-tuning approaches.
  • Stay current on evolving eval and post-training methods, including dynamic benchmarks, safety evals, SFT, RLHF, DPO, RLAIF, rejection sampling, and distillation.
  • Represent Innodata externally through customer reviews, conferences, and technical content.

Requirements

  • 7+ years of experience in applied ML, ML engineering, ML research, or technical solutions roles.
  • At least 2+ years focused specifically on LLM evaluation and/or post-training.
  • Hands-on experience fine-tuning LLMs, with SFT at minimum.
  • Experience with preference optimization methods such as RLHF, DPO, or KTO is strongly preferred.
  • Experience designing the data pipelines that support LLM fine-tuning.
  • Deep familiarity with LLM evaluation methodology, including benchmark limitations, custom benchmark construction, LLM-as-judge design, inter-annotator agreement, and human eval workflow design.
  • Strong fluency in Python and the modern LLM toolchain, including Hugging Face, PyTorch, vLLM, and evaluation frameworks such as lm-evaluation-harness or lighteval.
  • Excellent technical communication skills with the ability to work across research scientists and non-technical executives.
  • A consultative mindset with the ability to ask sharp questions, challenge assumptions, and own recommendations.
  • Bachelor’s or advanced degree in computer science, machine learning, computational linguistics, or a related field, or equivalent demonstrated experience.

Benefits

  • Expected salary range of $140,000–$160,000 USD per year, based on experience, skills, and qualifications.
  • Opportunity to work on high-impact AI evaluation and fine-tuning programs with leading model labs and enterprise teams.
  • Exposure to a global SME workforce across 85+ languages and domains.
  • Role includes customer-facing external representation at reviews, conferences, and through technical content.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Wittij Consulting

Senior Solution Architect (Security)

Wittij Consulting 11-50 Internet Software & Services

A fully remote architecture consultant role at Smart focused on leading enterprise and solution architecture for complex client initiatives such as security programs, mergers, modernizations, and cloud migrations.

United States Full-time Senior Solutions Architect
$125k-$185k
17 hours, 54 minutes ago
Resource Innovations

SaaS Configuration Lead

Resource Innovations 251-1K Renewable Electricity

Resource Innovations is hiring a Software Implementation Configuration Lead to lead enterprise SaaS implementations for utility clean energy programs and deliver client-specific solutions from configuration through launch and support.

United States Full-time Senior Solutions Architect
$75k-$126k
CRM CSS ERP HTML JavaScript REST API
18 hours, 9 minutes ago
Elastic

Solution Architect (Pre-sales)

Elastic 1K-5K Internet Software & Services

Elastic is hiring a Solution Architect (Pre-sales) to work with customers and partners on designing, implementing, and optimizing Elastic Stack solutions for search, observability, and security use cases.

Spain Full-time Mid Level Solutions Architect
AWS Azure Elasticsearch GCP Java Kibana Logstash Python
18 hours, 24 minutes ago
Tiger Analytics

Martech Solution/Enterprise Architect

Tiger Analytics 1K-5K Professional Services

Tiger Analytics is seeking a Martech Solution/Enterprise Architect to provide hands-on architecture and integration support for marketing technology consolidation, helping simplify the stack and align platform strategy with business priorities and AI-driven initiatives.

Canada Full-time Senior Solutions Architect
1 day, 17 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers