RemoteLeaf Logo
Log in Sign up

Technical Solutions Architect, Evals & Fine-Tuning

3 weeks, 2 days ago
United States
Full-time
Senior
Solutions Architect
Software Development
Hugging Face Machine Learning NLP Python PyTorch
Apply Now
Innodata

Innodata

Innodata Inc. is a global leader in data engineering, offering end-to-end AI solutions and platforms for businesses worldwide, combining AI and human expertise to solve complex data challenges.

IT Services
1K-5K
Founded 1988
View All Jobs 24

Description

  • Lead technical discovery with prospective and existing customers to understand model objectives, gaps, and constraints.
  • Design end-to-end solutions across the post-training stack, including SFT data curation, preference data collection, custom benchmarks, LLM-as-judge pipelines, human-in-the-loop evaluation, red teaming, and multimodal eval.
  • Architect engagements that combine Innodata’s platforms with its global SME workforce across languages and domains.
  • Author technical proposals, statements of work, solution diagrams, and pricing models in partnership with sales, delivery, and finance.
  • Run technical workshops, proofs of concept, and pilot designs to de-risk larger programs and demonstrate value quickly.
  • Serve as an ongoing technical advisor during delivery and keep solutions aligned with customer intent.
  • Feed customer feedback into Innodata’s R&D and product roadmap to influence benchmarks and fine-tuning approaches.
  • Stay current on evolving eval and post-training methods, including dynamic benchmarks, safety evals, SFT, RLHF, DPO, RLAIF, rejection sampling, and distillation.
  • Represent Innodata externally through customer reviews, conferences, and technical content.

Requirements

  • 7+ years of experience in applied ML, ML engineering, ML research, or technical solutions roles.
  • At least 2+ years focused specifically on LLM evaluation and/or post-training.
  • Hands-on experience fine-tuning LLMs, with SFT at minimum.
  • Experience with preference optimization methods such as RLHF, DPO, or KTO is strongly preferred.
  • Experience designing the data pipelines that support LLM fine-tuning.
  • Deep familiarity with LLM evaluation methodology, including benchmark limitations, custom benchmark construction, LLM-as-judge design, inter-annotator agreement, and human eval workflow design.
  • Strong fluency in Python and the modern LLM toolchain, including Hugging Face, PyTorch, vLLM, and evaluation frameworks such as lm-evaluation-harness or lighteval.
  • Excellent technical communication skills with the ability to work across research scientists and non-technical executives.
  • A consultative mindset with the ability to ask sharp questions, challenge assumptions, and own recommendations.
  • Bachelor’s or advanced degree in computer science, machine learning, computational linguistics, or a related field, or equivalent demonstrated experience.

Benefits

  • Expected salary range of $140,000–$160,000 USD per year, based on experience, skills, and qualifications.
  • Opportunity to work on high-impact AI evaluation and fine-tuning programs with leading model labs and enterprise teams.
  • Exposure to a global SME workforce across 85+ languages and domains.
  • Role includes customer-facing external representation at reviews, conferences, and through technical content.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

ProArch

Technical Architect - Healthcare Platforms (FHIR | Angular | .NET | SQL Server)

ProArch 251-1K Internet Software & Services

A healthcare technology company is seeking a Solution/Technical Architect to assess, modernize, and scale a patient-care platform supporting growing user demand and FHIR-based interoperability.

United States Contract Lead Software Architect Solutions Architect
Angular Azure HIPAA Load Balancing Microservices .NET REST API SQL Server
24 minutes ago
Lingaro

Test - templatka właściwa na start

Lingaro 5K-10K IT Services

A consulting team is hiring a React/Next.js developer to build and maintain web applications for stakeholder-requested mobile web and desktop solutions while working across business and technical teams.

Poland Full-time Lead Frontend Engineer / Web Developer Solutions Architect
Agile GCP Next.js React
39 minutes ago
European Dynamics

Solution Architect

European Dynamics 251-1K IT Services

European Dynamics is hiring a Solution Architect to support a major European Institution in a remote EU-based assignment within a development team working closely with the client’s IT team.

Greece Europe Full-time Senior Solutions Architect
Agile Kafka PostgreSQL
39 minutes ago
Provectus

Solutions Architect (GenAI, Python/Data, AWS)

Provectus 251-1K Professional Services

A company is hiring a Senior AI/Cloud Engineer to lead client-facing delivery of production AI, LLM, and data solutions from discovery through implementation.

Poland Armenia Serbia Slovakia Slovenia Spain Ukraine Full-time Lead Solutions Architect
Apache Airflow Apache Spark AWS CI/CD Docker FastAPI Flask GCP GitHub Actions GitLab CI Go Kafka Kubernetes MLOps Node.js Python REST API Rust
54 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers