RemoteLeaf Logo
Log in Sign up

Applied Data Scientist, Finance AI Evaluation & Datasets

2 months, 1 week ago
United States
Full-time
Senior
AI (Artificial Intelligence)
Data Science and Analytics
Generative AI Hugging Face LLM Pandas Python PyTorch Scikit-learn SQL
Apply Now
Innodata

Innodata

Innodata Inc. is a global leader in data engineering, offering end-to-end AI solutions and platforms for businesses worldwide, combining AI and human expertise to solve complex data challenges.

IT Services
1K-5K
Founded 1988
View All Jobs 23

Description

  • Translate customer goals into dataset specifications, taxonomies, rubrics, and acceptance criteria.
  • Design training and evaluation datasets across financial QA, filings analysis, credit and underwriting, fraud/AML investigation, and compliance workflows.
  • Work with unstructured and multimodal financial data such as PDFs, scanned statements, spreadsheets, charts, and call transcripts.
  • Design datasets and evaluations for retrieval-augmented and source-grounded systems, including evidence citation, faithfulness, freshness, and conflict resolution.
  • Evaluate agentic and workflow-integrated financial AI systems, including tool use, retrieval, transaction boundaries, escalation behavior, and safety controls.
  • Develop evaluation methodology beyond surface accuracy, including numerical consistency, hallucination rates, robustness, refusal/escalation behavior, and fairness.
  • Define sampling strategies, label schemas, adjudication workflows, and annotation guidelines with finance SMEs and Language Data Scientists.
  • Build statistical and ML tooling for dataset trustworthiness, including stratified sampling, bias analysis, leakage detection, and distribution shift checks.
  • Own data quality end-to-end, including PII handling, provenance tracking, versioning, and modality-specific QA checks.
  • Support customer discovery and proposals by scoping dataset programs, estimating annotation effort, and explaining methodology to stakeholders.

Requirements

  • 5+ years of data science experience, including at least 2+ years in financial services, fintech, banking, or a comparable regulated data environment.
  • Real working knowledge of financial data and workflows, including financial statements, SEC filings, transaction data, and related document types.
  • Hands-on experience with unstructured and multimodal financial data such as PDFs, scanned documents, spreadsheets, charts, or call transcripts.
  • Hands-on experience designing datasets for machine learning, including writing annotation guidelines, sizing cohorts, and setting quality thresholds.
  • Familiarity with LLM-based and multimodal financial AI workflows, including prompt design, rubric-based evaluation, RAG, and LLM-as-judge methods.
  • Strong Python and SQL skills, with comfort using pandas, scikit-learn, and familiarity with Hugging Face, PyTorch, or model APIs.
  • Statistical literacy, including sampling design, inter-annotator agreement metrics, confidence intervals, and interpreting results appropriately.
  • Solid understanding of financial-services privacy, compliance, and governance, including PII handling, GLBA or equivalent regimes, MNPI sensitivity, and regulated documentation.
  • A degree in statistics, data science, economics, finance, or a related quantitative field, or equivalent demonstrated experience.
  • Preferred experience with XBRL, ISO 20022, GAAP/IFRS reporting concepts, document AI, OCR/post-OCR quality, table and chart extraction, Weights & Biases, LangFuse, model risk management, fairness auditing, multilingual financial data, or published/open-source work in financial AI or model governance.

Benefits

  • Expected salary range of $150,000–$175,000 USD per year.
  • Opportunity to work on high-stakes financial AI systems for major enterprise customers.
  • Work on a cross-functional pod with technical architecture, research, engineering, and language data science partners.
  • Chance to contribute reusable internal IP such as taxonomies, evaluation rubrics, golden datasets, and methodology templates.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

ePlus

AI Presales Leader, Solutions Architect (Req #1411)

ePlus 1K-5K IT Services

The AI Presales Leader, Solutions Architect at ePlus leads enterprise AI adoption and customer advisory efforts, turning practical use cases into scalable solutions, business cases, and repeatable offerings.

United States Full-time Lead AI (Artificial Intelligence) Solutions Engineer
$160k-$200k
Cybersecurity Generative AI Machine Learning
3 hours ago
Veeam Software

AI & Automation Engineering Intern - Summer 2027

Veeam Software 1K-5K Internet Software & Services

Veeam’s 10-week Summer Internship Program seeks an AI & Automation Engineering Intern to build practical AI-assisted tools, agents, and workflow automations that help the global Center of Expertise scale its impact.

United States Internship Entry Level AI (Artificial Intelligence)
$42k-$52k
LLM Python
3 hours, 15 minutes ago
RWS Group

Generative Audio Evaluation - Spanish (Spain)

RWS Group 5K-10K Internet Software & Services

RWS is seeking remote, part-time Generative Audio Evaluation Specialists in Spain to assess AI-generated audio for video and text-to-audio tasks and provide structured feedback to improve the model.

Spain Freelance Entry Level AI (Artificial Intelligence)
$25k-$25k
3 hours, 30 minutes ago
RWS Group

Generative Audio Evaluation - Japanese (Japan)

RWS Group 5K-10K Internet Software & Services

RWS is seeking remote, part-time Generative Audio Evaluation Specialists in Japan to assess AI-generated audio and provide structured feedback that improves a generative audio model.

Japan Freelance Entry Level AI (Artificial Intelligence)
$35k-$35k
Generative AI
4 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers