RemoteLeaf Logo
Log in Sign up

Applied Data Scientist, Finance AI Evaluation & Datasets

1 month, 1 week ago
United States
Full-time
Senior
AI (Artificial Intelligence)
Data Science and Analytics
Generative AI Hugging Face LLM Pandas Python PyTorch Scikit-learn SQL
Apply Now
Innodata

Innodata

Innodata Inc. is a global leader in data engineering, offering end-to-end AI solutions and platforms for businesses worldwide, combining AI and human expertise to solve complex data challenges.

IT Services
1K-5K
Founded 1988
View All Jobs 24

Description

  • Translate customer goals into dataset specifications, taxonomies, rubrics, and acceptance criteria.
  • Design training and evaluation datasets across financial QA, filings analysis, credit and underwriting, fraud/AML investigation, and compliance workflows.
  • Work with unstructured and multimodal financial data such as PDFs, scanned statements, spreadsheets, charts, and call transcripts.
  • Design datasets and evaluations for retrieval-augmented and source-grounded systems, including evidence citation, faithfulness, freshness, and conflict resolution.
  • Evaluate agentic and workflow-integrated financial AI systems, including tool use, retrieval, transaction boundaries, escalation behavior, and safety controls.
  • Develop evaluation methodology beyond surface accuracy, including numerical consistency, hallucination rates, robustness, refusal/escalation behavior, and fairness.
  • Define sampling strategies, label schemas, adjudication workflows, and annotation guidelines with finance SMEs and Language Data Scientists.
  • Build statistical and ML tooling for dataset trustworthiness, including stratified sampling, bias analysis, leakage detection, and distribution shift checks.
  • Own data quality end-to-end, including PII handling, provenance tracking, versioning, and modality-specific QA checks.
  • Support customer discovery and proposals by scoping dataset programs, estimating annotation effort, and explaining methodology to stakeholders.

Requirements

  • 5+ years of data science experience, including at least 2+ years in financial services, fintech, banking, or a comparable regulated data environment.
  • Real working knowledge of financial data and workflows, including financial statements, SEC filings, transaction data, and related document types.
  • Hands-on experience with unstructured and multimodal financial data such as PDFs, scanned documents, spreadsheets, charts, or call transcripts.
  • Hands-on experience designing datasets for machine learning, including writing annotation guidelines, sizing cohorts, and setting quality thresholds.
  • Familiarity with LLM-based and multimodal financial AI workflows, including prompt design, rubric-based evaluation, RAG, and LLM-as-judge methods.
  • Strong Python and SQL skills, with comfort using pandas, scikit-learn, and familiarity with Hugging Face, PyTorch, or model APIs.
  • Statistical literacy, including sampling design, inter-annotator agreement metrics, confidence intervals, and interpreting results appropriately.
  • Solid understanding of financial-services privacy, compliance, and governance, including PII handling, GLBA or equivalent regimes, MNPI sensitivity, and regulated documentation.
  • A degree in statistics, data science, economics, finance, or a related quantitative field, or equivalent demonstrated experience.
  • Preferred experience with XBRL, ISO 20022, GAAP/IFRS reporting concepts, document AI, OCR/post-OCR quality, table and chart extraction, Weights & Biases, LangFuse, model risk management, fairness auditing, multilingual financial data, or published/open-source work in financial AI or model governance.

Benefits

  • Expected salary range of $150,000–$175,000 USD per year.
  • Opportunity to work on high-stakes financial AI systems for major enterprise customers.
  • Work on a cross-functional pod with technical architecture, research, engineering, and language data science partners.
  • Chance to contribute reusable internal IP such as taxonomies, evaluation rubrics, golden datasets, and methodology templates.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

SoFi

Staff Data Scientist

SoFi 1K-5K Capital Markets

SoFi Bank N.A. is seeking a Staff Data Scientist in San Francisco to develop and deploy machine learning models that improve credit underwriting and risk decisioning.

United States Full-time Senior Data Scientist
$224k-$246k
CI/CD Hive Machine Learning Python SQL
5 hours, 21 minutes ago
OURA

Senior Data Scientist - Member Experience

OURA 251-1K Health Care Providers & Services

Senior Data Scientist at Oura will partner with the Member Experience organization to build data systems, analytics, and models that improve support operations and member outcomes.

United States Full-time Senior Data Scientist
$148k-$203k
AWS Databricks dbt LLM Machine Learning NLP Python Snowflake SQL
5 hours, 36 minutes ago
GiveWell

Cause Prioritization Lead

GiveWell 51-250 Capital Markets

GiveWell is hiring a Cause Prioritization Lead to build and lead a new sub-team that identifies promising emerging cause areas and guides grantmaking decisions for its New Areas portfolio.

Anywhere United States Full-time Lead Data Scientist Program Manager
$280k-$308k
Agile
6 hours, 6 minutes ago
Capgemini

FBS - Data Scientist

Capgemini 100K+ Internet Software & Services

Data Scientist role at a major U.S. insurer focused on using advanced analytics, predictive modeling, and machine learning to improve contact center, workforce management, and customer experience operations.

Mexico Brazil Full-time Mid Level Data Scientist
Databricks GitHub Machine Learning MLOps Python SQL
6 hours, 36 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers