RemoteLeaf Logo
Log in Sign up

Applied Data Scientist, Finance AI Evaluation & Datasets

3 weeks, 2 days ago
United States
Full-time
Senior
AI (Artificial Intelligence)
Data Science and Analytics
Generative AI Hugging Face LLM Pandas Python PyTorch Scikit-learn SQL
Apply Now
Innodata

Innodata

Innodata Inc. is a global leader in data engineering, offering end-to-end AI solutions and platforms for businesses worldwide, combining AI and human expertise to solve complex data challenges.

IT Services
1K-5K
Founded 1988
View All Jobs 24

Description

  • Translate customer goals into dataset specifications, taxonomies, rubrics, and acceptance criteria.
  • Design training and evaluation datasets across financial QA, filings analysis, credit and underwriting, fraud/AML investigation, and compliance workflows.
  • Work with unstructured and multimodal financial data such as PDFs, scanned statements, spreadsheets, charts, and call transcripts.
  • Design datasets and evaluations for retrieval-augmented and source-grounded systems, including evidence citation, faithfulness, freshness, and conflict resolution.
  • Evaluate agentic and workflow-integrated financial AI systems, including tool use, retrieval, transaction boundaries, escalation behavior, and safety controls.
  • Develop evaluation methodology beyond surface accuracy, including numerical consistency, hallucination rates, robustness, refusal/escalation behavior, and fairness.
  • Define sampling strategies, label schemas, adjudication workflows, and annotation guidelines with finance SMEs and Language Data Scientists.
  • Build statistical and ML tooling for dataset trustworthiness, including stratified sampling, bias analysis, leakage detection, and distribution shift checks.
  • Own data quality end-to-end, including PII handling, provenance tracking, versioning, and modality-specific QA checks.
  • Support customer discovery and proposals by scoping dataset programs, estimating annotation effort, and explaining methodology to stakeholders.

Requirements

  • 5+ years of data science experience, including at least 2+ years in financial services, fintech, banking, or a comparable regulated data environment.
  • Real working knowledge of financial data and workflows, including financial statements, SEC filings, transaction data, and related document types.
  • Hands-on experience with unstructured and multimodal financial data such as PDFs, scanned documents, spreadsheets, charts, or call transcripts.
  • Hands-on experience designing datasets for machine learning, including writing annotation guidelines, sizing cohorts, and setting quality thresholds.
  • Familiarity with LLM-based and multimodal financial AI workflows, including prompt design, rubric-based evaluation, RAG, and LLM-as-judge methods.
  • Strong Python and SQL skills, with comfort using pandas, scikit-learn, and familiarity with Hugging Face, PyTorch, or model APIs.
  • Statistical literacy, including sampling design, inter-annotator agreement metrics, confidence intervals, and interpreting results appropriately.
  • Solid understanding of financial-services privacy, compliance, and governance, including PII handling, GLBA or equivalent regimes, MNPI sensitivity, and regulated documentation.
  • A degree in statistics, data science, economics, finance, or a related quantitative field, or equivalent demonstrated experience.
  • Preferred experience with XBRL, ISO 20022, GAAP/IFRS reporting concepts, document AI, OCR/post-OCR quality, table and chart extraction, Weights & Biases, LangFuse, model risk management, fairness auditing, multilingual financial data, or published/open-source work in financial AI or model governance.

Benefits

  • Expected salary range of $150,000–$175,000 USD per year.
  • Opportunity to work on high-stakes financial AI systems for major enterprise customers.
  • Work on a cross-functional pod with technical architecture, research, engineering, and language data science partners.
  • Chance to contribute reusable internal IP such as taxonomies, evaluation rubrics, golden datasets, and methodology templates.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Launch Potato

Lead Data Scientist, AdTech

Launch Potato 51-250 Professional Services

Launch Potato is hiring a Senior Applied Data Scientist to own end-to-end modeling for a priority vertical focused on driving ROAS, revenue, and media efficiency across its digital media portfolio.

United States Full-time Lead Data Scientist
$175k-$200k
AWS Deep Learning Digital Marketing Facebook Ads Google Ads LLM Looker Machine Learning Python Reinforcement Learning SQL
9 minutes ago
Launch Potato

Lead Data Scientist, AdTech

Launch Potato 51-250 Professional Services

Launch Potato is hiring a Senior Applied Data Science professional to own end-to-end modeling for a priority revenue vertical, starting with insurance and advertiser quality, with the goal of improving ROAS and media efficiency.

Colombia Lead Data Scientist
AWS Deep Learning Digital Marketing Facebook Ads Google Ads LLM Looker Machine Learning Python Reinforcement Learning SQL
24 minutes ago
Duolingo

Senior Data Science Manager, User Growth

Duolingo 251-1K Diversified Consumer Services

Duolingo is hiring a Senior Data Science Manager to lead the Growth pillar’s data science function, guiding analytics for user growth decisions across a rapidly scaling product and business.

United States Full-time Lead Data Scientist
$204k-$306k
Machine Learning Python R SQL Statistics
24 minutes ago
Launch Potato

Lead Applied Scientist, Marketing

Launch Potato 51-250 Professional Services

Launch Potato is hiring a Senior Applied Data Scientist to own end-to-end modeling for priority growth verticals, starting with Insurance and Advertiser Quality, with the goal of improving ROAS and revenue performance.

United States Full-time Senior Data Scientist
$175k-$200k
AWS Deep Learning Digital Marketing Facebook Ads Google Ads LLM Looker Machine Learning Python Reinforcement Learning SQL
24 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers