RemoteLeaf Logo
Log in Sign up

Applied Data Scientist, Finance AI Evaluation & Datasets

18 hours, 23 minutes ago
United States
Full-time
Senior
AI (Artificial Intelligence)
Data Science and Analytics
Generative AI Hugging Face LLM Pandas Python PyTorch Scikit-learn SQL
Apply Now
Innodata

Innodata

Innodata Inc. is a global leader in data engineering, offering end-to-end AI solutions and platforms for businesses worldwide, combining AI and human expertise to solve complex data challenges.

IT Services
1K-5K
Founded 1988
View All Jobs 14

Description

  • Translate customer goals into dataset specifications, taxonomies, rubrics, and acceptance criteria.
  • Design training and evaluation datasets across financial QA, filings analysis, credit and underwriting, fraud/AML investigation, and compliance workflows.
  • Work with unstructured and multimodal financial data such as PDFs, scanned statements, spreadsheets, charts, and call transcripts.
  • Design datasets and evaluations for retrieval-augmented and source-grounded systems, including evidence citation, faithfulness, freshness, and conflict resolution.
  • Evaluate agentic and workflow-integrated financial AI systems, including tool use, retrieval, transaction boundaries, escalation behavior, and safety controls.
  • Develop evaluation methodology beyond surface accuracy, including numerical consistency, hallucination rates, robustness, refusal/escalation behavior, and fairness.
  • Define sampling strategies, label schemas, adjudication workflows, and annotation guidelines with finance SMEs and Language Data Scientists.
  • Build statistical and ML tooling for dataset trustworthiness, including stratified sampling, bias analysis, leakage detection, and distribution shift checks.
  • Own data quality end-to-end, including PII handling, provenance tracking, versioning, and modality-specific QA checks.
  • Support customer discovery and proposals by scoping dataset programs, estimating annotation effort, and explaining methodology to stakeholders.

Requirements

  • 5+ years of data science experience, including at least 2+ years in financial services, fintech, banking, or a comparable regulated data environment.
  • Real working knowledge of financial data and workflows, including financial statements, SEC filings, transaction data, and related document types.
  • Hands-on experience with unstructured and multimodal financial data such as PDFs, scanned documents, spreadsheets, charts, or call transcripts.
  • Hands-on experience designing datasets for machine learning, including writing annotation guidelines, sizing cohorts, and setting quality thresholds.
  • Familiarity with LLM-based and multimodal financial AI workflows, including prompt design, rubric-based evaluation, RAG, and LLM-as-judge methods.
  • Strong Python and SQL skills, with comfort using pandas, scikit-learn, and familiarity with Hugging Face, PyTorch, or model APIs.
  • Statistical literacy, including sampling design, inter-annotator agreement metrics, confidence intervals, and interpreting results appropriately.
  • Solid understanding of financial-services privacy, compliance, and governance, including PII handling, GLBA or equivalent regimes, MNPI sensitivity, and regulated documentation.
  • A degree in statistics, data science, economics, finance, or a related quantitative field, or equivalent demonstrated experience.
  • Preferred experience with XBRL, ISO 20022, GAAP/IFRS reporting concepts, document AI, OCR/post-OCR quality, table and chart extraction, Weights & Biases, LangFuse, model risk management, fairness auditing, multilingual financial data, or published/open-source work in financial AI or model governance.

Benefits

  • Expected salary range of $150,000–$175,000 USD per year.
  • Opportunity to work on high-stakes financial AI systems for major enterprise customers.
  • Work on a cross-functional pod with technical architecture, research, engineering, and language data science partners.
  • Chance to contribute reusable internal IP such as taxonomies, evaluation rubrics, golden datasets, and methodology templates.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Booksy

Senior AI Business Analyst (6 months FTC) (m/f/d)

Booksy 251-1K Diversified Consumer Services

Booksy is seeking a Senior AI Business Analyst to help lead its global AI transformation program by mapping current operations, identifying high-value AI opportunities, and turning them into scalable workflows.

Poland Spain United Kingdom Contract Senior AI (Artificial Intelligence) Project Manager
Generative AI LLM
17 hours, 38 minutes ago
Mindrift.ai: Be the “I” in AI

Customer Support Specialist - Freelance AI Trainer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking English-proficient project-based contributors to test, evaluate, and improve AI systems through customer service and financial-services scenario work.

United Kingdom Part-time Junior AI (Artificial Intelligence) Customer Support
Up to $125k
17 hours, 38 minutes ago
Volga Partners

Earn $25 for Up to 20 Minutes Voice Recording Only - Standard American English

Volga Partners 51-250 Internet Software & Services

Volga Partners is recruiting remote participants for a one-time English voice biometrics research study that helps build AI speech technology.

United States Freelance Entry Level AI (Artificial Intelligence)
$0k-$0k
Android iOS macOS Monday.com
17 hours, 38 minutes ago
Mindrift.ai: Be the “I” in AI

Customer Support Specialist - Freelance AI Trainer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking project-based contributors to test, evaluate, and improve AI systems for customer-service scenarios in banking and financial services.

United States Part-time Junior AI (Artificial Intelligence) Customer Support
Up to $125k
17 hours, 38 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers