RemoteLeaf Logo
Log in Sign up

Principal Data Scientist - Agent Builder

6 hours, 37 minutes ago
Portugal
Full-time
Lead
AI Research Scientist
Data Science and Analytics
Elasticsearch NLP Pandas Python PyTorch
Apply Now
Elastic

Elastic

Elastic is a leading platform for search-powered solutions, providing real-time insights and making data usable for developers and enterprises worldwide.

Internet Software & Services
1K-5K
Founded 2010
View All Jobs 108

Description

  • Define the evaluation strategy for conversational and agentic search across offline and online methods.
  • Lead the design of quality metrics and decision frameworks for RAG, agents, tools, routing, model selection, and cost/latency trade-offs.
  • Build and improve retrieval and re-ranking approaches, including sparse and dense retrieval, vector search, query understanding, and context enrichment.
  • Translate experimental results into product and business decisions about models, routing, tool exposure, and agent customization.
  • Partner with engineering to productionize evaluation pipelines, telemetry, dashboards, CI guardrails, and regression detection.
  • Identify the highest-leverage quality gaps and influence the roadmap with practical recommendations and clear trade-offs.
  • Mentor data scientists and engineers on experiment design, evaluation methodology, and statistical rigor.
  • Document and communicate findings through docs, notebooks, PRs, dashboards, technical proposals, and cross-functional reviews.

Requirements

  • 8+ years of applied data science or machine learning experience.
  • Deep expertise in information retrieval, NLP, ranking, semantic search, RAG, or LLM-powered product experiences.
  • Proven experience leading evaluation for production AI/ML systems, including offline metrics, online experimentation, LLM-as-judge methods, groundedness, citation quality, and model comparison.
  • Experience influencing product and technical strategy in ambiguous or emerging domains.
  • Hands-on proficiency with Python, PyTorch/Transformers, Pandas, notebooks, reproducible experiments, versioned datasets, and clean code.
  • Strong understanding of retrieval systems, including dense and sparse retrieval, re-ranking, vector search, query understanding, and metrics such as nDCG, MRR, Recall@k, precision, and latency/cost trade-offs.
  • Experience working with engineering teams to move prototypes into production, including telemetry design, dashboards, CI guardrails, and quality regression tracking.
  • Practical Elasticsearch experience or experience with similar search and distributed data systems; ES|QL familiarity is a plus.
  • Excellent written and verbal communication skills for technical and non-technical stakeholders.
  • Collaborative, low-ego working style with strong mentoring ability.

Benefits

  • Base salary compensation with no variable compensation component.
  • Typical starting salary range of €67,000 to €106,000 EUR.
  • Competitive pay based on the work you do, not your previous salary.
  • Health coverage for you and your family in many locations.
  • Flexible locations and schedules for many roles.
  • Generous number of vacation days each year.
  • Up to 40 hours each year for volunteer projects.
  • Minimum of 16 weeks of parental leave.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Adswerve

Data Scientist 1

Adswerve 251-1K Media

Adswerve is hiring a Data Scientist 1 for its Tech Services team to work on client-facing data engineering and analytics solutions that improve marketing performance and business outcomes.

United States Full-time Entry Level Data Scientist
$95k-$115k
AWS Azure GCP Google Ads Google Analytics HTML JavaScript Machine Learning Python SQL
6 hours, 7 minutes ago
Adswerve

Sr. Data Scientist 3 (South Africa)

Adswerve 251-1K Media

Adswerve is seeking a Senior Data Scientist 3 to join its Technical Services team, leading client-facing data science and data engineering work that turns complex marketing data into actionable business solutions.

South Africa Full-time Lead Data Scientist
AWS Azure GCP Generative AI Google Analytics HTML JavaScript Machine Learning Python SQL Statistics
6 hours, 52 minutes ago
American Institutes for Research

Senior Economist, Healthcare Innovations

American Institutes for Research 1K-5K Professional Services

AIR is seeking a Senior Economist to join its Healthcare Innovations team, conducting policy-related research and evaluation studies on U.S. health care programs and public health policy.

United States Full-time Senior Data Scientist Program Manager
$130k-$160k
1 day, 6 hours ago
Pinterest

Staff Data Scientist, Ads Product

Pinterest 5K-10K Internet Software & Services

Pinterest is hiring a senior individual contributor to lead data-driven strategy and decision support for its Ads organization, connecting product, engineering, business, and finance teams around reliable insights and analytical excellence.

United States Full-time Lead Data Scientist
$165k-$339k
Looker Machine Learning Python SQL Statistics Tableau
1 day, 6 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers