Senior Manager – GenAI Performance & Analytics

2 hours, 4 minutes ago
Full-time
Senior
Data Science and Analytics
Netomi

Netomi

Netomi is an AI-first customer service platform revolutionizing customer support with industry-leading resolution rates and multilingual support.

IT Services
51-250
Founded 2015
$52M raised

Description

  • Analyze large volumes of chatbot and AI conversation data to identify performance gaps and improvement opportunities.
  • Define and track AI performance metrics such as resolution rate, containment rate, response quality, and customer satisfaction.
  • Build dashboards and reporting frameworks using SQL, Python, and BI tools such as Tableau.
  • Design frameworks to evaluate LLM response quality across factuality, groundedness, relevance, instruction adherence, tone, safety, and compliance.
  • Build scalable processes for automated LLM evaluation and human review workflows.
  • Identify systematic failure patterns in AI responses and recommend improvements.
  • Partner with AI and engineering teams to evaluate model upgrades, prompt changes, and new AI capabilities.
  • Design and analyze experiments to measure the impact of model improvements.
  • Translate conversational analytics into actionable recommendations for product and AI teams.
  • Lead and mentor a team of data analysts focused on AI performance and conversational analytics.

Requirements

  • 7–10 years of experience in data analytics, data science, or AI analytics.
  • Experience managing or mentoring analytics teams.
  • Experience analyzing conversational data, customer support data, or AI-driven products is a strong plus.
  • Strong SQL and data analysis skills.
  • Experience with BI tools such as Tableau or similar platforms.
  • Experience using Python for data analysis or experimentation.
  • Familiarity with machine learning or large language models (LLMs).
  • Experience evaluating AI or machine learning model performance (preferred).
  • Experience working with conversational AI platforms or chatbots (preferred).
  • Experience with experimentation frameworks and A/B testing (preferred).

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Epidemiology Director

N-Power Medicine 11-50 Biotechnology

N-Power Medicine is seeking a remote Epidemiology Director to support real-world evidence, external control arms, and clinical research initiatives across its data and technology platform.

Python R SQL
4 minutes ago

LLM Applied Data Scientist (RAG/ NLP)

Binance 5K-10K Capital Markets

Binance is hiring an LLM Applied Data Scientist in Taipei, Hong Kong, or Brisbane to improve large foundation model reasoning and retrieval systems for production AI applications.

C++ Computer Vision Deep Learning LLM Machine Learning NLP Python PyTorch Reinforcement Learning
4 minutes ago

Shape the Future of AI - Kazakh Talent Hub

Welocalize 1K-5K Professional Services

Welo Data, part of Welocalize, is seeking Kazakh contributors to join its global remote talent network for flexible AI data projects involving annotation, evaluation, and prompt creation.

LLM
19 minutes ago

Binance Accelerator Program - Quantitative Researcher

Binance 5K-10K Capital Markets

Binance’s Accelerator Program seeks a Quantitative Researcher to support quantitative trading work within its global blockchain and Web3 ecosystem.

Machine Learning Python
19 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers