Senior Prompt Engineer - Data Science & Quality Analysis (India)

9 hours, 33 minutes ago
Senior
Data Science and Analytics
ItsaCheckmate

ItsaCheckmate

ItsaCheckmate simplifies digital orders and menu management for restaurants by integrating with various platforms directly into the POS, eliminating the need for tablets and manual transfers. Operators benefit from increased order volume, reduced error...

Hotels, Restaurants & Leisure
51-250
Founded 2016
$3M raised

Description

  • Design, test, and optimize LLM prompts for conversational AI, text classification, and structured data extraction tasks.
  • Build evaluation pipelines to assess prompt performance using quantitative metrics, human-in-the-loop feedback, and business KPIs.
  • Run prompt experiments and regression tests to maintain stability, accuracy, and safety as models change.
  • Collaborate with Machine Learning, Product, and Operations teams to turn business objectives into scalable prompt-engineering strategies.
  • Use Python and SQL to analyze model outputs, identify anomalies, and automate quality checks.
  • Document best practices and contribute to internal frameworks for prompt evaluation and continuous improvement.
  • Communicate findings to both technical and non-technical stakeholders to support data-driven decisions.
  • Translate requirements into prompt-engineering solutions that improve model accuracy, efficiency, and usability.

Requirements

  • B.S. or higher in a quantitative discipline such as Data Science, Computer Science, Engineering, or a related field, or in a field relevant to language models such as Linguistics, Philosophy, or Cognitive Science.
  • 5+ years of relevant experience with a B.S. degree, or 3+ years of experience with a Master’s degree.
  • Demonstrated proficiency in Python for automation, evaluation, and experimentation with LLM workflows.
  • Proven experience in prompt engineering and working with LLMs such as GPT-4, Claude, Gemini, and LLaMA.
  • Proficiency in Python and SQL for data analysis, evaluation scripting, and workflow automation.
  • Strong background in A/B testing, statistical analysis, and performance metric evaluation.
  • Familiarity with prompt-evaluation tools such as LangFuse or Galileo, and Weights and Biases for experiment management and regression testing.
  • Deep understanding of advanced prompting techniques, including few-shot prompting, reasoning-based prompting, multi-turn dialogue design, agentic orchestration, and DSPy/AdaFlow-style programmatic prompting frameworks.
  • Experience applying CO-STAR and TIDD-EC! prompting frameworks for structured reasoning, instruction design, and context control in production-grade LLM systems.
  • Excellent requirement-elicitation and communication skills, with the ability to translate business objectives into prompt-engineering solutions.
  • Academic or applied research experience related to language models, prompt engineering, or LLM-based systems is a strong plus.
  • Familiarity with LLM architectures, embeddings, and fine-tuning techniques is preferred.
  • Experience with LLM red-teaming, adversarial evaluation, or model safety testing is a plus.
  • Candidates must be flexible and work during US hours at least until 5 pm EST.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Prompt Engineer – Conversational AI

Outsourced Staff 11-50 Professional Services

Voice AI Solutions is hiring a remote Prompt Engineer to design and refine conversational AI prompts for voice and chat agents that support business operations across multiple industries.

LLM NLP
3 hours, 3 minutes ago

Shape the Future of AI — Hebrew Speakers in Israel

Welocalize 1K-5K Professional Services

Welo Data is building a freelance talent pool for future AI language projects in Tel Aviv and remote settings, where Hebrew speakers in Israel will help shape language data, annotation, evaluation, and prompt creation work.

3 hours, 48 minutes ago

Prompt Engineer (LLM Systems, Evals & Safety)

Webook 1-10 Media

webook.com is hiring a Prompt Engineer to design and improve LLM prompts, instructions, and tooling for production event-ticketing features in Saudi Arabia.

Python TypeScript
9 hours, 3 minutes ago

Prompt Engineer

Firework 251-1K Internet Software & Services

Firework is hiring a remote AI Prompt Engineer in Mexico to optimize prompts and evaluation workflows for large language models and multi-modal generative AI products.

Generative AI LLM Python
12 hours, 48 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers