RemoteLeaf Logo
Log in Sign up

Project Lion - Lead Prompt Engineer - United States (Remote, Part-Time)

1 hour, 55 minutes ago
United States
Freelance
Lead
Prompt Engineer
Data Science and Analytics
LLM SQL
Apply Now
Welocalize

Welocalize

Welocalize provides translation and localization services to help businesses reach global audiences with precision and efficiency.

Professional Services
1K-5K
Founded 1997
$34M raised
View All Jobs 144

Description

  • Lead and manage the technical migration process from templates to LLM autoraters.
  • Use Automatic Prompt Generation (APG) tools to create baseline prompts for complex parent-child template clusters.
  • Run and supervise Automated Prompt Optimization (APO) workflows and identify deadlocks or plateaus in outputs.
  • Manually draft, test, and refine prompts for complex architectures, anti-patterns, and edge cases when tooling is unavailable or broken.
  • Design and refine manual prompt solutions for difficult edge-case scenarios.
  • Monitor shadowbot runs to ensure human and LLM rating disagreements are generated and tracked appropriately.
  • Run prompt versions against gold data to evaluate autorater quality against the human crowd baseline.
  • Calculate and report model quality metrics such as F1 score, precision, and recall.
  • Draft technical launch readiness documentation and final launch certification justifications.
  • Mentor junior engineers and help shape the strategy for ongoing AI system improvement.

Requirements

  • Native fluency in English.
  • Must be based in the United States.
  • Master’s or Doctorate degree in Computer Science, Data Science, Computational Linguistics, Human-Computer Interaction (HCI), Cognitive Science, or a related analytical field.
  • At least 7 years of experience as a Prompt Engineer.
  • Proven experience tuning LLMs for strict, structured outputs and complex classification tasks.
  • Familiarity with chain-of-thought and few-shot learning approaches.
  • Strong proficiency in identifying error patterns, analyzing model performance, and using SQL or other data analytics tools.
  • Ability to quickly learn and master proprietary tools with minimal supervision.
  • Excellent verbal and written communication skills.
  • Familiarity with enterprise-grade LLM interfaces such as the Goose API (preferred).
  • Experience in AI model evaluation, data science, computational linguistics, or software engineering (preferred).
  • Hands-on experience with Automated Prompt Optimization (APO) systems or tuning workflows (preferred).
  • Linguistic expertise, including understanding of semantics and logic (preferred).

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Agiloft

Legal Knowledge Engineer

Agiloft 51-250 Capital Markets

Agiloft is hiring a remote Legal Knowledge Engineer in Canada to help shape AI-powered contract lifecycle management products by translating commercial legal expertise into real-world AI workflows and product improvements.

Canada Full-time Senior Legal Counsel Prompt Engineer
Generative AI LLM MLOps
1 hour, 55 minutes ago
Agiloft

Legal Knowledge Engineer

Agiloft 51-250 Capital Markets

Agiloft is hiring a remote Legal Knowledge Engineer to help shape its AI-powered contract lifecycle management platform by applying legal expertise to real-world contract workflows and product development.

United States Full-time Mid Level Legal Counsel Prompt Engineer
Generative AI MLOps
10 hours, 40 minutes ago
Spotify

Content Designer, Personalization

Spotify Media

Spotify is hiring a Content Designer for its Personalization Design team to shape the language and content systems behind AI-driven music discovery experiences at global scale.

United States Full-time Mid Level Prompt Engineer
$119k-$171k
Generative AI LLM
6 days, 2 hours ago
Cynet Group

AI Prompt Engineering Lead (Agentic AI & Hiring Automation) - Remote

Cynet Group 251-1K Professional Services

Senior AI Prompt Engineering Lead at Cynet Corp to architect, govern, and optimize Agentic AI systems that autonomously recruit, evaluate, and interact with talent using production-grade LLM architectures.

India Contract Lead Prompt Engineer
1 month, 3 weeks ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers