Prompt Engineer (LLM Systems, Evals & Safety)

4 months, 1 week ago
Mid Level
Software Development
Webook

Webook

WEbook.com is a writing community platform empowering writers, shaping future literary stars, and transforming the publishing world.

Media
1-10
Founded 2007

Description

  • Author, refactor, and chain prompts for a range of LLM tasks.
  • Design and maintain system, tool, and policy instructions.
  • Create offline and online evaluation harnesses, including rubrics, golden sets, and metrics.
  • Build and manage prompt libraries with versioning, A/B testing, and telemetry.
  • Reduce hallucinations through verification, constrained decoding, and tool use.
  • Implement safety checks for jailbreaks, prompt injection, content policy, and PII handling.
  • Partner with engineers to integrate prompts into production features.
  • Own evaluation and continuous improvement for LLM feature quality, safety, and cost effectiveness.

Requirements

  • Demonstrated prompt design experience across multiple task types and models.
  • Experience building evaluation datasets and automated scoring for accuracy, faithfulness, utility, cost, and latency.
  • Familiarity with retrieval-augmented generation concepts and tool/function calling.
  • Strong scripting skills in Python or TypeScript for data preparation, evaluation, and analysis.
  • Ability to translate business goals into measurable prompt specifications.
  • Experience with LangChain, LLM orchestration, vector stores, and rerankers is a plus.
  • Knowledge of safety tooling and red-teaming techniques is a plus.
  • Experience with experiment platforms such as feature flags and A/B tests, as well as analytics, is a plus.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Maintenance Coordinator

Remote Raven 11-50 Professional Services

Mason is hiring an experienced remote Maintenance Coordinator to oversee AI-assisted work order and resident communication for a property management portfolio.

2 weeks ago

Prompt Engineer

Netomi 51-250 IT Services

Netomi is hiring a Prompt Engineer to develop, optimize, and evaluate AI prompts for enterprise customer experience solutions used by major global brands.

Agile Hugging Face LLM Machine Learning NLP Python PyTorch TensorFlow
3 weeks, 3 days ago

Biology Expert (PhD) - AI Safety

Weekday 11-50 Construction & Engineering

A client is seeking a PhD-level biologist for a remote AI safety project that evaluates and improves AI responses on complex biological sciences.

3 weeks, 4 days ago

Legal Knowledge Engineer

Modern Family Law 51-250 Specialized Consumer Services

Modern Family Law is hiring a Legal Knowledge Engineer to help design AI-native legal service delivery systems for family law operations.

Generative AI
4 weeks, 2 days ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers