Prompt Systems Engineer

48 minutes ago
Full-time
Junior
Software Development
Noodle

Noodle

Noodle is a U.S. based organization that empowers academic excellence worldwide by providing tech-enabled strategies and services for higher education. They collaborate with universities to design solutions that align with each institution's unique vis...

Diversified Consumer Services
251-1K
Founded 2010
$86M raised

Description

  • Write, iterate, and maintain system prompts and instruction sets for AI agents across the student journey.
  • Translate educational and operational requirements into clear agent behavior, including persona, tone, scope constraints, fallback handling, and multi-turn logic.
  • Design prompt architectures for multi-step workflows and RAG-augmented agents that use program-specific knowledge bases.
  • Collaborate with engineers to configure and deploy agents through the AI orchestration platform.
  • Create learner experiences that involve three-way chat between learners, support staff, and AI assistants.
  • Build and maintain evaluation frameworks to measure accuracy, tone, hallucination rate, task completion, and learning objective alignment.
  • Use Langfuse to monitor prompt performance in production, identify regressions, and prioritize improvements.
  • Design red-teaming and adversarial testing protocols to uncover edge cases and failure modes before launch.
  • Establish prompt versioning practices and maintain a library of reusable prompt components.
  • Partner with university stakeholders and internal teams to design, test, and document agents and prompt best practices.

Requirements

  • 2+ years of experience designing and iterating prompts for LLM-powered applications in a production environment.
  • Deep familiarity with prompt patterns, including few-shot examples, chain-of-thought, system vs. user roles, output formatting constraints, tool-use prompting, and RAG integration.
  • Strong written communication skills.
  • Experience building and running prompt evaluations, including defining metrics, writing test cases, and interpreting results.
  • Comfort working across disciplines, from engineers to faculty members.
  • Experience with LLM observability tools such as Langfuse, Weights & Biases, or similar.
  • Familiarity with LTI and how AI tools integrate with LMS platforms like Moodle, Canvas, or Instructure.
  • Experience with voice agents or multimodal prompting.
  • Comfort reading and writing basic JavaScript or Python to support prompt automation and testing pipelines.
  • Knowledge of OpenAI, Anthropic, or other model provider APIs.
  • Ability to thrive in ambiguity and iterate quickly based on user feedback.
  • Must be legally authorized to live and work in the United States and reside in the United States for the duration of employment.

Benefits

  • Remote work option in the U.S. or access to a New York City office.
  • Computer and charger provided.
  • 401(k) with company match and equity opportunities.
  • Medical, dental, and vision insurance for employees and their families, including mental health benefits.
  • Employer-paid life and disability insurance.
  • Health Savings Account, Flexible Spending Accounts, and commuter benefits.
  • Paid parental leave, 4 weeks of paid vacation, 10 paid holidays, and paid sick leave.
  • Monthly mobile connectivity stipend, annual education stipend, wellness resources, and a free Calm subscription.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Sheratan-Prompt Optimization Korean (Korea)

Welo Global Professional Services

Welo Data is hiring a freelance remote Korean language specialist in South Korea to review and refine AI translation prompts for attraction and travel content.

1 hour, 27 minutes ago

Sheratan-Prompt Optimization Hebrew (Israel)

Welo Global Professional Services

Welo Data is hiring a remote freelance Hebrew language specialist to review and optimize AI translation prompts for attraction and travel content in the Israel market.

1 hour, 47 minutes ago

Sheratan-Prompt Optimization Argentinian (Spanish)

Welo Global Professional Services

Welo Data is hiring a freelance remote Argentinian Spanish language specialist to review and refine AI translation prompts for travel and attraction content.

2 hours, 42 minutes ago

Sheratan-Prompt Optimization Catalan (Spain)

Welo Global Professional Services

Welo Data is hiring a freelance remote language specialist in Catalan (Spain) to review and refine AI translation prompts for travel and attraction content, ensuring natural, fluent, and culturally relevant output.

3 hours, 7 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers