Bask Health

Bask Health

Bask Health is the compliant, accessible, and no-code platform for telehealth, providing EMR, payment processing, pharmacy fulfillment, website builder, and patient management system. It's the Shopify for E-Prescribing.

Internet Software & Services
1-10

Description

  • Design and implement evaluation workflows for AI-powered applications across the platform.
  • Build automated tests, datasets, and realistic scenarios for LLM outputs and AI application behavior.
  • Validate response accuracy, relevance, completeness, consistency, structured outputs, and schema compatibility.
  • Test retrieval-augmented generation, tool-calling behavior, permissions, tenant boundaries, and authorized actions.
  • Develop regression and comparison tests for prompts, model versions, configurations, and application-code changes.
  • Integrate evaluation checks into development, release, controlled rollout, and production-monitoring workflows.
  • Investigate failed evaluations, reproduce root issues, and verify that fixes resolve underlying problems.
  • Collaborate with AI, backend, product, and quality teams to define expected behavior and acceptance criteria.
  • Identify edge cases, prompt-injection risks, unsupported medical claims, and behavior that could affect clinical or pharmacy workflows.
  • Build reporting tools, document test coverage and limitations, and track recurring AI failure patterns.

Requirements

  • 3+ years of experience in software engineering, AI engineering, test automation, quality engineering, or a related technical field.
  • Experience developing automated tests, testing scripts, frameworks, or validation tools for production software.
  • Strong programming skills in Python, TypeScript, JavaScript, or comparable languages.
  • Familiarity with large language models, prompt engineering, common AI failure modes, and evaluating AI-generated responses.
  • Understanding of software testing methodologies, test-case design, regression testing, and release processes.
  • Experience with API testing, backend integrations, structured data, and application-output validation.
  • Understanding of authentication, authorization, secure application behavior, databases, and application data models.
  • Experience with Git, CI pipelines, logging, monitoring, and production troubleshooting.
  • Ability to debug failures, create reproducible test cases, document findings, and distinguish application defects from model limitations.
  • Experience with LLM evaluation, RAG systems, AI agents, evaluation tools such as LangSmith, DeepEval, Ragas, or promptfoo, and AI security testing is preferred.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

AI Trainer - Software Engineering (Remote, Full-Time) [MV001]

Smart Working Internet Software & Services

Smart Working is hiring a remote Technical AI Trainer to build and deliver practical internal training that helps software engineers apply AI across development, QA, and DevOps.

CI/CD Playwright
18 hours, 25 minutes ago

Clinical AI Quality Specialist

Hello Heart 1-10 Internet Software & Services

Hello Heart is seeking a Clinical AI Quality Specialist to evaluate, monitor, and improve the clinical safety and quality of Nia, its AI health assistant for cardiovascular care.

LLM
18 hours, 55 minutes ago

AI Artist

Magnopus 201-500 Media

Magnopus is hiring a freelance AI Artist to remotely create cinematic, photorealistic generative image and video content for an audience-focused storytelling project.

After Effects Generative AI Photoshop Unreal Engine
18 hours, 55 minutes ago

Prompt Qualtity Reviewers for Uzbek

Welo Global Professional Services

Welo Global is seeking a senior linguist reviewer to quality-assure native-language reasoning prompts for AI training, ensuring each item is accurate, complete, and appropriately challenging.

NLP
18 hours, 55 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers