QA Analyst

3 hours, 28 minutes ago
Full-time
Mid Level
Quality Assurance and Testing
Inizio Evoke

Inizio Evoke

Inizio Evoke is a global health marketing and communications platform that focuses on transforming healthcare by leveraging data-driven insights and human-centric strategies to enhance patient support and market access solutions.

Professional Services
Founded 2000

Description

  • Design and execute QA strategies to validate AI-generated artifacts, recommendations, and decisions from autonomous and semi-autonomous agents.
  • Evaluate AI outputs for accuracy, relevance, completeness, hallucination risk, and alignment with context and knowledge sources.
  • Validate that agent outputs are attributable, traceable, reproducible, and consistent with execution and memory models.
  • Test end-to-end multi-agent workflows across the full lifecycle from intent through execution, artifacts, memory, and evolution.
  • Verify smooth promotion between ad-hoc requests and structured deliverables without loss of context or artifacts.
  • Develop test cases for plan creation, phase execution, step execution, retries, halts, and re-runs across human and AI agents.
  • Validate execution logs, artifacts, decisions, and audit events for completeness and consistency.
  • Test failure modes, edge cases, and recovery scenarios in agent execution paths.
  • Partner with Legal, Compliance, Data, Product, Business Analysis, and Engineering teams to embed quality and governance signals into the platform.
  • Define and track quality metrics and provide feedback to improve agent behavior, prompts, orchestration logic, and overall output quality.

Requirements

  • 4+ years of experience in Quality Assurance, Test Engineering, or Product Quality roles within complex software platforms.
  • Hands-on experience testing AI-enabled, data-driven, or workflow-oriented systems.
  • Proven ability to validate non-deterministic outputs where correctness is probabilistic rather than binary.
  • Strong understanding of QA methodologies including functional, integration, regression, and UAT testing.
  • Experience designing test cases for LLMs, generative AI systems, or intelligent automation.
  • Ability to reason about agent behavior, execution traces, and output quality across multi-step workflows.
  • Familiarity with Agile delivery environments and tools such as JIRA and Confluence, or similar tools.
  • Exceptional attention to detail and comfort working with complex system states and abstractions.
  • Experience with agentic AI frameworks, orchestration systems, or multi-agent workflows preferred.
  • Background in regulated industries such as healthcare, life sciences, or financial services preferred.
  • Exposure to LLM evaluation techniques, prompt testing, or AI output benchmarking preferred.
  • Familiarity with audit logging, compliance validation, or responsible AI practices preferred.

Benefits

  • US-based remote work.
  • Flexible work environment.
  • Competitive compensation with a base salary range of $70,000 to $105,000 USD.
  • Medical and dental coverage paid by the company.
  • 401(k) retirement plan.
  • Tuition reimbursement.
  • Flexible time off.
  • Personal support and tailored learning and development opportunities.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Benefit Operations Specialist

Capital Rx 251-1K Health Care Providers & Services

Judi Health is hiring a Benefit Operations Specialist to support integrated medical and pharmacy benefit programs by configuring plans, testing claims, and ensuring accurate, efficient implementations for clients.

Agile SQL
13 minutes ago

Senior Quality Assurance Auditor

Celcuity 11-50 Biotechnology

Celcuity is seeking a Senior Auditor to independently lead GxP audits and strengthen quality and compliance across its clinical-stage biotechnology operations.

42 minutes ago

Manager, CX Quality

Ethos Solutions 1-10 Professional Services

Ethos is hiring a Customer Experience Quality Assurance Manager to oversee support QA, training, and complaints management while improving service quality across a regulated life insurance technology platform.

58 minutes ago

1054- Business QA

GoFasti 51-250 Internet Software & Services

GoFasti is hiring an English-fluent Business QA in Latin America to work remotely on validating business requirements and ensuring the quality and stability of client software releases.

Power BI
58 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers