RemoteLeaf Logo
Log in Sign up

QA Analyst - LOIS for Word

3 hours, 28 minutes ago
United States
Full-time
Junior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
CI/CD LLM UX Design
Apply Now
Filevine

Filevine

Filevine is a top legal tech company revolutionizing legal work with AI-powered case management software, empowering law firms to streamline operations and enhance client services.

Specialized Consumer Services
251-1K
Founded 2015
$226M raised
View All Jobs 36

Description

  • Define and own the test strategy, QA process, and QA playbook for the product.
  • Partner with product and engineering early to move QA upstream before features ship.
  • Evaluate AI output quality for accuracy, citation reliability, and appropriately scoped reasoning.
  • Design and maintain evaluation sets that handle non-deterministic output and catch regressions over time.
  • Test AI-generated document edits for accuracy and formatting fidelity across different document structures.
  • Build automated guardrails integrated into CI/CD for critical product flows.
  • Write structured, reproducible bug reports backed by evidence.
  • Partner with legal domain experts to translate customer workflows into test coverage.
  • Compare behavior across product surfaces and identify discrepancies.
  • Provide product and design-minded feedback on how workflows could be improved.

Requirements

  • 2+ years of QA experience, including ownership of test strategy for a product or major feature area.
  • Experience building QA process and automation from scratch in a fast-moving environment.
  • Comfort operating in high ambiguity and testing quickly as features ship.
  • Strong analytical and problem-solving skills with attention to functional correctness and AI output quality.
  • Ability to reason about non-deterministic systems and define what 'correct' means when there is no single right answer.
  • Excellent written communication skills for clear, concise, actionable bug reports.
  • Experience with bug tracking tools and test management workflows.
  • Experience testing AI or LLM-based products, especially for output accuracy and evaluation design, preferred.
  • CI/CD experience setting up pipelines and release gates for automated tests, preferred.
  • A design or UX sensibility and/or prior experience in legal technology or reading legal contracts, preferred.
  • Programming experience beyond test automation, preferred.

Benefits

  • Salary range of $105,000 to $150,000.
  • Medical, dental, and vision insurance for full-time employees.
  • Competitive and fair pay.
  • Maternity and paternity leave for full-time employees.
  • Short- and long-term disability coverage.
  • Opportunity to learn from a dedicated leadership team.
  • Top-of-the-line company swag.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Mindrift.ai: Be the “I” in AI

Biology & Python Expert - Freelance AI Trainer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking biology specialists for project-based work designing and testing computational biology problems that challenge frontier AI models using specialized scientific tools.

United Kingdom Ireland Mexico Costa Rica Australia New Zealand Part-time Junior AI (Artificial Intelligence)
Linux Python
17 minutes ago
Mindrift.ai: Be the “I” in AI

Chemistry & Python Expert - Freelance AI Trainer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking chemistry specialists for project-based work designing and testing computational chemistry tasks that challenge frontier AI models using specialized simulation and quantum chemistry tools.

India Malaysia Philippines Singapore South Africa Part-time Junior AI (Artificial Intelligence)
Up to $0k
Linux Python
1 hour ago
Mindrift.ai: Be the “I” in AI

Freelance Agent Evaluation Engineer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking experienced technical contributors for a project-based role evaluating AI coding agents by designing realistic developer tasks, test suites, and edge-case scenarios for leading tech companies.

Spain Part-time Senior AI (Artificial Intelligence) QA Engineer
Docker FastAPI GitHub Actions JavaScript Kafka Linux PostgreSQL Pytest Python React Redis REST API TypeScript
1 hour, 41 minutes ago
Mindrift.ai: Be the “I” in AI

Biology & Python Expert - Freelance AI Trainer

Mindrift.ai: Be the “I” in AI Internet Software & Services

Mindrift is seeking biology specialists to create and calibrate project-based computational biology tasks that challenge frontier AI models using specialized scientific tools in a part-time, non-permanent setting.

Argentina Brazil Chile Colombia Part-time Junior AI (Artificial Intelligence)
$35k-$35k
Linux Python
1 hour, 42 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers