RemoteLeaf Logo
Log in Sign up

QA Analyst - LOIS for Word

3 months, 2 weeks ago
United States
Full-time
Junior
AI (Artificial Intelligence)
Artificial Intelligence and Machine Learning
CI/CD LLM UX Design
Apply Now
Filevine

Filevine

Filevine is a top legal tech company revolutionizing legal work with AI-powered case management software, empowering law firms to streamline operations and enhance client services.

Specialized Consumer Services
251-1K
Founded 2015
$226M raised
View All Jobs 33

Description

  • Define and own the test strategy, QA process, and QA playbook for the product.
  • Partner with product and engineering early to move QA upstream before features ship.
  • Evaluate AI output quality for accuracy, citation reliability, and appropriately scoped reasoning.
  • Design and maintain evaluation sets that handle non-deterministic output and catch regressions over time.
  • Test AI-generated document edits for accuracy and formatting fidelity across different document structures.
  • Build automated guardrails integrated into CI/CD for critical product flows.
  • Write structured, reproducible bug reports backed by evidence.
  • Partner with legal domain experts to translate customer workflows into test coverage.
  • Compare behavior across product surfaces and identify discrepancies.
  • Provide product and design-minded feedback on how workflows could be improved.

Requirements

  • 2+ years of QA experience, including ownership of test strategy for a product or major feature area.
  • Experience building QA process and automation from scratch in a fast-moving environment.
  • Comfort operating in high ambiguity and testing quickly as features ship.
  • Strong analytical and problem-solving skills with attention to functional correctness and AI output quality.
  • Ability to reason about non-deterministic systems and define what 'correct' means when there is no single right answer.
  • Excellent written communication skills for clear, concise, actionable bug reports.
  • Experience with bug tracking tools and test management workflows.
  • Experience testing AI or LLM-based products, especially for output accuracy and evaluation design, preferred.
  • CI/CD experience setting up pipelines and release gates for automated tests, preferred.
  • A design or UX sensibility and/or prior experience in legal technology or reading legal contracts, preferred.
  • Programming experience beyond test automation, preferred.

Benefits

  • Salary range of $105,000 to $150,000.
  • Medical, dental, and vision insurance for full-time employees.
  • Competitive and fair pay.
  • Maternity and paternity leave for full-time employees.
  • Short- and long-term disability coverage.
  • Opportunity to learn from a dedicated leadership team.
  • Top-of-the-line company swag.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

NewRocket

AI/ML Architect

NewRocket 251-1K Internet Software & Services

NewRocket is hiring an AI/ML Architect in India to support a large global client by designing, deploying, and maintaining scalable machine learning and AI applications.

India Senior AI (Artificial Intelligence)
Apache Spark AWS Azure BERT GCP Generative AI GPT Hugging Face LLM Machine Learning Matplotlib MongoDB MySQL NLP NumPy Pandas PostgreSQL Power BI Python PyTorch Scikit-learn Seaborn Snowflake SQL Tableau TensorFlow Vertex AI
10 hours, 54 minutes ago
Gramian Consultancy Group

Dermatologist (AI Evaluation)

Gramian Consultancy Group Professional Services

Gramian Consultancy is seeking India-based Dermatologists for a four-week remote contractor assignment evaluating dermatology images and AI-generated clinical content to improve healthcare AI reliability and safety.

India Contract Senior AI (Artificial Intelligence)
10 hours, 54 minutes ago
Innodata

AI Voice Evaluation Specialist

Innodata 1K-5K IT Services

Innodata is hiring Voice Specialists to conduct consistent, real-time conversations with AI models and evaluate which model provides the stronger conversational experience.

United States Mid Level AI (Artificial Intelligence)
$42k-$56k
Generative AI
1 day, 10 hours ago
Definium Therapeutics

Director, Supply and Distribution Quality

Definium Therapeutics 51-250 Health Care Providers & Services

Definium Therapeutics is seeking a Director, Commercial Supply Quality to lead quality oversight of outsourced commercial packaging, distribution, logistics, serialization, and product-traceability operations supporting launch readiness and ongoing supply.

Anywhere Full-time Executive Operations Manager QA Engineer
$180k-$215k
Git
1 day, 10 hours ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers