RemoteLeaf Logo
Log in Sign up

QA/Test Engineer

5 days, 4 hours ago
United States
Part-time
Junior
Test Engineer
Quality Assurance and Testing
Git LLM Machine Learning Python
Apply Now
Weekday

Weekday

Weekday helps companies hire engineers who are vouched by other software engineers, enabling passive income for engineers. They offer services like drafting outreach messages, shortlisting candidates, and conducting reference checks. Backed by Y Combin...

Construction & Engineering
11-50
Founded 2020
View All Jobs 175

Description

  • Design comprehensive test cases for evaluation tasks, including edge cases and unexpected scenarios.
  • Review benchmark tasks and reference solutions to identify ambiguity, inconsistencies, missing requirements, and grading gaps.
  • Debug task environments and Python-based evaluation scripts to ensure reliable execution and accurate results.
  • Develop repeatable quality assurance processes, validation checklists, and testing frameworks for benchmark creation.
  • Identify shortcuts, exploits, or weaknesses that could compromise evaluation accuracy or benchmark integrity.
  • Collaborate with AI researchers, engineers, and task authors to improve task quality, reproducibility, and technical rigor.

Requirements

  • Master's degree, PhD, or equivalent practical experience in a STEM discipline involving software engineering, research, or advanced technical problem solving.
  • Minimum 1 year of professional experience in Quality Assurance, Test Engineering, Software Engineering, Research Engineering, or a related technical field.
  • Proven experience designing test cases, validating complex software systems, and debugging end-to-end workflows.
  • Strong proficiency in Python and Git, with the ability to troubleshoot unfamiliar codebases and technical environments.
  • Excellent analytical thinking, problem-solving skills, and exceptional attention to detail.
  • Experience documenting bugs, test strategies, and technical findings with clear written communication.
  • Experience evaluating AI systems, machine learning models, or AI-generated outputs is preferred.
  • Ability to work independently while managing multiple complex tasks with minimal supervision.
  • Ability to commit approximately 35 hours per week on a consistent basis.
  • At this time, H1-B and STEM OPT candidates are not supported.

Benefits

  • Compensation of $60-$90 per hour.
  • Fully remote engagement with flexible working hours.
  • Independent contractor engagement.
  • Weekly payments based on approved work completed.
  • Opportunity to contribute to cutting-edge AI research and benchmark development.
  • Reasonable accommodations available throughout the application and engagement process.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Natera

Senior Software V&V Engineer

Natera 1K-5K Pharmaceuticals

Foresight Diagnostics, now part of Natera, is hiring a Senior Software V&V Engineer to own verification and validation for regulated bioinformatics and clinical software supporting MRD cancer detection.

United States Canada Full-time Senior QA Engineer Test Engineer
AWS GCP
3 days, 4 hours ago
ELEKS

Strong Middle / Senior Test Engineer (eExcise), JR620

ELEKS 1K-5K Construction & Engineering

ELEKS is hiring a Strong Middle/Senior Test Engineer in Ukraine to support JTI Ukraine’s integration of internal systems with the Ukrainian state eExcise platform under upcoming regulatory requirements.

Ukraine Senior Test Engineer
Agile Android Azure
3 days, 5 hours ago
ELEKS

(insurance) Middle Test Engineer, JR614

ELEKS 1K-5K Construction & Engineering

ELEKS Quality Assurance Office is hiring a Middle Test Engineer in Poland to support quality assurance for a major global insurance broker’s client-server and desktop application systems.

Poland Mid Level Test Engineer
Agile CI/CD JIRA Python
6 days, 4 hours ago
9th Way Insignia

Senior Test Engineer

9th Way Insignia 51-250 Internet Software & Services

9th Way Insignia is hiring a Senior Test Engineer to support the Department of Veterans Affairs’ eCMS program with functional, regression, integration, API, and automation testing.

United States Full-time Senior Test Engineer
$98k-$125k
Agile Azure C# CI/CD .NET REST API Scrum
1 week, 2 days ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers