RemoteLeaf Logo
Log in Sign up

QA/Test Engineer

1 month ago
United States
Part-time
Junior
Test Engineer
Quality Assurance and Testing
Git LLM Machine Learning Python
Apply Now
Weekday

Weekday

Weekday helps companies hire engineers who are vouched by other software engineers, enabling passive income for engineers. They offer services like drafting outreach messages, shortlisting candidates, and conducting reference checks. Backed by Y Combin...

Construction & Engineering
11-50
Founded 2020
View All Jobs 147

Description

  • Design comprehensive test cases for evaluation tasks, including edge cases and unexpected scenarios.
  • Review benchmark tasks and reference solutions to identify ambiguity, inconsistencies, missing requirements, and grading gaps.
  • Debug task environments and Python-based evaluation scripts to ensure reliable execution and accurate results.
  • Develop repeatable quality assurance processes, validation checklists, and testing frameworks for benchmark creation.
  • Identify shortcuts, exploits, or weaknesses that could compromise evaluation accuracy or benchmark integrity.
  • Collaborate with AI researchers, engineers, and task authors to improve task quality, reproducibility, and technical rigor.

Requirements

  • Master's degree, PhD, or equivalent practical experience in a STEM discipline involving software engineering, research, or advanced technical problem solving.
  • Minimum 1 year of professional experience in Quality Assurance, Test Engineering, Software Engineering, Research Engineering, or a related technical field.
  • Proven experience designing test cases, validating complex software systems, and debugging end-to-end workflows.
  • Strong proficiency in Python and Git, with the ability to troubleshoot unfamiliar codebases and technical environments.
  • Excellent analytical thinking, problem-solving skills, and exceptional attention to detail.
  • Experience documenting bugs, test strategies, and technical findings with clear written communication.
  • Experience evaluating AI systems, machine learning models, or AI-generated outputs is preferred.
  • Ability to work independently while managing multiple complex tasks with minimal supervision.
  • Ability to commit approximately 35 hours per week on a consistent basis.
  • At this time, H1-B and STEM OPT candidates are not supported.

Benefits

  • Compensation of $60-$90 per hour.
  • Fully remote engagement with flexible working hours.
  • Independent contractor engagement.
  • Weekly payments based on approved work completed.
  • Opportunity to contribute to cutting-edge AI research and benchmark development.
  • Reasonable accommodations available throughout the application and engagement process.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

CompassX Group

Test Engineer (Remote - 6 Month Contract Role)

CompassX Group 11-50 Internet Software & Services

CompassX is seeking a remote Test Engineer to support a Fortune 500 quick service restaurant client on a six-month engagement, owning quality assurance and automated validation for GraphQL, UI, and event-driven applications.

United States Contract Senior Test Engineer
$94k-$114k
Appium CI/CD Docker Git GitHub Actions GitLab CI GraphQL Jenkins JIRA Mocha MQTT Node.js Playwright TypeScript WebSockets
3 days, 16 hours ago
STR

Director of Test

STR 251-1K Aerospace & Defense

STR is hiring a Director of Test to lead a new group delivering advanced, secure test and evaluation campaigns for defense, intelligence, and national security customers.

United States Full-time Executive Program Manager Test Engineer
$216k-$295k
3 days, 17 hours ago
Oowlish

Test Architect – Regression & Quality Strategy

Oowlish 51-250 Internet Software & Services

Oowlish is hiring an experienced Test Architect to lead quality strategy and regression validation for a complex observability and audit platform serving enterprise environments.

Mexico Colombia Argentina Brazil Full-time Lead QA Engineer Test Engineer
Agile AWS ERP OpenTelemetry Terraform
1 week, 2 days ago
Talent2Win

Software Verification Engineer

Talent2Win 1-10 Professional Services

Talent2Win is hiring a Software Verification Engineer for a specialized software engineering client building safety-critical embedded solutions for the aerospace and automotive industries.

Latin America Full-time Senior Test Engineer
C C++ Embedded Systems Git JIRA OpenGL Vulkan
1 week, 4 days ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers