Senior Software Engineer - AI Coding Benchmarks

1 hour, 59 minutes ago
Part-time
Senior
Software Development
Mindrift.ai: Be the “I” in AI

Mindrift.ai: Be the “I” in AI

Join 10,000+ experts earning $15-50/hr training AI models remotely. Flexible freelance work, weekly payments. No AI experience required. Apply in 5 minutes.

Internet Software & Services

Description

  • Build realistic developer environments with codebases, infrastructure, and supporting context such as tickets, docs, and conversations.
  • Design tasks from intermediate states of those environments, including prompts and clear definitions of what counts as a solved task.
  • Write tests that validate agent solutions by accepting all correct approaches and rejecting incorrect ones.
  • Review agent outputs, analyze failures, and refine tasks and tests based on QA feedback.
  • Iterate on evaluation criteria to make tasks fair, robust, and solvable by AI agents.
  • Assess whether tasks are appropriately challenging for frontier coding models.
  • Guide and evaluate the agent’s work rather than writing the full code solution from scratch.

Requirements

  • 5+ years of experience in software development.
  • Experience with Python and FastAPI.
  • Experience with JavaScript/TypeScript and React.
  • Experience with Docker, Postgres, Kafka, and Redis.
  • Experience writing functional and integration tests.
  • English proficiency at B2 level or higher.
  • CV must be submitted in English.
  • Project-based participation with the ability to complete tasks by the deadline.
  • Ability to estimate work on tasks that take around 20 hours each, depending on complexity.

Benefits

  • Up to $40/hr equivalent compensation, depending on level and pace.
  • Flexible schedule with work completed on your own time.
  • Project-based work with estimated 20-hour task assignments.
  • Opportunity to contribute to AI evaluation work for leading tech companies.

Interested in this position?

Apply directly on the company website

Apply Now

Similar Roles

Senior ServiceNow Developer - Public Trust Clearance

LightFeather 51-250 Internet Software & Services

LightFeather is hiring a Senior ServiceNow Developer to design, build, and deploy secure, scalable ServiceNow solutions for federal government projects in a fully remote, full-time role.

Agile Angular AWS Azure CI/CD CSS GCP HTML JavaScript JIRA Kanban REST API Scrum
1 hour, 29 minutes ago

Data Quality Engineer

MediaRadar 51-250 Media

MediaRadar is hiring a Data Quality Engineer to build and operate the automated tooling, monitoring, and dashboards that keep governed data accurate, measurable, and continuously validated across the company’s data pipelines and platforms.

Python SQL
1 hour, 44 minutes ago

Senior Software Engineer

qode Internet Software & Services

Incedo is hiring a Senior Engineer to lead feature delivery for its MDM core platform, working with cross-functional teams on complex enterprise software development and support.

AWS Azure GCP Java Kanban Kubernetes
1 hour, 59 minutes ago

QA Lead, Data & Reporting

AssistRx 251-1K Pharmaceuticals

AssistRx is hiring a QA Lead of Data & Reporting to oversee quality assurance for data and reporting systems, lead a team of QA professionals, and ensure accurate, reliable client-facing reporting processes.

Agile Azure dbt Git HIPAA Postman Power BI Python REST API Snowflake SQL Tableau
1 hour, 59 minutes ago

You're on a roll! Sign up now to keep applying.

Sign Up

Already have an account? Log in

Used by 14,729+ remote workers