Software Engineering Researcher (Research Coding Tasks)

$80/hr

rate as listed

Workload: ~15h/week
Engagement: Project-based
Location: Worldwide
Field: STEM and Coding

Turn published software engineering research and released code into benchmarked coding tasks for frontier AI agents. The work includes designing hidden evaluations, setting metric thresholds, building reference solutions and reviewing agent attempts.

The work

Start from a recent software engineering paper with released code, ideally your own. Remove the implementation of the paper's core contribution, then package the remaining repository as a task in which an AI agent must reconstruct that contribution and outperform the published results on the paper's benchmarks.

Task design and review

You will write precise task statements, create held-out evaluation workloads, define pass thresholds for each metric and provide both a hidden reference solution and a second solution demonstrating that the target can be exceeded. After agents attempt the task, you will review their implementations and results. Relevant research depth includes test generation, program repair, static analysis and mining software repositories.

What the platform screens for

The screen focuses on verifiable publication history, released research code and a completed or in-progress PhD in computer science or a related field. Expect follow-up on experimental design, benchmark leakage, reproducibility and whether your proposed evaluation distinguishes genuine technical progress from benchmark-specific shortcuts. First-author work reporting measured improvements over a baseline, Docker experience and reproducible benchmark setups are preferred rather than required.

Logistics

The contract is fully remote, asynchronous and flexible. Each task typically takes a few hours, and candidates who meet the stated requirements and provide the requested paper and artifact materials are prioritised for review; the process also includes background checks.

Pay band: $80/hr

What they look for

Published software engineering research at ICSE, FSE, ASE, ISSTA or a comparable venue
Released code for at least one published paper
PhD in computer science or a related field, completed or in progress
Experimental design for held-out workloads, metrics and pass thresholds
Research coding in test generation, program repair, static analysis or mining software repositories
Reproducible benchmarking and Docker

Included with this listing

Interview prep — 10 questions

Matched to what this platform's screening actually measures. Answer each out loud in 60–90 seconds: claim → concrete example → implication.

What it tests: This checks that you can identify a verifiable paper, accessible codebase and separable core contribution rather than proposing an abstract research idea.
A strong answer: Name the paper, venue, repository or artifact link, your authorship role and the specific implementation that would be removed. Briefly explain why the remaining repository can still build, run and expose a meaningful reconstruction task.
Where people lose it: Candidates lose points by naming a paper without released code, or by choosing a contribution so entangled with the infrastructure that removing it would leave no runnable task scaffold.

Summary, eligibility guidance and screening preparation are prepared by Sidequest from the platform's posting. Check the source listing before applying.

STEM AND CODING on Sidequest

What stem and coding AI work pays

Rates by specialty, the platforms hiring, and every open role in this field.