Programming Languages Researcher (Research Coding Tasks)

$80/hr

rate as listed

Workload: ~15h/week
Engagement: Project-based
Location: Worldwide
Field: Languages

Turn published programming languages research and released code into coding tasks for frontier AI agents. Define hidden evaluations and performance thresholds, build reproducible solutions, and review agent attempts.

The work

You will start from a recent programming languages paper with released code, ideally your own. The relevant research may involve compilers, type systems, program synthesis, verification, or SAT/SMT solving.

Day to day, you will remove the implementation of the paper's core contribution while preserving enough infrastructure for an AI agent to rebuild it. You will write the task statement, create held-out evaluation workloads, set pass thresholds for each metric, and provide both a hidden reference solution and a second solution demonstrating that the stated bar can be exceeded. You may later review agent attempts against those criteria.

What the platform screens for

The platform looks for a PhD in computer science or a related field, completed or in progress, along with published programming languages research and released code. Screening is likely to examine your exact contribution to a paper and artifact, your understanding of its experimental design, and whether you can turn a research result into a fair, leak-resistant, reproducible evaluation. First-author work with measured improvements and experience with Docker-based benchmarks are preferred.

Logistics

The contract is fully remote, asynchronous, and flexible. Each task is described as taking a few hours, though the time required will depend on the artifact and evaluation design, and applicants should expect a thorough background check. The observed platform band is not a guarantee.

Pay band: $80/hr

What they look for

Published programming languages research at a recognized or comparable venue
Released code for at least one published paper
PhD in computer science or a related field, completed or in progress
Experimental evaluation using benchmarks and quantitative metrics
Depth in compilers, type systems, program synthesis, verification, or SAT/SMT solving
Reproducible research artifacts and Docker-based benchmark environments

Included with this listing

Interview prep — 10 questions

Matched to what this platform's screening actually measures. Answer each out loud in 60–90 seconds: claim → concrete example → implication.

What it tests: This verifies publication history, artifact ownership, venue quality, and whether the candidate can identify their personal contribution rather than speaking for the whole author team.
A strong answer: Name the paper, venue, year, repository or artifact link, and your role in both the research and implementation. Briefly identify the core result and how the released code reproduces it.
Where people lose it: Candidates lose points by naming a paper without accessible code, describing a group project without clarifying their own contribution, or relying on unpublished work.

Summary, eligibility guidance and screening preparation are prepared by Sidequest from the platform's posting. Check the source listing before applying.