What the work involves

Turing is constructing LLM evaluation and training datasets built on the commit and issue histories of well-maintained public repositories. Your job is the human-in-the-loop half: find issues in trending open-source libraries that are hard for models but unambiguously solvable, reconstruct the repository state at the relevant commit, and package it into a task that can be verified automatically. That means Dockerizing environments, pinning dependencies so a build reproduces months from now, checking whether the existing unit tests actually gate the bug, and writing or strengthening tests when they don't.

A large share of the time goes to judgment rather than typing. You will decide whether an issue is genuinely a bug or a feature discussion, whether the linked PR's diff is the only correct fix or one of several, and whether a passing test suite really proves the fix landed. You will also run codebases locally to observe how models behave on candidate tasks and feed that back to researchers designing coverage across languages and difficulty tiers. Senior contributors get opportunities to lead a small group of junior engineers on the same pipeline.

What the platform screens for

Expect the screen to push on specifics rather than resume summaries: which repositories you have worked in, what their build systems look like, and what broke when you tried to containerize them. Naming real projects with 500+ stars and describing their test layout, flaky suites, or CI quirks reads far stronger than generic "I use Docker daily." Screeners also probe evaluation judgment — how you distinguish a well-specified task from an underspecified one, and how you handle a test that passes for the wrong reason.

Logistics

  • Fully remote, contractor assignment; no medical or paid leave.
  • 20 hours per week with some overlap with PST business hours.
  • Three-month initial contract, with starts moving quickly — often within a week of clearing the screen.
  • Pay band undisclosed on this listing; Turing coding roles are commonly hourly, but treat any figure you hear as observed rather than guaranteed until it is in your contract.