What the work involves

Each task starts from a real commit or issue in a high-quality public Python repository. Your job is to turn that history into something a model can be measured against: pin the repo at a pre-fix commit, build a Docker image that installs dependencies and runs the test suite deterministically, identify the tests that fail before the fix and pass after it, and write up the issue context a model would receive. Along the way you triage which issues are genuinely hard for an LLM versus trivially solvable, and you assess whether the repo's existing unit tests are strong enough to serve as a verifier at all — weak or flaky tests are the most common reason a candidate task gets thrown out.

Expect a lot of hands-on terminal work: cloning, installing, reading `tox.ini` and `pyproject.toml`, chasing native build failures, isolating network calls, and re-running suites until a build is reproducible from scratch. Researchers set direction on coverage — language, difficulty tier, task type — and you feed back on which repositories are worth mining. Engineers who perform well are sometimes asked to review or lead a small group of junior contributors.

What the screen looks for

The platform screen is AI-led and pushes on specifics. It wants evidence you have worked inside unfamiliar large codebases rather than only greenfield projects, that you can describe Dockerizing a Python project with real dependency pain (C extensions, pinned versions, system packages), and that you understand the difference between coverage percentage and test quality. Expect scenario questions where a test suite passes for the wrong reason, or a bug fix is one line but the failing test is flaky, and you have to say what you would do.

Logistics

  • Fully remote, asynchronous, contractor assignment with no medical or paid leave.
  • Three-month contract with a start date typically within a week or two of clearing the screen.
  • Hiring restricted to India, Pakistan, Nigeria, Egypt, Ghana, Bangladesh, Turkey, and Mexico.
  • Pay is observed at $100 per accepted task, not per hour — throughput depends heavily on how quickly you can get a repo building cleanly.
  • Minimum 3+ years of engineering experience; the listing describes this as tech-lead level.