What the work actually involves

This is a hybrid role: part data quality judgment, part pipeline engineering. On any given day you might be drafting or revising a rubric that defines what separates an acceptable model response from an excellent one, labeling or auditing preference pairs against that rubric, then writing the tooling that lets a larger annotation pool apply the same standard consistently. The write-up matters as much as the work — Turing expects findings synthesized into concise, structured updates that a researcher can act on without a follow-up meeting.

The engineering surface is Python for pipeline and infrastructure work, or TypeScript if your background is in large application codebases (contributions to or familiarity with something on the scale of VSCode is called out as a plus, which signals agentic coding and IDE-adjacent evaluation work). You will not be handed a spec. The listing is explicit about ambiguity and a bias toward shipping over deliberating.

What the screen is looking for

  • Prior hands-on preference data or rubric experience. This is the sharpest filter. Reviewers want examples where you defined criteria, not just applied someone else's.
  • Articulable taste. The phrase used is "able to articulate why something is good, not just that it is." Expect to be pushed on the reasoning behind a quality call until it either holds up or collapses.
  • Real code depth. Follow-ups tend to move from what you built to why you structured it that way, and how it failed.
  • Communication under compression. Several prompts are effectively "summarize this in a way a busy researcher can act on."

Logistics

Evaluation runs about 60 minutes total: a roughly 25-minute AI interview followed by a ~30-minute hiring manager conversation, scheduled quickly for selected candidates. The engagement is independent contractor with no medical or paid leave benefits, running three months with an ASAP start. Compensation is described as market rate and undisclosed publicly — Turing asks for a single specific number rather than a range, so decide yours before the screen. Location is restricted to North America.