Careerflow Human Data Labs partners with AI companies to bring real-world professional expertise into their products.
We're looking for a high-taste engineer with a strong eye for quality — someone who's worked hands-on with preference data and rubric creation, and knows what separates a good example from a great one. You communicate clearly and can synthesize complex technical work the way a strong MAI dev would: concise, structured, and easy for others to act on. You're comfortable in ambiguous, fast-moving environments and default to shipping over deliberating.
Design, refine, and iterate on rubrics and evaluation criteria for preference data
Review and label data with a high bar for quality, catching subtle issues others might miss
Build and maintain pipelines/infra that support data generation, collection, and evaluation workflows
Synthesize findings from data work into clear write-ups, updates, or recommendations for the team
Collaborate closely with researchers/engineers to translate qualitative judgment into scalable processes
5+ yrs of proficiency in Python (especially building/maintaining pipelines and infra) or Typescript/Golang/JavaScript (experience contributing to or navigating large codebases like VSCode a plus)
Hands-on experience with preference data collection and/or rubric creation
Strong written and verbal communication — able to distill technical work into clear, actionable summaries
Demonstrated high-taste judgment — able to articulate why something is good, not just that it is
Work in a fully remote environment.
Opportunity to work on cutting-edge AI projects with leading LLM companies.
Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST.
Employment type : Contractor assignment (no medical/paid leave)
Duration of contract : 3 month; [expected start date is next week]
Location : USA, Canada
One round of interview (30 mins of technical & cultural discussion)
Total number of positions: 320
Location : USA (US Approved Regions), Canada
Engagement Length: 4 weeks, Full-time (8 hours/day) 20 hours per week. Overlap 4 hours with PST
6-week project, with a preferred commitment of up to 20 hours per week.
Fully remote, with payment in USD.
Start as soon as you successfully pass the assessment.
Work at careerflow.ai? Sign in to claim this org and manage its profile.