About the job
Role Overview
Mercor is partnering with a top AI lab on a project to train and assess the ability of LLMs to evaluate video-based content. We’re looking for detail-oriented contributors to help create question-and-answer pairs and evaluation rubrics. This is a flexible opportunity that can fit any schedule.
Key Responsibilities
- Review short and long-form videos to create question-and-answer (QA) pairs.
- Develop rubric-based evaluation questions and answers to assess multiple dimensions of model output.
- Apply structured guidelines to ensure high-quality evaluations.
Ideal Qualifications
- Experience in annotation, data labeling, or related content evaluation work.
- Strong English writing skills and ability to craft clear, specific questions and answers.
- Comfortable working independently and managing flexible tasks with attention to detail.
- General familiarity with multimedia content (e.g., videos from social media, how-tos, documentaries).
More About the Opportunity
- Remote and asynchronous.
- Expected commitment: 30-40 hours per week
Compensation & Contract Terms
- Applicants must be based in the US or Canada.
- This position is structured with clear expectations.
- Payments issued weekly via Stripe Connect
Application Process
- Submit your resume to begin
- After submitting your resume, please complete a short AI interview and form.
- Follow-up typically occurs within 3–5 business day
About Mercor
- Mercor is a talent marketplace that connects top experts with leading AI labs and research organizations.
- Our investors include Benchmark, General Catalyst, Adam D’Angelo, Larry Summers, and Jack Dorsey.
No comments:
Post a Comment