Rubric Evaluation Specialist - Writing

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: Generalist Writer
Type: Contract
Compensation: $25–$45/hour
Location: Remote

Role Responsibilities

• Evaluate AI-generated written conversations using predefined rubrics to enhance performance.
• Draft and refine evaluation rubrics for generalist writing tasks.
• Apply structured scoring criteria to assess clarity, coherence, accuracy, and style.
• Review edge cases and ambiguous outputs to ensure consistent grading standards.
• Work independently and asynchronously to meet deadlines and improve AI model performance.

Qualifications

Must-Have

• Experience with rubric-based grading, editing, or content evaluation at Mercor.
• Strong generalist writing skills across multiple domains (e.g., business, academic, creative).
• Excellent attention to detail and consistency in applying guidelines.
• Ability to interpret nuanced criteria and make sound judgment calls.

Preferred

• Background in English, journalism, communications, or a related field.

Compensation & Legal

• Opportunities for expanded reviewer/lead responsibilities.

Application Process (Takes 20–30 mins to complete)

• Submit a resume or writing sample to get started.
• Selected applicants will be contacted within a few days regarding next steps.

Resources & Support

• For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
• For any help or support, reach out to: [email protected]

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.