Remote Autonomous AI Agent Workflow & Tool-Use Evaluator — Specialist Placement
Hiring Agency: Remote Job Recruiting • Updated Today
Position Description
Remote Job Recruiting is recruiting an experienced Remote Autonomous AI Agent Workflow & Tool-Use Evaluator — Specialist Placement for a 100% remote contract supporting advanced artificial intelligence model development across the US, UK, Canada, Australia, New Zealand, and Ireland. Compensated at $25 - $45/hour, this role collaborates with frontier machine learning teams to author high-precision domain benchmarks, evaluate reasoning outputs, and refine domain-specific model accuracy.
Core Responsibilities & Deliverables
- Run real-life scenarios: bills, returns, email follow-ups, bookings, shopping, groceries, travel, family logistics, outings, and forms
- Approve or stop the agent at key steps
- Screen record every attempt
- Score each run on completion, quality, control issues, satisfaction, and time
- Share clear written feedback on the agent experience
- Who we’re looking for
- US-based, with a Bachelor’s degree
Required Qualifications & Domain Skills
- Have a direct say in how next-generation AI agents get built and improved
- How the selection process works
- Complete the screening questions
- Interview: 15-30 minutes max
- Profile review: if everything looks good, we present you to the client
- Onboarding: our Talent Ops team will contact you to get you started
- Demonstrated professional or academic background in ChatGPT, Muse, AI agents, Attention to Detail, Following Instructions, Screen Recording.
Engagement Details & Compensation
- Compensation Rate: $25 - $45/hour (contract specialist remuneration)
- Work Arrangement: 100% Remote / Work From Home
- Eligible Remote Countries: United States, United Kingdom, Canada, Australia, New Zealand, Ireland, and South Africa
- Core Competencies: ChatGPT, Muse, AI agents, Attention to Detail, Following Instructions, Screen Recording
- Hiring Agency: Remote Job Recruiting
Direct Application Process: Click the Apply button on this page to proceed directly to our partner application portal and submit your profile for immediate consideration.