Note: The job is a remote job and is open to candidates in USA. Productive Playhouse is a global data company specializing in language and data services. They are seeking experienced Hindi speakers with coding knowledge to join their AI Model Rater project, focusing on evaluating the accuracy and quality of AI-generated responses and their integration with digital services.
Responsibilities
- Analyze and rate AI-generated responses and their associated tool outputs against a detailed set of quality guidelines
- Verify that the AI successfully integrates information from different digital services (APIs) as required by the prompt
- Evaluate the accuracy, completeness, and overall quality of the AI's response and tool execution
- Follow provided instructions to achieve task goals, with all evaluations recorded in English
Skills
- Advanced/Expert fluency in Hindi language, as spoken in the India region (ISO: hi_in)
- Strong comprehension of English (read and write)
- Strong conceptual knowledge of APIs and digital services (ability to understand how AI tools interact with external apps like search, maps, or calendar, without needing to write code)
- Ability to follow complex technical guidelines precisely
- All participants must have, or be willing to create, an Upwork account
- Applicants must pass a quick skill verification assessment (MC, duration ~10 minutes)
- Work must be performed on participant's own devices
- You must use your own laptop, and smartphone (if required)
- A Gmail account may be required to access certain tools. If you do not have a Gmail account, you must be willing to create one
- Using AI during the assessment and work is STRICTLY forbidden, and results/inputs will be monitored to detect AI usage
Benefits
- Flexible schedule
Company Overview
Apply Now