Music & Design Expert - AI Agent Evaluation
Workada
What You'll Do
As a contributor, you'll take problems from your own work and turn each one into a self-contained Task that an AI agent has to solve in a Linux terminal. For every Task, you'll write a clear brief, set up the environment with the files and tools the agent needs, build a reference solution, and write automated checks on the final result. The strongest Tasks are the ones today's best agents still get wrong. Your firsthand knowledge of where real work in your field goes wrong is what makes them valuable. That might be a color profile that shifts on conversion, a MIDI file whose tempo map drifts, a mix that clips after resampling, an SVG export that breaks a layout, or a design-token change that fails accessibility contrast.
Specialties We're Looking For
We're building depth across: Music composition and theory Music technology and audio production Sound design and audio engineering Graphic and visual design UX/UI and product design Motion design and animation
You're a Great Fit If
You hold, or are completing, a PhD in music, audio, or design or a closely related field, or you have 5+ years of professional experience in it. You've done technical or computational work in your field and are comfortable in a Linux terminal: shell commands, scripting in Python or Bash, and installing packages. You've used Docker, or can pick it up quickly. We share templates and worked examples. You can draw on your own research, projects, code, or files as the source material for your Tasks. You have a precise, adversarial eye; you enjoy finding the exact point where a workflow breaks and explaining it cleanly. You're honest about uncertainty and write instructions that leave no room for misreading. You're based in the US or Canada. Academics and practitioners are equally welcome. No AI or machine-learning background is needed. Your expertise and your comfort working in a terminal are what we're after.
Materials You Can Use
Almost any piece of your work can seed a Task, for example: Audio stems and session files MIDI files and notation Design files (Figma, SVG, PSD exports) and brand systems Motion and animation projects Scripts you use to batch-process assets Bugs, edge cases, and failure modes you've had to debug yourself
- ...competitive advantage. Cresta’s unified AI platform combines conversational AI agents, real-time human agent augmentation... ...s you. As our first AI Agent Evaluation Engineer, you define what quality... ...analysis, rigorous evaluation design, and engineering execution: reading...SuggestedFull timeWork at officeRemote workHome officeFlexible hours
- ...competitive advantage. Cresta’s unified AI platform combines conversational AI agents, real-time human agent augmentation... ...That’s you. As our first AI Agent Evaluation Engineer, you define what quality... ...analysis, rigorous evaluation design, and engineering execution: reading...SuggestedFull time
$20 - $45 per hour
...Required Skills Analytical Listening Music Theory Knowledge Conservatory Degree Audio... ...micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve...SuggestedPart timeFor contractorsRemote work$75 - $90 per hour
...professionals to create realistic cases for testing AI. You’ll draw on the kinds of files you’ve... ...’s rules. Include the information an expert would need to make a decision. Work... ...plus. Experience writing or reviewing AI evaluation cases or training data is a plus....SuggestedContract workFreelanceManual laborRemote work- ...and what changed In later phases: review AI model descriptions of clips and mark what'... ..., producers, or mixing engineers Music educators, ear-training instructors, band/... ...graduates Location sound recordists, sound designers, foley artists, or podcast/documentary audio...SuggestedPart time
- ...the intersection of human creativity and AI. We partner with leading AI labs and product... ...companies to improve their tools across design, video, development, and beyond. Our research... ...and mobile applications. Skills evaluation: You'll rate AI-generated outputs across...Full timeContract work
$70 per hour
...is a 10-minute call. Ethos's AI interviewer asks about your... ...matches you to projects like Letting Agent. Keep your CV open and cover... ...scheduling, banquet event orders Expert-level document, spreadsheet,... ...or event production to create, evaluate, and refine AI-generated...Hourly payWeekly payContract workImmediate startRemote workFlexible hours$100 per hour
...~60% Within 3 months ~80% Why this role AI labs pay $100/hr for a AI Engineer (LLMs & Agents) Expert because getting Software Engineering nuance wrong... ...) Expert, expect to draw on real domain judgment (evaluating outputs, correcting errors, or providing expert reasoning...Hourly payContract workRemote work$250 per unit
...is a 10-minute call. Ethos's AI interviewer asks about your... ...matches you to projects like AI Agent Sandboxes: Paid Developer Interview... ...About Ethos Ethos is a new expert network built by a McKinsey/... ...'re given complex scenarios to evaluate: grading the AI's logic, correcting...Hourly payOngoing contractContract workRemote work- ...Evaluate and annotate AI-generated audio by identifying instruments. About Turing: Based in San... ...STEM, multilinguality, multimodality, and agents; and second, by applying that expertise... ...Role Overview: We’re looking for music professionals with formal music education...Part timeRemote work
- ...About the role AI assistants are becoming part of everyday life, helping people with... ...and select your preferred experience. Evaluate conversational quality, helpfulness,... ...simple testing guidelines and document your evaluations accurately. Who we're looking for Active...Hourly payTemporary workPart timeRemote work
- ...the intersection of human creativity and AI. We partner with leading AI labs and product... ...companies to improve their tools across design, video, development, and beyond. Our research... ...through. What You May Work On Evaluate AI-generated content design outputs with a...Full timeContract work
- ...the intersection of human creativity and AI. We partner with leading AI labs and product... ...companies to improve their tools across design, video, development, and beyond. Our research... ...through. What You May Work On Evaluate AI-generated graphic design outputs across...Contract workImmediate start
- ...the intersection of human creativity and AI. We partner with leading AI labs and product... ...companies to improve their tools across design, video, development, and beyond. Our research... ...through. What You May Work On Evaluate AI-generated advertisements and marketing...Full timeContract work
$100 per hour
...application is a 10-minute call. Ethos's AI interviewer asks about your career,... ...you to projects like Voice AI Phone Agents: Paid User Interview. Keep your CV... ...hr) No Degree Required Creative & Design Voice & Audio Expert AI Training Related jobs Sales...$70 per hour
...is a 10-minute call. Ethos's AI interviewer asks about your... ...you to projects like Real Estate Agent. Keep your CV open and cover these... ..., banquet event orders Expert-level document, spreadsheet, and... ...or event production to create, evaluate, and refine AI-generated documents...Hourly payWeekly payContract workImmediate startRemote workFlexible hoursShift work$55 - $80 per hour
...more About the Opportunity Our client, a leading AI research organization, is looking for experienced commercial composers, music producers and audio post-production specialists to support AI training and evaluation. You’ll develop rights-cleared multi-stem music...Contract work$50 - $105 per hour
...50 - $105/hour Required Skills Graphic Design Layout design Attention to detail... ...About micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve...Hourly payPart timeFor contractorsRemote workWorldwide$50 - $105 per hour
...SketchUp AutoCAD Vectorworks Scripts Briefs Set designer About micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve LLM...Hourly payPart timeFor contractorsRemote workWorldwide$10 - $24 per hour
...live online classes for developers on building AI agents in code: tool definitions, the agent loop, planning, memory, evaluation and guardrails. You teach in English, remotely... ...code labs or reviewing other people's system designs. Patience with developers who expect an agent...Contract workManual laborRemote work$10 - $24 per hour
...Teach live online classes on grading one AI response against written criteria: accuracy, instruction following and safety, with a justification... .... What the work is for YPAI Learn trains people for AI evaluation work in live classes. Your classes take learners from a gut...Contract workRemote work- ...position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for an AI Voice Evaluation Specialist based in Canada. Join an innovative AI-focused environment as an AI Voice Evaluation Specialist, where you will...
- ...What You'll Do Design benchmark evaluations based on real-world data analytics scenarios. Format complex data problems for systematic testing and review. Commit 20 or more hours per week to continuous project work. Who Should Apply Professional experience in data analytics...Permanent employmentContract workFor contractorsRemote work
$25 - $45 per hour
...Required Skills ChatGPT Muse AI agents Attention to Detail Following Instructions... ...models and agents. We provide frontier evaluations and reinforcement learning environments... ...agent sources and vets domain experts for our data platform that enables rapid...For contractorsWork at officeWeekend work$10 - $30 per hour
...feedback digital delivery platforms ai-driven tools familiarity... ...Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments... ...recruiting agent sources and vets domain experts for our data platform that enables...Part timeFor contractorsRemote workShift work$70 per hour
...a 10-minute call. Ethos's AI interviewer asks about your career... ...projects like Senior Graphic Designer. Keep your CV open and cover... ...consulting firm, bank, or agency Expert-level PowerPoint skills... ...team or equivalent to create, evaluate, and refine AI-generated presentations...Hourly payContract workImmediate startRemote workFlexible hoursShift work- ...Curate and evaluate Java code for AI training and performance benchmarks. About Us Based in San Francisco... ..., multilinguality, multimodality, and agents; and second, by applying that expertise... ...cycle (prototyping, architecture design, API design, production implementation,...Part timeFor contractorsFlexible hours
$80 per hour
...a 10-minute call. Ethos's AI interviewer asks about your career... ...projects like Investor Deck Designer. Keep your CV open and cover... ...VCs look for slide by slide Expert-level PowerPoint and slide craftsmanship... ...have closed rounds to create, evaluate, and refine AI-generated...Hourly payWeekly payContract workImmediate startRemote workFlexible hours- ...requirements Help Shape the Future of AI in Healthcare What if the... ...healthcare information, evaluates medical scenarios, and delivers... ...opportunities to contribute to the evaluation and improvement of advanced... ...pathways . This opportunity is designed for candidates who can apply...Contract workFor contractorsTraineeshipRemote work
$90 - $140 per hour
...About micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to... ...recruiting agent sources and vets domain experts for our data platform that enables rapid...Contract workFor contractorsRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Music & Design Expert - AI Agent Evaluation. Be the first to apply!
