Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Music & Design Expert - AI Agent Evaluation

Temporary

Workada

What You'll Do

As a contributor, you'll take problems from your own work and turn each one into a self-contained Task that an AI agent has to solve in a Linux terminal. For every Task, you'll write a clear brief, set up the environment with the files and tools the agent needs, build a reference solution, and write automated checks on the final result. The strongest Tasks are the ones today's best agents still get wrong. Your firsthand knowledge of where real work in your field goes wrong is what makes them valuable. That might be a color profile that shifts on conversion, a MIDI file whose tempo map drifts, a mix that clips after resampling, an SVG export that breaks a layout, or a design-token change that fails accessibility contrast.

Specialties We're Looking For

We're building depth across: Music composition and theory Music technology and audio production Sound design and audio engineering Graphic and visual design UX/UI and product design Motion design and animation

You're a Great Fit If

You hold, or are completing, a PhD in music, audio, or design or a closely related field, or you have 5+ years of professional experience in it. You've done technical or computational work in your field and are comfortable in a Linux terminal: shell commands, scripting in Python or Bash, and installing packages. You've used Docker, or can pick it up quickly. We share templates and worked examples. You can draw on your own research, projects, code, or files as the source material for your Tasks. You have a precise, adversarial eye; you enjoy finding the exact point where a workflow breaks and explaining it cleanly. You're honest about uncertainty and write instructions that leave no room for misreading. You're based in the US or Canada. Academics and practitioners are equally welcome. No AI or machine-learning background is needed. Your expertise and your comfort working in a terminal are what we're after.

Materials You Can Use

Almost any piece of your work can seed a Task, for example: Audio stems and session files MIDI files and notation Design files (Figma, SVG, PSD exports) and brand systems Motion and animation projects Scripts you use to batch-process assets Bugs, edge cases, and failure modes you've had to debug yourself

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Music & Design Expert - AI Agent Evaluation in Remote vacancy
  •  ...competitive advantage. Cresta’s unified AI platform combines conversational AI agents, real-time human agent augmentation...  ...s you.   As our first AI Agent Evaluation Engineer, you define what quality...  ...analysis, rigorous evaluation design, and engineering execution: reading... 
    Suggested
    Full time
    Work at office
    Remote work
    Home office
    Flexible hours

    Cresta

    Remote
    8 days ago
  •  ...competitive advantage. Cresta’s unified AI platform combines conversational AI agents, real-time human agent augmentation...  ...That’s you. As our first AI Agent Evaluation Engineer, you define what quality...  ...analysis, rigorous evaluation design, and engineering execution: reading... 
    Suggested
    Full time

    Cresta

    Remote
    9 days ago
  • $20 - $45 per hour

     ...Required Skills Analytical Listening Music Theory Knowledge Conservatory Degree Audio...  ...micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve... 
    Suggested
    Part time
    For contractors
    Remote work

    micro1

    Remote
    11 days ago
  • $75 - $90 per hour

     ...professionals to create realistic cases for testing AI. You’ll draw on the kinds of files you’ve...  ...’s rules. Include the information an expert would need to make a decision. Work...  ...plus. Experience writing or reviewing AI evaluation cases or training data is a plus.... 
    Suggested
    Contract work
    Freelance
    Manual labor
    Remote work

    Braintrust

    Remote
    2 days ago
  •  ...and what changed In later phases: review AI model descriptions of clips and mark what'...  ..., producers, or mixing engineers Music educators, ear-training instructors, band/...  ...graduates Location sound recordists, sound designers, foley artists, or podcast/documentary audio... 
    Suggested
    Part time

    micro1

    Remote
    21 days ago
  •  ...the intersection of human creativity and AI. We partner with leading AI labs and product...  ...companies to improve their tools across design, video, development, and beyond. Our research...  ...and mobile applications. Skills evaluation: You'll rate AI-generated outputs across... 
    Full time
    Contract work

    Contra Labs

    Remote
    11 days ago
  • $70 per hour

     ...is a 10-minute call. Ethos's AI interviewer asks about your...  ...matches you to projects like Letting Agent. Keep your CV open and cover...  ...scheduling, banquet event orders Expert-level document, spreadsheet,...  ...or event production to create, evaluate, and refine AI-generated... 
    Hourly pay
    Weekly pay
    Contract work
    Immediate start
    Remote work
    Flexible hours

    Ethos

    Remote
    15 days ago
  • $100 per hour

     ...~60% Within 3 months ~80% Why this role AI labs pay $100/hr for a AI Engineer (LLMs & Agents) Expert because getting Software Engineering nuance wrong...  ...) Expert, expect to draw on real domain judgment (evaluating outputs, correcting errors, or providing expert reasoning... 
    Hourly pay
    Contract work
    Remote work

    Ethos

    Remote
    a month ago
  • $250 per unit

     ...is a 10-minute call. Ethos's AI interviewer asks about your...  ...matches you to projects like AI Agent Sandboxes: Paid Developer Interview...  ...About Ethos Ethos is a new expert network built by a McKinsey/...  ...'re given complex scenarios to evaluate: grading the AI's logic, correcting... 
    Hourly pay
    Ongoing contract
    Contract work
    Remote work

    Ethos

    Remote
    26 days ago
  •  ...Evaluate and annotate AI-generated audio by identifying instruments. About Turing: Based in San...  ...STEM, multilinguality, multimodality, and agents; and second, by applying that expertise...  ...Role Overview: We’re looking for music professionals with formal music education... 
    Part time
    Remote work

    Turing

    Remote
    7 days ago
  •  ...About the role AI assistants are becoming part of everyday life, helping people with...  ...and select your preferred experience. Evaluate conversational quality, helpfulness,...  ...simple testing guidelines and document your evaluations accurately. Who we're looking for Active... 
    Hourly pay
    Temporary work
    Part time
    Remote work

    micro1

    Remote
    9 days ago
  •  ...the intersection of human creativity and AI. We partner with leading AI labs and product...  ...companies to improve their tools across design, video, development, and beyond. Our research...  ...through. What You May Work On Evaluate AI-generated content design outputs with a... 
    Full time
    Contract work

    Contra Labs

    Remote
    a month ago
  •  ...the intersection of human creativity and AI. We partner with leading AI labs and product...  ...companies to improve their tools across design, video, development, and beyond. Our research...  ...through. What You May Work On Evaluate AI-generated graphic design outputs across... 
    Contract work
    Immediate start

    Contra Labs

    Remote
    a month ago
  •  ...the intersection of human creativity and AI. We partner with leading AI labs and product...  ...companies to improve their tools across design, video, development, and beyond. Our research...  ...through. What You May Work On Evaluate AI-generated advertisements and marketing... 
    Full time
    Contract work

    Contra Labs

    Remote
    a month ago
  • $100 per hour

     ...application is a 10-minute call. Ethos's AI interviewer asks about your career,...  ...you to projects like Voice AI Phone Agents: Paid User Interview. Keep your CV...  ...hr) No Degree Required Creative & Design Voice & Audio Expert AI Training Related jobs Sales... 

    Ethos

    Remote
    14 days ago
  • $70 per hour

     ...is a 10-minute call. Ethos's AI interviewer asks about your...  ...you to projects like Real Estate Agent. Keep your CV open and cover these...  ..., banquet event orders Expert-level document, spreadsheet, and...  ...or event production to create, evaluate, and refine AI-generated documents... 
    Hourly pay
    Weekly pay
    Contract work
    Immediate start
    Remote work
    Flexible hours
    Shift work

    Ethos

    Remote
    17 days ago
  • $55 - $80 per hour

     ...more About the Opportunity Our client, a leading AI research organization, is looking for experienced commercial composers, music producers and audio post-production specialists to support AI training and evaluation. You’ll develop rights-cleared multi-stem music... 
    Contract work

    Braintrust

    Remote
    4 days ago
  • $50 - $105 per hour

     ...50 - $105/hour Required Skills Graphic Design Layout design Attention to detail...  ...About micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve... 
    Hourly pay
    Part time
    For contractors
    Remote work
    Worldwide

    micro1

    Remote
    17 days ago
  • $50 - $105 per hour

     ...SketchUp AutoCAD Vectorworks Scripts Briefs Set designer About micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve LLM... 
    Hourly pay
    Part time
    For contractors
    Remote work
    Worldwide

    micro1

    Remote
    17 days ago
  • $10 - $24 per hour

     ...live online classes for developers on building AI agents in code: tool definitions, the agent loop, planning, memory, evaluation and guardrails. You teach in English, remotely...  ...code labs or reviewing other people's system designs. Patience with developers who expect an agent... 
    Contract work
    Manual labor
    Remote work

    YPAI

    Remote
    10 days ago
  • $10 - $24 per hour

     ...Teach live online classes on grading one AI response against written criteria: accuracy, instruction following and safety, with a justification...  .... What the work is for YPAI Learn trains people for AI evaluation work in live classes. Your classes take learners from a gut... 
    Contract work
    Remote work

    YPAI

    Remote
    9 days ago
  •  ...position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for an AI Voice Evaluation Specialist based in Canada. Join an innovative AI-focused environment as an AI Voice Evaluation Specialist, where you will... 

    Jobgether

    Remote
    4 days ago
  •  ...What You'll Do Design benchmark evaluations based on real-world data analytics scenarios. Format complex data problems for systematic testing and review. Commit 20 or more hours per week to continuous project work. Who Should Apply Professional experience in data analytics... 
    Permanent employment
    Contract work
    For contractors
    Remote work

    Terac

    Remote
    3 days ago
  • $25 - $45 per hour

     ...Required Skills ChatGPT Muse AI agents Attention to Detail Following Instructions...  ...models and agents. We provide frontier evaluations and reinforcement learning environments...  ...agent sources and vets domain experts for our data platform that enables rapid... 
    For contractors
    Work at office
    Weekend work

    micro1

    Remote
    17 days ago
  • $10 - $30 per hour

     ...feedback digital delivery platforms ai-driven tools familiarity...  ...Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments...  ...recruiting agent sources and vets domain experts for our data platform that enables... 
    Part time
    For contractors
    Remote work
    Shift work

    micro1

    Remote
    25 days ago
  • $70 per hour

     ...a 10-minute call. Ethos's AI interviewer asks about your career...  ...projects like Senior Graphic Designer. Keep your CV open and cover...  ...consulting firm, bank, or agency Expert-level PowerPoint skills...  ...team or equivalent to create, evaluate, and refine AI-generated presentations... 
    Hourly pay
    Contract work
    Immediate start
    Remote work
    Flexible hours
    Shift work

    Ethos

    Remote
    15 days ago
  •  ...Curate and evaluate Java code for AI training and performance benchmarks. About Us Based in San Francisco...  ..., multilinguality, multimodality, and agents; and second, by applying that expertise...  ...cycle (prototyping, architecture design, API design, production implementation,... 
    Part time
    For contractors
    Flexible hours

    Turing

    Remote
    15 days ago
  • $80 per hour

     ...a 10-minute call. Ethos's AI interviewer asks about your career...  ...projects like Investor Deck Designer. Keep your CV open and cover...  ...VCs look for slide by slide Expert-level PowerPoint and slide craftsmanship...  ...have closed rounds to create, evaluate, and refine AI-generated... 
    Hourly pay
    Weekly pay
    Contract work
    Immediate start
    Remote work
    Flexible hours

    Ethos

    Remote
    17 days ago
  •  ...requirements Help Shape the Future of AI in Healthcare What if the...  ...healthcare information, evaluates medical scenarios, and delivers...  ...opportunities to contribute to the evaluation and improvement of advanced...  ...pathways . This opportunity is designed for candidates who can apply... 
    Contract work
    For contractors
    Traineeship
    Remote work

    Volga Partners

    Remote
    a month ago
  • $90 - $140 per hour

     ...About micro1 micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to...  ...recruiting agent sources and vets domain experts for our data platform that enables rapid... 
    Contract work
    For contractors
    Remote work

    micro1

    Remote
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Music & Design Expert - AI Agent Evaluation. Be the first to apply!