Product Designer - AI Creative Evaluation (Contra Labs)
Temporary
Contra Labs
About Contra Labs
Contra Labs sits at the intersection of human creativity and AI. We partner with leading AI labs and product companies to improve their tools across design, video, development, and beyond. Our research is grounded in how top creatives actually work, studying the patterns behind what gets created, what gets chosen, and what resonates. We believe human input will always be at the center of net new creation, and our mission is to help shape technology so human creativity can be amplified to create what today is unimaginable.
By joining the Contra Labs network you gain access to paid projects with some of the most innovative companies building the next generation of creative and AI-powered tools.
How It Works
This is an application to join the network and be eligible for roles as they arise. Projects are staffed on an as-needed basis, typically 5 to 10 hours at a time, though some may extend into longer engagements. In unique cases, we may have full-time roles available as well. When a project matches your skill set, we'll reach out with details and timeline.
To Join the Network
- Portfolio review - We'll assess your Contra profile and portfolio for strength in product design, specifically across desktop and mobile applications.
- Record a 30 second intro video - Share a quick 30 second introduction so we can get to know you.
- Complete profile - Once you’ve applied, we recommend completing your profile for better job matches and setting up your wallet so you can get paid when a job comes through.
What You May Work On
- Evaluate AI-generated product design outputs across desktop and mobile app interfaces, applying professional design standards and best practices
- Provide clear, structured feedback on UI/UX quality including layout, hierarchy, interaction patterns, accessibility, and visual polish
- Help define creative benchmarks and taste standards for what great product design looks like in the next generation of AI-powered tools
- Contribute to research on how convergence (shared best practices) and divergence (individual taste and creative direction) shape design quality
What We're Looking For
- Strong portfolio showcasing high-quality product design work across desktop and/or mobile applications, shared via your Contra profile
- Deep understanding of what makes app interfaces intuitive, polished, and well-crafted, from information architecture to micro-interactions
- Ability to articulate design and UX feedback clearly and constructively
- Comfort working in fast-paced, experimental environments (experience with AI or design tools is a plus)
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Product Designer - AI Creative Evaluation (Contra Labs) in Remote vacancy
- ...About Contra Labs Contra Labs sits at the intersection of human creativity and AI. We partner with leading AI labs and product companies to improve their tools across design, video, development, and beyond. Our research is grounded in how top creatives actually work, studying...CreativeFull timeContract work
- ...About Contra Labs Contra Labs sits at the intersection of human creativity and AI. We partner with leading AI labs and product companies to improve their tools across design, video, development, and beyond. Our research is grounded in how top creatives actually work, studying...CreativeContract workImmediate start
- ...About Contra Labs Contra Labs sits at the intersection of human creativity and technology. We partner with leading product companies to improve the creative tools that professionals use every day. Our research is grounded in how top creatives actually work, studying the...CreativeFull timeContract work
- ...Evaluate AI model safety by designing adversarial test prompts and classifications. About Turing: Based in San Francisco, California, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced...CreativeOngoing contractPart timeFor contractors
- ...and next steps. Our partner is looking for an AI Evaluation Engineer based in Canada. As a founding AI Evaluation Engineer, you will design and build the evaluation, testing, and observability infrastructure that ensures production-grade AI agents perform reliably, safely,...SuggestedFull timeRemote work
- ...We're building a research cohort of senior domain experts in creative technology — spanning audio, design, and UX — to help evaluate how frontier AI models reason about hard, real-world expert-level problems. This is research-and-evaluation work, not production engineering...CreativeContract work
- © 2026 Smart Ecosystems. All rights reserved.Contract work
- ...Key Responsibilities • Design and implement evaluation metrics for AI models • Conduct thorough testing of model performance across different scenarios • Analyze model outputs for bias, fairness, and accuracy • Collaborate with ML engineers to implement improvements...Contract work
$40 per hour
...of our site so we can improve and personalize our products and services, for analytics and marketing, and to fulfill... ...All Employers Job seekers Career centers Handshake AI Enterprise AI Research Log in Sign up FRENCH AI EVALUATION SPECIALIST (CANADA) Sign up now Up to $40/hr...Part timeRemote workFlexible hoursusd16 per hour
...We are looking for AI Linguistic Evaluators to assess how effectively an AI application performs in Marathi for a short collaboration. Job Type... ...naturalness, and cultural appropriateness. - Providing structured evaluations and feedback following the project instructions and quality...Hourly payDaily paidFull timeFreelanceImmediate startRemote workFlexible hours$35 - $49 per hour
...Put frontier AI models through real process engineering problems and mark... ...About the role Sovrano AI builds evaluation and training data for frontier AI labs, and this project tests how well... ...for the paid bootcamp on Monday and production from Tuesday Able to commit roughly...Hourly payContract workPart timeFor contractorsManual laborRemote workShift work- ...steps. Our partner is looking for a Senior Product Designer based in Canada. Join an innovative, fast... ...recordkeeping toward intelligent, AI-powered experiences that actively help customers... ..., and designers to define requirements, evaluate technical trade-offs, and translate...Full timeWork at office
- ...s shaping an exciting array of product solutions, including Video-Based... ...the long term. The Associate Creative Director will play a pivotal role... ...strategic thinking, hands-on design expertise, and collaboration across... ...experiences can be in an AI-first world — moving beyond conventional...CreativeLong term contractFull timeRemote work
$75 - $90 per hour
...years Location: United States | Canada + 21 more Help a top AI lab evaluate and improve large language models through security-focused... ...code review, coding annotation, LLM/code evaluation or benchmark design. Knowledge of another programming language. Team...Long term contractContract workRemote work$13 per hour
...Location: Argentina | Belize | Bolivia + 15 more Help evaluate how an AI assistant uses information from connected Google apps to respond... .... • Follow project guidelines for consistent, confidential evaluation work. Important account-access requirement • This project...Hourly payContract workFor contractorsRemote work- ...Digital is building a global community of experts to evaluate, challenge and improve advanced AI systems . We are looking for university students,... ...scenarios Responsible AI behaviour Domain-Specific AI Evaluation Healthcare & Medicine Law Finance, Economics &...Part time
- ...shaping an exciting array of product solutions, including Video-Based... ...this role: As a Staff Product Designer at Samsara, you will set a... ...and best efforts to bring forth creative, ambitious ideas for our customers... ...product decisions. - Leverage AI tools through every step of...CreativeLong term contractFull timeRemote work
- ...Join a leading AI lab's cutting-edge GenAI team to be at the core of the AI revolution, where your expertise fuels the... ...done in practice. Benchmarks and domain depth: Design challenging finance tasks and evaluation sets, and help build finance-specific skills and tools...Full timeContract workPart timeFreelanceLive InManual laborRelocationRelocation package
- ...Happy! About the Role You'll own product experiences end-to-end. From... ...is a high-autonomy role for designers who operate independently, think... ...and are already building with AI – not just aware of it. We're... ...them. You balance rigour with creativity. - High autonomy. You move work...Full timeRemote workFlexible hours
- ...contractor Remote · Contract · Radiological Science · Health Physics · AI Safety · CBRN What this pays Rates are set by the client... ...a figure we cannot stand behind. About the Role AI Safety Evaluation Project | Remote | Contract Locations: US, Canada, UK, Europe,...Hourly payContract workFor contractorsRemote work
$55 - $100 per hour
...Pay: $55–100/hour You'll evaluate AI-generated responses to legal questions — covering case analysis, statutory interpretation, procedural questions, and litigation strategy. Your primary task is assessing whether a model's answer reflects sound legal reasoning, accurate...Contract work- ...develop systems to enhance frontier AI capabilities. About Turing:... ...accelerator for frontier AI labs and a trusted partner for... ...enterprises. Our environments evaluate and improve AI models on complex... ...investigate high-impact questions, design rigorous experiments, build research...Contract workPart timeFor contractors
$30 - $90 per hour
...backgrounds to support quality control on AI reasoning datasets. This role requires professional... ...Nice to Have Experience with AI/ML evaluation or data annotation Background in AI,... ...on location, experience, and educatioanl background). Company: Leading AI Lab...Contract workRemote work10 hours per weekFlexible hours- ...What You'll Do Apply your subject-matter expertise to advanced AI training and evaluation work Develop or evaluate challenging problems within your area of expertise Assess AI-generated reasoning for technical accuracy and identify subtle errors or failure modes...Contract workImmediate startRemote workFlexible hours
- ...What You'll Do Review real user interaction traces with an AI shopping assistant Identify logical failures, inaccuracies, or poor... ...verifiers to judge response quality Commit to 20+ hours per week of evaluation work on our internal platform Who Should Apply Experience...Contract workFor contractorsRemote work
- ...faster, stay on brand, and get AI marketing that actually works.... ...ideas, and generate creative at a speed and quality that wasn... ...We are looking for a Staff AI Productivity Engineer to accelerate how our... ...technical interviews focus on how you design systems because we believe...CreativeFull timeRemote work
$70 per hour
...Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is... ...evaluate clinical EHR vignettes for a medical AI evaluation program. General and internal medicine are our...Hourly payPermanent employmentPart time10 hours per week- ...shape, prioritize, and launch the products they truly need. That’s what... ...seeking a Lead Product Growth Designer to lead design initiatives... ...balance of being data-driven, creative problem-solving, product strategy... ..., and collaboration. - Build AI-enhanced workflows and use AI...CreativeFull time
- ...steps. Our partner is looking for a Staff AI Product Manager based in Canada. This is an... ...deep understanding of AI architectures, evaluation frameworks, and responsible deployment.... ...influence how enterprise AI products are designed, evaluated, and delivered. If you thrive...Long term contractFull time
- ...Help a leading AI lab define what great design looks like, and build the prototypes that prove it. # Overview A leading AI lab is looking for senior product designers and design-minded engineers with exceptional taste to help shape how its AI models design and build...CreativeFull timeContract workPart timeWeekday work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Product Designer - AI Creative Evaluation (Contra Labs). Be the first to apply!
