Expert AI Scientist (model evaluation, DeepEval, AWS Bedrock), banking, Montreal
Abbeal
Permanent role at Abbeal Montreal: evaluate, harden and improve the generative AI models of a major Canadian bank's AI Factory. Hybrid, 2 days on site.
Abbeal is hiring an Expert AI Scientist to join the AI Factory team of a major Canadian bank in Montreal. This is a permanent position with Abbeal.
Context
The AI Factory is a team of about 25 people that takes the bank's generative AI use cases to production. Shipping a model is not enough: you need to measure how good it is, fix it and keep watching it over time. That is the core of this role.
The role
- Evaluate AI model performance with advanced metrics (DeepEval or similar)
- Fine-tune models for banking use cases
- Design and optimise prompts for generative AI applications
- Set up monitoring of models in production
- Work with the analysts and developers of the team, on AWS Bedrock and Terraform-managed infrastructure
- Keep up with the field and share best practices
What we are looking for
- Bachelor's or master's degree in computer science, mathematics or AI
- At least 5 years in data science or AI model development
- Strong Python and AI libraries
- Hands-on Terraform and AWS
- Experience with model evaluation tools such as DeepEval
- Ability to explain technical topics simply
- Nice to have: AWS Bedrock
- Fluent French and English
Terms
- Permanent position with Abbeal (employee), no freelance or subcontracting for this mandate
- Montreal, hybrid: 2 days a week in the office
- Target start: November 2026
- 12-month assignment within the AI Factory
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Expert AI Scientist (model evaluation, DeepEval, AWS Bedrock), banking, Montreal in Montréal, QC vacancy
- ...international group of 3,000 consultants and experts across 30 countries, built on a single conviction: AI does not transform businesses.... ...you'll architect As a Senior AWS Cloud Architect, you'll design... ...AI services, including Amazon Bedrock and SageMaker, into cloud...Amazon Web ServiceFull timeContract work
Keyrus Canada
Montréal, QC1 day ago - ...partenariats clients. En tant qu'ingénieur QA / Évaluation IA, vous rejoindrez une équipe dynamique... ...and client partnerships. As a QA / AI Evaluation Engineer, you will join a highly... ...advantage of new concepts, business models, and technologies You have prior experience...SuggestedFull timeApprenticeshipImmediate start
Appnovation Technologies
Montréal, QCa month ago - ...Founded in 2017, Unit8 is a fast-growing Swiss AI and data analytics consulting and services... ...ready to operate with real autonomy in an expert-led environment, this is the role.... ...Bridging the gap between a senior data scientist and a C-suite conversation — translating technical...SuggestedFull timeWork at office
Unit8
Montréal, QC12 days ago - ...emissions. We are looking for a data scientist to join the Science and... ...preference for our office in Montreal, Canada. Key responsibilities... ...and optimize machine learning models for image classification,... ...stakeholders, including domain experts and data validators to improve...SuggestedFull timeWork at officeLocal areaRemote workFlexible hours
Ghgsat
Montréal, QC3 days ago - ...Join one of Montreal’s Top Employers in 2026. We are a leading international banking institution fully committed to building a more sustainable... ...for the use of various types of models and, accordingly, is in charge... ...quantitative reviews of the AI models used within BNP Paribas –...SuggestedLong term contractWork at officeMonday to fridayFlexible hours3 days per week1 day per week
- ...Digital Engineering, Cloud, Data & AI, Experience Design, and... ...methodology is a unique engagement model designed to align interests and... ...skills and expertise of over 4,000+ expert team members across the... ...day in-office role based out of Montreal. Information Security Responsibilities...Amazon Web ServiceFull timeWork at officeLocal areaWorldwideWork visa
Bounteous
Montréal, QCmore than 2 months ago - ...confiance. --- Aperçu du rôle Nous recherchons un DevOps IT (AWS) pour jouer un rôle clé dans l’évolution, la stabilité et la... ...Infrastructure as Code (IaC) tools such as Terraform, Ansible, GitHub, and AI‑driven automation Experience with digital certificates, PKI, and...Amazon Web ServiceFull timeFlexible hours
Portage Cybertech
Montréal, QCmore than 2 months ago - ...entreprise, vous développerez des modèles prédictifs, identifierez des... ...opérationnels Surveiller, évaluer et améliorer en permanence les... ...team’s dynamic: The Data Scientist will leverage advanced analytics... ...governance, and responsible AI Document analytical methodologies...Full timeApprenticeshipFlexible hours
Genetec
Montréal, QCmore than 2 months ago - ...include frontend and backend engineers, AI research scientists, and others from Amazon, Microsoft, and... ..., enrichment pipelines, scoring models, and sequencing workflows that run at scale... ...keep data clean and systems in sync Evaluate and adopt new AI and GTM tooling rapidly...Full time
Speechify
Montréal, QCmore than 2 months ago $84k - $126k per year
...Type de poste : Permanent Modèle de travail : Hybride... ...132340 Lieu principal : Montreal, QC Emplacements additionnels... ...connaissances grâce au mentorat d’experts et à du coaching au travail.... ...complexes, notamment des évaluations indépendantes du dérivé. Renseignements...Permanent employmentApprenticeshipWork at officeFlexible hoursDay shift- ...entreprise canadienne spécialisée dans la data, avec son siège à Montréal et des filiales à Québec et Nantes (France). Depuis près de... ...AgileDSS. Actuellement, nous avons besoin d'un Data Engineer AWS. Pré-requis du poste Ce que nous recherchons Ton rôle Dans...Amazon Web ServiceFull timeRemote workFlexible hours
agileDSS
Montréal, QCa month ago $150k - $160k per year
...Requisition Number: 106471 AI Engineer Location You will... ...strategies, guardrails, and evaluation loops to improve reliability,... ...Foundry Agent Service and Foundry Models. ~ Experience designing and... ...Notes: Toronto, Calgary, or Montreal || Ontario (CA-ON) || Canada...Full timeLocal areaRemote workInsight
Montréal, QC13 days ago$84k - $126k per year
...Type de poste : Permanent Modèle de travail : Hybride... ...132340 Lieu principal : Montréal, QC Emplacements additionnels... ...connaissances grâce au mentorat d’experts et à du coaching au travail.... ...complexes, notamment des évaluations indépendantes du dérivé. Renseignements...Permanent employmentApprenticeshipWork at officeFlexible hoursDay shift- ...seeking an Technical Project Manager for a contract position in Montreal. Description: The client is seeking a Technical Project Manager... ...vendors, internal technology managers, security subject matter experts, and risk officers in the course of managing security testing...Full timeContract workWork at office
Trigyn Technologies Inc
Montréal, QCmore than 2 months ago $60k - $120k per year
Développeur expert IA Description de poste CGI recherche un Développeur... ...Développer des serveurs MCP (Model Context Protocol) pour... ..., la supervision et l'évaluation des agents IA en production.... ...Certification IA sur Azure, AWS ou GCP. Expérience en consultation...Amazon Web Service- ...épanouit face au changement. Faites votre marque au sein du groupe Évaluation d’entreprises à titre de conseiller(ère). Cette équipe... ...servant de comparaison. Préparer des flux de trésorerie, des modèles d’évaluation et d’autres analyses et projections de nature technique...Full timeWork at office
Postes Avec Expérience
Montréal, QC11 days ago $168k - $193k per year
...Senior Machine Learning Engineer – Camera Models, you will develop and deploy machine... ...areas, from data curation and training to evaluation and deployment Write production-quality... ...this position, we are open to hiring in Montreal, Canada or remote within Canada Perks...Remote jobFull timeWork at officeRelocationTorc Robotics
Montréal, QC25 days ago- ...Faites votre marque au sein du groupe Fiscalité foncière à titre d'Évaluateur Agrée, Evaluation valeur foncière. Cette équipe diversifiée de professionnels élabore des stratégies fiscales sur mesure dans un contexte réglementaire complexe et dynamique. En tant que conseiller...Full timeWork at officeLocal area
Postes Avec Expérience
Montréal, QCmore than 2 months ago $115k - $130k per year
...group of 2,800 consultants and experts across 28 countries , built on a single conviction: AI does not transform businesses. Architected... ...Location: Hybrid – Toronto or Montreal, Canada (2 days per week at the... ...and on cloud platforms (Azure, AWS, or GCP). Working knowledge...Amazon Web ServicePermanent employmentFull timeContract workWork at officeImmediate startWork from homeFlexible hours2 days per weekKeyrus Canada
Montréal, QCmore than 2 months ago- ...in other locations in Canada Location: Montreal, QC As a Solutions Architect at Databricks... ...3rd party applications. Become an expert in, and promote Databricks inspired open-source... ...with public cloud providers such as AWS, Azure, or GCP ~[Desired] Degree in a quantitative...Amazon Web ServiceFull timeRemote work
Databricks
Montréal, QCmore than 2 months ago - ...achievements and a strong impact in the industry. About the job: Work under a supervisor with a team of experts to develop a set of power system and control component models for power system EMT simulation. Implement the models in C++ as objects. Ensure the long-term...Long term contractFull timeContract workInternshipImmediate start
Huawei Technologies Canada Co.,
Montréal, QCa month ago - ...technos qui va des agents et grands modèles de langage à la computer... ...Comprendre le besoin d'un client, évaluer le ROI et définir un scope de... ...à nos bureaux de Québec ou de Montréal quelques fois par année pour... ...mix entre un.e scientifique AI et un.e vendeur.euse : tu vois...Daily paidWork at officeFlexible hoursNight shift
- ...épanouit face au changement. Faites votre marque au sein du groupe Évaluation d’entreprises à titre de directeur(trice). Cette équipe... ...comparaison. Préparer et passer en revue des flux de trésorerie, des modèles d’évaluation et d’autres analyses et projections de nature...Full timeWork at officeLocal area
Postes Avec Expérience
Montréal, QCmore than 2 months ago - ...En tant qu'AI Scientist, tu n'as jamais cru que l'IA se résumait à appeler une API. Plutôt que d'assembler des modèles préfabriqués en espérant que ça tienne, tu comprends ce qui se passe... ...ça clique. Tu travailles avec d'autres experts et directement avec les clients, du...Work at officeFlexible hours
- ...revalorisables en temps réel | Région de Montréal ou de Trois-Rivières, Québec... ...continents. Ici, vos modèles ne restent pas en laboratoire... ...ction en temps réel : entraîner, évaluer, comparer et livrer des... ...logiciel, matériel, opérations, experts du domaine). ~ Capacité...Full timeApprenticeshipRemote work
Waste Robotics
Montréal, QCa month ago - ...Mistral Mistral provides full-stack AI solutions: from frontier models to developer tools, applications, and... ...testing. We welcome offensive security experts at all levels who are passionate about... ...: internal / Active Directory, cloud (AWS/GCP/Azure), CI/CD & supply chain •...Amazon Web ServiceFull timeRelocation package
Mistral
Montréal, QCmore than 2 months ago $70 per hour
...Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is... ...evaluate clinical EHR vignettes for a medical AI evaluation program. General and internal medicine are our...Hourly payPermanent employmentPart time10 hours per week- ...AI Evaluation Specialist Role Type: Contractor Location: Remote — US, Canada, UK, Ireland, Australia, New Zealand We are seeking experienced AI Evaluation Specialists to assess and improve the quality of AI assistant outputs for an enterprise AI training project...Remote jobFor contractors
- ...opportunity The Decision Modelling team sits at the intersection... ...develop integrated financial models and decision tools that support... ...Transaction and investment evaluation (buy-side, sell-side, infrastructure... .... Enabled by data, AI and advanced technology, EY teams...Weekend work
- ...revenue and expenses • Explain budget discrepancies and forecast differences • Create and analyze reports, dashboards and financial models in order to monitor performance Controls, Processes and Systems • Follow internal financial processes and ensure accounting...Full timeWork at office
Pharma Science Inc
Montréal, QCmore than 2 months ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Expert AI Scientist (model evaluation, DeepEval, AWS Bedrock), banking, Montreal. Be the first to apply!
