Etl Qa Engineer Databricks Pyspark
KUMARAN SYSTEMS
Validate enterprise data pipelines with Databricks and PySpark as an ETL QA Engineer at Kumaran Systems. Apply AI-assisted testing expertise.This ETL QA Engineer validates large-scale data pipelines and ensures data quality. The work centers on Databricks, PySpark, ETL/ELT concepts, and data warehousing. You will use GitHub, GitHub Copilot, and GitHub Actions/Runners for CI/CD pipelines. The position also includes prompt engineering, AI agents, and AI-powered knowledge bases.Key Responsibilities:
Validate large-scale data pipelines across enterprise data environmentsPerform data quality checks and reconciliation techniquesUse Databricks notebooks, jobs, clusters, Delta Lake, and SQLImprove testing efficiency and coverage with AI-assisted toolsBuild and validate AI-powered knowledge basesRequirements:
Strong hands-on Databricks experience across core platform capabilitiesExpert PySpark skills in transformations, actions, windows, and optimizationUnderstanding of ETL/ELT, warehousing, and dimensional data modelingExperience with GitHub, Copilot, and Actions/RunnersExperience crafting prompts for coding, testing, and documentationBring Databricks, PySpark, data validation, and AI-assisted QA skills to Kumaran Systems’ transformation engineering practice.
#J-18808-Ljbffr
Validate large-scale data pipelines across enterprise data environmentsPerform data quality checks and reconciliation techniquesUse Databricks notebooks, jobs, clusters, Delta Lake, and SQLImprove testing efficiency and coverage with AI-assisted toolsBuild and validate AI-powered knowledge basesRequirements:
Strong hands-on Databricks experience across core platform capabilitiesExpert PySpark skills in transformations, actions, windows, and optimizationUnderstanding of ETL/ELT, warehousing, and dimensional data modelingExperience with GitHub, Copilot, and Actions/RunnersExperience crafting prompts for coding, testing, and documentationBring Databricks, PySpark, data validation, and AI-assisted QA skills to Kumaran Systems’ transformation engineering practice.
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Etl Qa Engineer Databricks Pyspark in Winnipeg, MB vacancy
- Develop enterprise ETL pipelines with Databricks, Azure Data Factory, and PySpark in Toronto. Transform data from diverse systems into curated downstream feeds.Apex Systems seeks a Databricks Data Engineer for a large-scale data migration and centralization initiative. The assignment...Databricks
- Kumaran Systems: Engineering Future-Ready Transformations Since 1992, Kumaran Systems has been helping enterprises... ...DescriptionRole OverviewWe are looking for an ETL QA Engineer with strong experience inDatabricks and PySpark, combined with hands‑on exposure to AI‑assisted...Databricks
$80k - $120k per year
...and maintain data pipelines on Databricks for a financial services /... ...bronze → silver → gold) using PySpark, and publishes curated data to... ...You'll work alongside a senior engineer, writing transform code, fixing... ...environment healthy across dev/qa/uat. This is a good fit for...DatabricksContract work- A tech solutions company is looking for an experienced Python/PySpark Developer to design and optimize data-driven applications. This role involves developing applications, utilizing Databricks, and integrating data pipelines. The ideal candidate will possess strong skills...DatabricksRemote work
- Location: Montreal, QC (REMOTE) Type: Contract Role Overview As a Senior Databricks Data Platform Engineer you will own the engineering automation and day to day health of our enterprise Databricks Lakehouse platform This is a platform-first role rather than building...DatabricksContract workRemote work
- ...Software Developer specialized in ETL. This hybrid position requires 5+ years of experience in data engineering with expertise in Microsoft Azure, Databricks, and Power BI. Responsibilities include... ...should possess strong skills in PySpark, SQL, and quality assurance processes...Databricks
- ...ID153517BRInfosys is seeking an experienced Azure Databricks Data Engineer to design, develop, and optimize... ...in Azure data services, Databricks, PySpark, and modern data engineering practices... ...Design and develop scalable ETL/ELT pipelines using Azure Databricks and...DatabricksRelocation
- À Montréal, Avanquest recherche un ingénieur ETL maîtrisant Databricks, Python et SQL pour développer des pipelines de données fiables. Le poste s’exerce selon un mode hybride flexible, avec télétravail possible depuis l’étranger.Vous prendrez en charge l’ingestion, le nettoyage...DatabricksRemote workFlexible hours
- A leading technology firm in Toronto seeks a Software Developer - ETL to design and implement ETL processes using Azure Databricks. This role involves optimizing data flow for analytics and data lakes. Candidates should have 7+ years of ETL experience, proficiency in Python...DatabricksWork at office3 days per week
- A leading IT Services firm in Toronto is seeking a Mid-Senior Level Data Tester to join their team. You will be responsible for testing GCP applications and collaborating with senior team members to ensure project success. Ideal candidates will have 8-10 years of experience ...Full time
- The Databricks Engineer will design, develop, and optimize large-scale data pipelines using Databricks... ...requires advanced expertise in Spark/PySpark, Delta Lake technologies, data governance... ...Responsibilities Build and maintain scalable ETL/ELT data pipelines using Databricks. Use...Databricks
- A leading health information organization in Toronto seeks a Team Lead for Quality Assurance. This individual will oversee QA processes ensuring the delivery of high-quality applications. Responsibilities include leading a QA team, developing test strategies, and working on...
- ## Job Description# Databricks Data EngineerApply**Job#: 3057559****Job Description:****Databricks Data Engineer****Overview:**Apex Systems is seeking a Databricks Data... ...experience, including SQL, Python/PySpark, data migration and ETL development, with experience working...DatabricksHourly payContract workManual labor
$130k - $135k per year
...Cloud & DevOps, Data, and Software Engineering, servicing an array of noteworthy... ...with strong expertise in Databricks to join our data team. The ideal... ...RoleResponsibilities:Build and maintain scalable ETL/ELT pipelines using Databricks.Leverage PySpark/Spark and SQL to transform and...DatabricksLong term contractFull timeContract workFlexible hours- ...$95Job Overview As a Data Engineer, you will be responsible for designing... ...modular and scalable data ELT/ETL pipelines and data... ...Synapse Analytics Services, Azure Databricks, Microsoft Purview, and Power... ...Fluency with SQL, Python, and Spark/PySpark is required.A period of over 4...DatabricksLong term contract
- We are seeking an experienced Databricks Engineer to join our delivery team on a contract basis in... ...architectures on Databricks (including ETL, workload migrations, and Unity Catalog)... ...data pipelines using Azure Databricks (PySpark, SQL)Build and optimize batch and streaming...DatabricksContract work
- Join UNICEF Canada as a Data Integration Developer & QA to create ETL solutions that support child welfare initiatives. Your role will be crucial in a flexible hybrid setting. As part of UNICEF Canada, you will design and manage data integration solutions critical to our fundraising...Flexible hours
- Build data pipelines with Python, PySpark, and Snowflake while transforming large-scale datasets... .... Apply deep SQL expertise to reliable engineering work.This Data Engineer role focuses on... ...Snowflake development. You will create ETL and ELT solutions, integrate data, and support...Internship
- ...The Data Engineer will design, develop, and maintain scalable data pipelines and workflows using Python, PySpark, and cloud technologies. This role involves managing data storage solutions, orchestrating ETL processes, and deploying containerized applications to support...
- ...and progress8-10 years of overall experience with 3-4 years in Data testingKnowledge of GCP Big Query, Python and PysparkExperience in ETL and DB testing, Data migration, Report validationsExpert in writing understanding complex SQL queriesExcellent communication skills and...
- Senior Databricks Data Engineer Location: Toronto (Hybrid - minimum 2 days per week onsite at the client... ...Migrate and refactor existing ETL/ELT pipelines, scripts and processing workloads... ...-quality data pipelines using Python, PySpark and SQL. Translate existing...DatabricksContract workWork at office2 days per week
- Data Engineer - Databricks (Remote)We work with global enterprises to design and build data platforms that power digital products... ...’ll Do Design, build, and maintain scalable ETL / ELT pipelines using Databricks, PySpark, and AWS Work with large, complex datasets...DatabricksFull timeRemote workMonday to fridayFlexible hours
- Azure Data Engineers with Databricks expertise will build scalable pipelines using SQL, Python, and cloud ETL tools across Azure environments.This position focuses on designing, implementing... .... You will transform datasets with PySpark and Spark, integrate Azure data sources, and...Databricks
- Lead Databricks migrations with Python, PySpark, and SQL from a hybrid Toronto engagement. Modernize production pipelines... ...two-week Agile delivery.This senior data engineering contract spans two months and focuses on migrating existing ETL/ELT workloads into Databricks. You will...DatabricksContract work
- Build GCP data migration solutions with PySpark and Salesforce APIs, focusing on Dataproc, BigQuery, and scalable ETL pipelines.This Data Migration Engineer role requires four to five years minimum of experience. You will use Python, Apache Spark, BigQuery, GCS, and Salesforce...
- ...Description Role OverviewWe are seeking an experienced Senior Databricks Engineer with a deep background in distributed data systems to... ...batch and real-time streaming ingestion pipelines using PySpark, Spark SQL, Delta Lake, Auto Loader, and Databricks Workflows...Databricks
- Databricks Lead – Python, PySpark, SQL & Supply Chain Analytics Experience Required: 6-8 YearsRole SummarySeeking a Databricks Lead with strong expertise in Databricks to design, build, and operationalize scalable Supply Chain data products that deliver measurable...Databricks
- A leading data solutions firm in Toronto is seeking a QA Lead to manage ETL testing across data migration and analytics projects. With over 5 years of ETL/Data QA experience, including at least 2 years in a leadership role, the ideal candidate will have strong SQL and ETL skills...Contract work
$34.88 - $54.5 per hour
Data Engineer – Databricks (Mining Operations) We are seeking a Data Engineer to maintain and enhance existing data pipelines that ingest data from... ...optimization in Databricks environments Python / PySpark Spanish language proficiency The pay range that the employer...DatabricksContract work- ...We are seeking an experienced Python/PySpark Developer to design, develop, and optimize data... ...strong expertise in Python, PySpark, and Databricks, along with experience in Azure DevOps and... ...applications using Python and PySpark. Implement ETL/ELT processes and integrate data pipelines...DatabricksFull time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Etl Qa Engineer Databricks Pyspark. Be the first to apply!
Related searches
- software quality assurance engineer Winnipeg, MB
- software test engineer Winnipeg, MB
- qa engineer Winnipeg, MB
- senior quality assurance engineer Winnipeg, MB
- remote quality assurance Winnipeg, MB
- qa software testing Winnipeg, MB
- quality assurance Winnipeg, MB
- stage assurance qualité Winnipeg, MB
- responsable assurance qualité Winnipeg, MB
- entry level qa Winnipeg, MB

