Data Engineer Python/Pyspark/Openshift
Compunnel Inc.
The Data Engineer will design, develop, and maintain scalable data pipelines and workflows using Python, PySpark, and cloud technologies.
This role involves managing data storage solutions, orchestrating ETL processes, and deploying containerized applications to support enterprise data initiatives.
Key Responsibilities
Develop, construct, test, and maintain scalable data pipelines and ETL/ELT workflows using Python, PySpark, and SQL.
Manage and optimize data storage solutions in Snowflake and Data Lakes.
Integrate data from multiple sources while ensuring data quality, integrity, and security.
Design and implement automated workflows using Azure Data Factory or similar orchestration tools.
Deploy and manage containerized applications and services on OpenShift.
Collaborate with Data Scientists, Analysts, and stakeholders to understand data requirements.
Monitor and troubleshoot data pipelines, resolving performance bottlenecks.
Maintain documentation of data architecture, workflows, and best practices.
Ensure compliance with data governance and security policies.
Required Qualifications
Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
Minimum 10 years of overall experience in data engineering or similar roles.
Strong proficiency in Python and PySpark.
Expertise in SQL and experience with Snowflake Data Warehouse.
Hands-on experience with data orchestration tools such as Azure Data Factory.
Knowledge of Data Lakes architecture and management.
Experience deploying and managing containerized applications on OpenShift or Kubernetes.
Familiarity with cloud platforms (Azure, AWS, GCP).
Understanding of data security, governance, and compliance best practices.
Strong problem-solving skills and attention to detail.
Excellent communication and teamwork abilities.
Preferred Qualifications
Experience with big data tools (e.G., Kafka, Hadoop).
Knowledge of CI/CD pipelines and DevOps practices.
Familiarity with additional cloud data services and tools.
#J-18808-Ljbffr
This role involves managing data storage solutions, orchestrating ETL processes, and deploying containerized applications to support enterprise data initiatives.
Key Responsibilities
Develop, construct, test, and maintain scalable data pipelines and ETL/ELT workflows using Python, PySpark, and SQL.
Manage and optimize data storage solutions in Snowflake and Data Lakes.
Integrate data from multiple sources while ensuring data quality, integrity, and security.
Design and implement automated workflows using Azure Data Factory or similar orchestration tools.
Deploy and manage containerized applications and services on OpenShift.
Collaborate with Data Scientists, Analysts, and stakeholders to understand data requirements.
Monitor and troubleshoot data pipelines, resolving performance bottlenecks.
Maintain documentation of data architecture, workflows, and best practices.
Ensure compliance with data governance and security policies.
Required Qualifications
Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
Minimum 10 years of overall experience in data engineering or similar roles.
Strong proficiency in Python and PySpark.
Expertise in SQL and experience with Snowflake Data Warehouse.
Hands-on experience with data orchestration tools such as Azure Data Factory.
Knowledge of Data Lakes architecture and management.
Experience deploying and managing containerized applications on OpenShift or Kubernetes.
Familiarity with cloud platforms (Azure, AWS, GCP).
Understanding of data security, governance, and compliance best practices.
Strong problem-solving skills and attention to detail.
Excellent communication and teamwork abilities.
Preferred Qualifications
Experience with big data tools (e.G., Kafka, Hadoop).
Knowledge of CI/CD pipelines and DevOps practices.
Familiarity with additional cloud data services and tools.
#J-18808-Ljbffr
Vacancy posted 58 minutes ago
Similar jobs that could be interesting for youBased on the Data Engineer Python/Pyspark/Openshift in Winnipeg, MB vacancy
- Build data pipelines with Python, PySpark, and Snowflake while transforming large-scale datasets through distributed processing. Apply deep SQL expertise to reliable engineering work.This Data Engineer role focuses on Python programming, PySpark transformations, and Snowflake...SuggestedInternship
Lancesoft
Winnipeg, MB57 minutes ago - A tech solutions company is looking for an experienced Python/PySpark Developer to design and optimize data-driven applications. This role involves developing applications, utilizing Databricks, and integrating data pipelines. The ideal candidate will possess strong skills in...SuggestedRemote work
Compunnel, Inc.
Winnipeg, MB1 hour ago - ...Luxoft in Toronto is seeking a senior data engineer to support the Analytical Work Bench integration, including AWS-based data pipelines and... ...quality with standard testing practices. Join a team that leverages PySpark, PyTest, and Pylint, while integrating with CI/CD tooling such...Suggested
Luxoft (old)
Winnipeg, MB58 minutes ago - # Forward Deployed Engineer (Python/Pyspark Developer)Apptoza Inc. · Toronto, Ontario, CABack to resultsForward Deployed Engineer (Python/Pyspark Developer... ...Engineer who combines production-grade software engineering, data engineering, AI application development, and client-facing...Suggested
Linuxassociation
Winnipeg, MB58 minutes ago - ...We are seeking an experienced Python/PySpark Developer to design, develop, and optimize data-driven applications and solutions. The ideal candidate will have strong expertise in Python, PySpark, and Databricks, along with experience in Azure DevOps and enterprise data integration...SuggestedFull time
Compunnel Inc.
Winnipeg, MB1 hour ago - Build analytical data solutions with Python, PySpark, and AWS in downtown Toronto, working onsite four days weekly on a hybrid contract.This senior development assignment supports a global IT services firm serving a leading investment and wealth management services firm. You...Contract workWork at office
C.G. Consulting Group
Winnipeg, MB57 minutes ago - Apply your advanced skills in a role as an Expert Python/PySpark Developer, dedicated to enhancing data-driven solutions. This position offers flexible modes of teamwork and application development. We are in search of an experienced Python/PySpark Developer, enriched with...Flexible hours
Compunnel Inc.
Winnipeg, MB56 minutes ago - ...Project description The project involves the data engineering organization’s Analytical Work Bench , which is an integrated development... ...solutions utilizing AWS EMR. Advanced proficiency in Python Core and PySpark, with over 8 years of hands-on experience. Practical...
Luxoft Holding
Winnipeg, MB1 hour ago $85k - $90k per year
...Description Design and maintain scalable data pipelines that collect, transform and... ...or Diploma in Computer Science, Data Engineering, Information Systems, or related field.... ...Engineer or similar role. · Proficiency in Python, PySpark, data modeling, ETL processes and data...Permanent employment- Design cloud data pipelines with PySpark and AWS Glue, supporting secure, high-performing ecosystems for analytics and operational efficiency.As a... ...requirements into technical designs with business analysts and data engineers.Key Responsibilities: Design end-to-end pipelines and...
Kaizen Technologies
Winnipeg, MB1 hour ago - Use Python, Spark, and Airflow to create scalable pipelines for Robinhood’s data lake. Work in Toronto on foundational financial datasets.This Senior Data Engineer role focuses on reliable data infrastructure supporting Robinhood’s metrics and products. You will develop application...
Robinhood
Winnipeg, MB1 hour ago - Python data engineering leads this intermediate position in Toronto, with backend services, analytics, and hybrid work two days onsite weekly.Randstad Canada presents an intermediate Software Engineer opportunity with a leading SaaS company. The work centers on backend development...Work at office
Linuxassociation
Winnipeg, MB55 minutes ago - Build trusted data foundations at Foresight using Python and advanced SQL. Work from New York, San Francisco, or Montreal, with occasional travel.As a Senior Data Engineer, you will develop production pipelines for complex project data. Your work supports predictions, AI recommendations...
Foresight Works
Winnipeg, MB57 minutes ago - Build Python systems for data lineage, code parsing, and metadata analysis with I8IS INC. in Mississauga... ..., Ontario. Develop accurate, scalable engineering solutions.This full-time developer... ...transformation.Key Responsibilities: Design engines parsing programming and query...Full time
Linuxassociation
Winnipeg, MB58 minutes ago - Step into a senior Data Engineer role focusing on developing data pipelines with Python and SQL. This position is ideal for experts in cloud technologies. You'll be responsible... ...and quality, while managing deployments on OpenShift and utilizing Azure Data Factory for process...
Compunnel Inc.
Winnipeg, MB56 minutes ago - ...professionals, and our annual growth rate exceeds 25%. Job Description Miratech is looking for a skilled Senior AI Engineer with Python, LangGraph and LangChain to join our US healthcare client. You will play a key role in developing and enhancing an enterprise AI...Full timeInternshipRemote workRelocation
- Build scalable data solutions in Markham, ON, using advanced SQL and Python with Pandas in a hybrid, three-days-on-site schedule.This Senior Data Engineer, Senior BI, or Analytics Engineering Lead designs reliable solutions for business intelligence, reporting, and advanced...
Altis
Winnipeg, MB1 hour ago - Engineer monitored, governed data flows with Python and Spark for Croesus's wealth management platforms. This hybrid position requires two weekly office days in Laval or Montréal.The Data Engineering Expert develops and operates modern Databricks Lakehouse pipelines supporting...Work at office
Croesus
Winnipeg, MB1 hour ago - Shape Python data processing workflows and backend APIs for an AI-driven risk intelligence platform, working remotely or hybrid in Montreal.As a Senior Software Engineer, you will contribute to an evolving product that screens media for compliance and due diligence. You will...Remote work
Mission
Winnipeg, MB1 hour ago - Build reliable data systems remotely at Intrinsic Development using Python, SQL, and modern ETL pipelines. Daily AI use shapes this engineering work.As a Data and AI Engineer, you will build and operate data platforms in a product-led environment. The work includes ETL and ELT...Remote work
Y-Axis
Winnipeg, MB1 hour ago - Build Python tooling and APIs as a Data Platform Engineer at BRP, improving platform efficiency through automation, CI/CD, and scalable operations.Working in BRP’s Data Solutions group, you will extend data platform capabilities and improve efficiency. You will develop custom...
BRP
Winnipeg, MB1 hour ago - Build scalable data pipelines with SQL and Python as an in-office Data and Software Engineer at General Bank of Canada in Edmonton. Develop reliable data solutions for enterprise banking operations.This permanent full-time role supports data integration, transformation, modeling...Permanent employmentFull timeWork at office
General Bank Of Canada
Winnipeg, MB57 minutes ago - Software Engineer, Data Systems (Python)Remote - Canada About NorthbeamNorthbeam is building the world's most advanced marketing intelligence platform, providing top eCommerce brands a unified view of their business data through powerful attribution modeling and customizable...Remote workWork from homeFlexible hours
Northbeam LLC
Winnipeg, MB1 hour ago - Apply advanced Python and SQL skills to large-scale data engineering at Ipsos. Build cloud pipelines and warehouses within the Ipsos DataHub ecosystem.This Senior Data Engineer role focuses on ingesting, processing, and storing complex datasets across cloud-based platforms....
Ipsos in Canada
Winnipeg, MB57 minutes ago - OpenShift and Kubernetes expertise define this hands-on DevOps Engineer role in Toronto, with hybrid work requiring four days onsite weekly.Support and maintain applications... ...and container technologiesShell, Bash, Python, or similar scripting experienceAnalytical, self-...
Epsilon Solutions Ltd.
Winnipeg, MB1 hour ago - Build complex data pipelines with Python and SQL Server while working onsite five days weekly alongside operational teams. Maintain extremely accurate reporting solutions and data systems.As a Data Engineer, you will develop, test, deploy, and maintain ELT pipelines from varied...
Petroglyph Development Group
Winnipeg, MB1 hour ago - OpenShift AI security expertise defines this remote part-time contract with MatchBox Consulting... ...located in Sask.As a Senior DevSecOps Engineer, you will secure OpenShift AI environments... ...forwarding OpenShift logs and audit data to SIEMApply senior DevSecOps expertise across...Contract workPart timeRemote work
MatchBox Consulting Group
Winnipeg, MB1 hour ago - Validate enterprise data pipelines with Databricks and PySpark as an ETL QA Engineer at Kumaran Systems. Apply AI-assisted testing expertise.This ETL QA Engineer validates large-scale data pipelines and ensures data quality. The work centers on Databricks, PySpark, ETL/ELT concepts...Work at office
KUMARAN SYSTEMS
Winnipeg, MB59 minutes ago - ...leading IT Services firm in Toronto is seeking a Mid-Senior Level Data Tester to join their team. You will be responsible for testing... ...years of experience in Data testing, proficient in GCP Big Query, Python, and SQL, and possess excellent communication skills. The position...Full time
Astra-North Infoteck Inc.
Winnipeg, MB59 minutes ago - ...scale analytical systems with Apache Spark, Core Java, Python, SQL, and Linux in this experienced Software Engineer opportunity.This Software Engineer role centers on... ...-performance application development and analytical data processing. The position calls for senior hands-on...
PeoplePilot
Winnipeg, MB58 minutes ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer Python/Pyspark/Openshift. Be the first to apply!
Related searches
- data integration developer Winnipeg, MB
- data centre engineer Winnipeg, MB
- data engineer Winnipeg, MB
- python data engineer Winnipeg, MB
- big data engineer Winnipeg, MB
- gcp data engineer Winnipeg, MB
- remote data engineer Winnipeg, MB
- data cabling engineer Winnipeg, MB
- data engineering intern summer Winnipeg, MB
- junior data engineer Winnipeg, MB