Expert Data Engineering

Job Responsibility: At CN, we work together to move our company—and North America—forward. Be part of our Information & Technology (I&T) team, a critical piece of the engine that keeps us in motion. From enterprise architecture to operational technology, our teams use the agile methodology to automate and digitize our railroad ensuring our operations run optimally and safely and our employees can focus on value-added tasks. You will be able to develop your skills and career in our close-knit, safety-focused culture working together as ONE TEAM. The careers we offer are meaningful because the work we do matters. Join us! Job Summary

The Expert, Data Engineering is responsible to lead/mentor a team of specialists in the process of building, managing, and optimizing data pipelines, moving them effectively into production for key data and analytics consumers, shaping the enterprise Data as a Service (DaaS) model and delivering on IT business models. The Expert Data Engineering develops best practices and optimizes data pipelines to deliver products and services aligned with business expectations. The incumbent plays a pivotal role in operationalizing data and analytics initiatives, defining and building CN's data integration and delivery roadmap.

Main Responsibilities

Data Engineering

• Ensure optimal data delivery architecture and processes are consistent throughout ongoing projects

• Optimize CN's data architecture to support the next generation of products and data initiatives

• Build, manage, and optimize data pipelines, moving them effectively into production for key data and analytics consumers such as business or data analysts and data scientists

• Build data and domain event models, implement business rules, and engineer scalable data pipelines

• Ensure compliance with data governance and security requirements while creating, improving and operationalizing integrated and reusable data pipelines

• Enable faster data access, integrate data reuse, and improve time-to-solution for data and analytics initiatives

• Integrate analytics and data science results with business processes

• Promote effective data management practices

Data Design

• Collaborate with stakeholders and architects to model data landscape and define secure data exchange approaches

• Meet with stakeholders to identify fit-for-purpose within CN's existing data ecosystem and deliver options and agile solutions

• Design and develop processing pipelines that ingest data into Data Hubs

• Expose the data into meaningful formats, like GraphQL

• Provide day-to-day support and technical expertise to both technical and non-technical teams

• Participate in building data engineering expertise and framework

• Translate business needs into technical requirements

Data Development

• Acquire, ingest and process data from multiple sources and systems

• Design and develop Extract, Transform, and Load (ETL) pipelines using multiple sources of data in various formats

• Conduct metadata management, data cleansing and preparation

• Produce well documented quality codes

Data Operations

• Use Agile methodologies to streamline project delivery aligned with goals, timelines, and budget

• Build monitoring and debugging tools to analyze data pipelines

• Help unify software development and operations seamlessly, efficiently, and cost effectively

• Improve software quality, automate processes, and accelerate software releases

Data Quality Assurance

• Use Agile development practises for code reviews and testing to develop and deliver data pipelines

• Develop and implement test plans and scripts for various data quality processes

• Maintain manual and automated test scripts

Working Conditions

The role has standard working conditions in an office environment with a regular workweek from Monday to Friday and offers remote work.

Requirements

Experience

Data Engineering

• Minimum 6 years of experience in a data engineering role, working in different data management disciplines including data integration, modelling, optimization and quality

• Experience working in cross-functional teams and collaborating with business stakeholders in support of a departmental or multi-departmental data management and analytics initiative

• Experience with at least 2 of the following technologies: Python, Scala, SQL, Java

• Experience with Spark Streaming with Java or Scala is a MUST

• Experience with Databricks is a MUST

• Experience with Cloud and non-Cloud based Hadoop ecosystem

• Experience with Cloud platforms such as Azure, Google Platform

• Experience with Linux and shell scripting

• Experience/Knowledge on GraphQL

• Experience deploying applications into production environments such as code packaging, integration testing, monitoring and release management

• Experience with multiple database technologies such as Distributed Processing (Spark, Hadoop, EMR), traditional RDBMS (MS SQL Server, Oracle, DB2, MySQL, PostgreSQL), NoSQL (MongoDB, DynamoDB, Cassandra)

• Experience working in an Agile team environment leading a team with parallel deliverables for multiple clients

Education/Certification/Designation

• Bachelor's Degree in Computer Science, Electrical Engineering or Software Engineering

• Master's Degree or PhD in Data Management, Data Analytics, Information Systems, or a related quantitative field*

• Professional Engineer (P.Eng.) Degree*

• Google or Azure Data Engineering or Databricks certification*

• Any designation for these above would be considered as an asset

Competencies

• Ability to communicate complex solutions

• Applies analytical thinking

• Innovates through problem solving

• Knows the business and stays current with industry trends to elevate expertise and work

• Demonstrates organizational abilities and detail oriented

• Collaborates with key internal stakeholders to enable higher productivity

• Works independently with little supervision

• Applies agile mindset

• Ability to work across structured, semi-structured, and unstructured data, extracting information and identifying linkages across disparate data sets

• Demonstrates commitment to high standards of ethics, regulatory compliance, customer service, and business integrity

About CN CN is a world-class transportation leader and trade-enabler. Essential to the economy, to the customers, and to the communities it serves, CN safely transports more than 300 million tons of natural resources, manufactured products, and finished goods throughout North America every year. As the only railroad connecting Canada's Eastern and Western coasts with the Southern tip of the U.S. through a 19,500 mile rail network, CN and its affiliates have been contributing to community prosperity and sustainable trade since 1919. CN is committed to programs supporting social responsibility and environmental stewardship. At CN, we work as ONE TEAM, focused on safety, sustainability and our customers, providing operational and supply chain excellence to deliver results. For internal candidates, note that the grade level of the position will depend on the employee's experience.

At CN, we are dedicated to building North America's safest, most inclusive and sustainable railroad, which includes reflecting the communities in which we operate. Research shows that candidates from underrepresented groups often don't apply unless they feel they fit the job posting at 100%. Even if you don't see yourself in every job requirement listed in a posting, we still encourage you to apply. If you require an accommodation for the recruitment process (including alternate formats of materials, accessible meeting rooms or other accommodations), please reach out to our team at View email address on ca.edajobs.com.

As an equal employment opportunity employer, all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, protected veteran status, and other protected status as required by applicable law. We thank all applicants for their interest, however, only candidates under consideration will be contacted. Please monitor your email on a regular basis, as communication is primarily made through email.