Python data engineer Job Description
Python data engineer Job Description Template
Python Data Engineers architect, build, and maintain data infrastructure. They manage data extraction, transformation, and loading (ETL) processes, ensuring efficient data pipeline workflows. Their tasks include data modeling, coding, and improving data systems.
Responsibilities:
- Design, develop and maintain Python-based data engineering pipelines for data processing, data integration, and data analysis
- Collaborate with cross-functional teams such as data scientists, data analysts, and developers to integrate data and analytics solutions into business processes
- Create, optimize and maintain data models and database schemas in a cloud environment
- Ensure data quality, integrity, and security throughout the data engineering process
- Perform data profiling, data cleansing, and data transformation to prepare data for analysis
- Implement data governance policies and procedures to ensure compliance and data privacy
- Stay up to date with new data technologies and best practices in data engineering
- Provide technical guidance and mentorship to junior data engineers and other team members
Requirements:
- Strong proficiency in Python programming language
- Experience in designing, implementing and maintaining data pipelines
- Expertise in working with SQL and NoSQL databases
- Knowledge of Big Data technologies such as Hadoop, Spark, and Hive
- Hands-on experience with ETL tools such as Airflow, Luigi, or Apache Nifi
- Experience in developing and deploying machine learning models
- Ability to troubleshoot and debug complex data issues
- Excellent communication and collaboration skills to work effectively in cross-functional teams