Software Engineer/Data Engineer Job Description Template
Our company is looking for a Software Engineer/Data Engineer to join our team.
Responsibilities:
- Develop, monitor and communicate data SLA’s to stakeholders;
- Work closely with other team members to promote consistency and reliability of infrastructure across environments;
- Develop, document, test a large variety of back end and server side components in Java, Python, Scala on Linux;
- Investigate and resolve performance issues and automation failures;
- Leverage massive datasets for modeling, recommendations, and reporting solutions;
- Drive framework for A/B tests, exposing the results through visualization tools like Tableau;
- Create, scale, and maintain data pipelines using Airflow and DBT;
- Design and implement data pipelines empowering real time insights;
- Maintain and develop operational automation and monitoring for tools and data components in production;
- Develop complex data processing and analytics pipelines using cutting edge tools like Apache Spark, Apache Kafka and Elasticsearch;
- Contribute to design;
- Build user-facing scalable systems powering ad targeting, push and/or in appmessaging;
- Evangelize best practices in continuous integration and delivery with Docker and Kubernetes;
- Scale and maintain Carta’s Redshift data warehouse.
Requirements:
- Hands-on experience with large scale Big Data environments (Spark, Kafka, Hive, Hadoop);
- A deep appreciation for agile software processes and data-driven development;
- MS/Bachelor or equivalent in Computer Science or related field, 3-5 years of relevant industry experience;
- Strong coding skills preferably in Java, Scala, or Python;
- A collaborative attitude and helpful personality;
- 3+ years of industry experience working with ETL pipelines and data warehousing;
- Good understanding of distributed systems and algorithms, network programming, concurrency, file system;
- Computer Science fundamentals – algorithms, data structures, programming paradigms;
- BS/MS in Computer Science or closely related field;
- Ability to handle multiple competing priorities in a fast-paced environment;
- Comfort with the Linux terminal and unix tools;
- Quick Learner, able to grasp and prototype new tools and ideas quickly;
- Proficiency with modern programming languages (Python, Java), datastores (Postgres, Redshift) and SQL;
- Strong written and verbal technical communication skills;
- Creative thinking, able to come up with alternative solutions to fit business requirements.