Software Engineer/Data Engineer

Software Engineer/Data Engineer Job Description Template

Our company is looking for a Software Engineer/Data Engineer to join our team.

Responsibilities:

  • Develop, monitor and communicate data SLA’s to stakeholders;
  • Work closely with other team members to promote consistency and reliability of infrastructure across environments;
  • Develop, document, test a large variety of back end and server side components in Java, Python, Scala on Linux;
  • Investigate and resolve performance issues and automation failures;
  • Leverage massive datasets for modeling, recommendations, and reporting solutions;
  • Drive framework for A/B tests, exposing the results through visualization tools like Tableau;
  • Create, scale, and maintain data pipelines using Airflow and DBT;
  • Design and implement data pipelines empowering real time insights;
  • Maintain and develop operational automation and monitoring for tools and data components in production;
  • Develop complex data processing and analytics pipelines using cutting edge tools like Apache Spark, Apache Kafka and Elasticsearch;
  • Contribute to design;
  • Build user-facing scalable systems powering ad targeting, push and/or in appmessaging;
  • Evangelize best practices in continuous integration and delivery with Docker and Kubernetes;
  • Scale and maintain Carta’s Redshift data warehouse.

Requirements:

  • Hands-on experience with large scale Big Data environments (Spark, Kafka, Hive, Hadoop);
  • A deep appreciation for agile software processes and data-driven development;
  • MS/Bachelor or equivalent in Computer Science or related field, 3-5 years of relevant industry experience;
  • Strong coding skills preferably in Java, Scala, or Python;
  • A collaborative attitude and helpful personality;
  • 3+ years of industry experience working with ETL pipelines and data warehousing;
  • Good understanding of distributed systems and algorithms, network programming, concurrency, file system;
  • Computer Science fundamentals – algorithms, data structures, programming paradigms;
  • BS/MS in Computer Science or closely related field;
  • Ability to handle multiple competing priorities in a fast-paced environment;
  • Comfort with the Linux terminal and unix tools;
  • Quick Learner, able to grasp and prototype new tools and ideas quickly;
  • Proficiency with modern programming languages (Python, Java), datastores (Postgres, Redshift) and SQL;
  • Strong written and verbal technical communication skills;
  • Creative thinking, able to come up with alternative solutions to fit business requirements.