Senior Software Engineer – Big Data Platform

Senior Software Engineer – Big Data Platform Job Description Template

Our company is looking for a Senior Software Engineer – Big Data Platform to join our team.

Responsibilities:

  • Research, development and delivery of Data Engineering’s reliability model and framework;
  • Offer Project delivery support, monitoring and reporting as required;
  • Interact with project owners, data engineering team leads and related internal customers to collect and document technical requirements;
  • Support the development and maintenance of data engineering guidelines, policies, standards and process narratives for in-scope business functions;
  • Design and implement the data infrastructure platforms including the data ingestion, data consumption and stream processing on Google cloud;
  • Contribute to the design, development and delivery of the team’s Data Enablement Program roadmap;
  • Oversee the collection and analysis of key metrics and reporting dashboards to monitor enterprise data platform performance and reliability;
  • Oversee at least one component of the enterprise data ingestion pipeline / data API within the organization’s Big Data platform;
  • Drive and take responsibility for the quality of the development and delivery, also mentor and train team members as needed.

Requirements:

  • Solid understanding and experience with any of the relational databases (MySQL, Postgres, SQLServer etc);
  • Hands-on experience in data engineering with a track record of manipulating, processing, and extracting value from large datasets in production;
  • 3+ years of programming experience in core Java/C# with solid understanding of the Software Development Life Cycle;
  • Experience with Apache Kafka, Apache Airflow, Google Cloud Platform and Google BigQuery;
  • Experience with CI/CD and Behavior Driven Development is nice to have;
  • Bachelor’s degree in Computer Science, Software Engineering, Computer Engineering, or other related degree;
  • Experience implementing and using streaming platforms such as SparkSQL, Flink, Storm etc;
  • Deep understanding of large scale distributed systems, design patterns, and object-oriented design principles with good component and modeling ability;
  • Experience with Linux, Docker and Kubernetes is nice to have;
  • 3+ years of production experience with large scale distributed system such as Hadoop and Kafka.