Specialist - Data Engineering

Job SummarySeeking a Specialist with 5 to 7 years of experience in Scala and Apache Spark to design develop and optimize scalable big data processing solutions within the Hadoop Spark SparkSQL ecosystem
Role DescriptionJob role : Spark and scala developer
Location : Mississauga ,Ontario
Job SummarySeeking a Specialist with 5 to 7 years of experience in Scala and Apache Spark to design develop and optimize scalable big data processing solutions within the Hadoop Spark SparkSQL ecosystem
Job DescriptionDesign develop and optimize scalable data processing applications using Scala and Apache Spark Collaborate with crossfunctional teams including data scientists analysts and architects to build and maintain robust data pipelines Analyze large and complex datasets to extract actionable insights supporting business objectives Develop highperformance applications in distributed computing environments leveraging SparkSQL Ensure code quality maintainability and adherence to Scala programming best practices Continuously evaluate and integrate emerging big data technologies and tools to enhance platform capabilities Troubleshoot and resolve complex data processing issues to maintain system reliability Document technical designs and implementation details for effective knowledge sharing
Roles and Responsibilities
Lead the development and deployment of big data solutions leveraging Scala Apache Spark and SparkSQL
Collaborate with stakeholders to gather requirements and deliver effective data solutions
Optimize Spark jobs for performance and scalability in production environments
Mentor junior team members and provide technical guidance on best practices
Conduct code reviews to ensure adherence to coding standards and promote continuous improvement
Monitor system performance proactively and address bottlenecks or failures
Participate in sprint planning estimation and agile ceremonies to deliver highquality software
Ensure compliance with data governance and security policies
#J-18808-Ljbffr