Azure Developer

We are seeking an experienced Databricks developer with
strong expertise in Azure Databricks, Apache Spark, and modern data engineering
practices. The ideal candidate will be responsible for designing, developing,
and optimising scalable data pipelines, data lakehouse solutions, and
cloud -based data platforms. The role requires hands -on experience with Spark
processing, Delta Lake, Unity Catalogue, data modelling, and enterprise -grade data
integration solutions. <\/span>
<\/p>

<\/div>

Key
Responsibilities <\/span><\/b><\/span>
<\/p>

Data
Engineering & Databricks Development <\/span><\/b>
<\/p>

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks,
  Apache Spark, and SQL <\/b> . <\/span>
  <\/li>
• Build and optimize batch and streaming data pipelines using PySpark,
  Spark Structured Streaming, and Auto Loader <\/b> . <\/span>
  <\/li>
• Develop and support enterprise data lakehouse solutions using Delta
  Lake <\/b> and Databricks technologies. <\/span>
  <\/li>
• Implement data ingestion, transformation, cleansing, and
  aggregation processes for large -scale datasets. <\/span>
  <\/li>
• Develop reusable frameworks and best practices for data engineering
  solutions. <\/span>
  <\/li><\/ul>

  Data
  Modeling & Performance Optimization <\/span><\/b>
  <\/p>

  • Design and implement data models to support reporting, analytics,
    and business requirements. <\/span>
    <\/li>
  • Build and maintain Slowly Changing Dimensions (SCD Type 1 &
    Type 2)<\/b> for data warehousing solutions. <\/span>
    <\/li>
  • Develop and optimize Change Data Capture (CDC)<\/b> pipelines. <\/span>
    <\/li>
  • Optimize Spark workloads through partitioning, clustering, caching,
    and performance tuning techniques. <\/span>
    <\/li>
  • Ensure efficient query performance and scalability across large
    datasets. <\/span>
    <\/li><\/ul>

    Unity
    Catalog & Data Governance <\/span><\/b>
    <\/p>

    • Configure and manage Databricks Unity Catalog <\/b> environments. <\/span>
      <\/li>
    • Create and manage catalogs, schemas, tables, materialized views,
      functions, and volumes. <\/span>
      <\/li>
    • Implement enterprise data governance, security, access control, and
      compliance standards. <\/span>
      <\/li>
    • Support metadata management and data lineage initiatives across the
      data platform. <\/span>
      <\/li><\/ul>

      Cloud &
      Integration <\/span><\/b>
      <\/p>

      • Develop cloud -native data solutions on Microsoft Azure <\/b> and
        related cloud services. <\/span>
        <\/li>
      • Integrate data from multiple internal and external data sources. <\/span>
        <\/li>
      • Implement Lakehouse Federation <\/b> and foreign catalogs to
        access external data platforms. <\/span>
        <\/li>
      • Collaborate with architects and stakeholders to design scalable
        cloud data solutions. <\/span>
        <\/li><\/ul>

        DevOps
        & Operational Excellence <\/span><\/b>
        <\/p>

        • Support CI/CD implementation and automated deployment processes. <\/span>
          <\/li>
        • Participate in code reviews, testing, and release activities. <\/span>
          <\/li>
        • Monitor and troubleshoot data pipeline failures and performance
          issues. <\/span>
          <\/li>
        • Ensure adherence to development standards, security policies, and
          operational best practices. <\/span>
          <\/li><\/ul>

          
          <\/div>

          Required
          Qualifications <\/span><\/b>
          <\/p>

          • Strong hands -on experience with Databricks <\/b> and Apache
            Spark (PySpark and/or Scala)<\/b> . <\/span>
            <\/li>
          • Extensive experience with SQL <\/b> and complex data
            transformation techniques. <\/span>
            <\/li>
          • Experience in ETL/ELT development and enterprise data pipeline
            implementation. <\/span>
            <\/li>
          • Strong experience with Microsoft Azure <\/b> and cloud -based data
            platforms. <\/span>
            <\/li>
          • Hands -on experience with Azure Databricks <\/b> and Delta Lake <\/b> . <\/span>
            <\/li>
          • Experience building batch processing pipelines using Auto Loader <\/b> and real -time pipelines using Spark Structured Streaming <\/b> . <\/span>
            <\/li>
          • Strong understanding of data warehousing concepts and dimensional
            modeling. <\/span>
            <\/li>
          • Experience implementing SCD Type 1<\/b> , SCD Type 2<\/b> , and CDC <\/b> processes. <\/span>
            <\/li>
          • Strong knowledge of Spark performance tuning, partitioning, and
            optimization techniques. <\/span>
            <\/li>
          • Experience with CI/CD pipelines and DevOps practices. <\/span>
            <\/li>
          • Strong analytical, troubleshooting, and problem -solving skills. <\/span>
            <\/li><\/ul>

            
            <\/div><\/span>