Senior Data Engineer - #186402

Senior Data Engineer (Azure | Databricks | PySpark)

Permanent | Hybrid | UK Public Sector Project

OVERViEW

We are seeking a hands-on Data Engineer to join a growing Data & Analytics practice. This role focuses on building, optimising, and maintaining modern cloud data platforms.

You will work across the full data lifecycle, delivering scalable, high-performance solutions using Azure and Databricks.

Who We’re Looking For

A strong Data Engineer with deep technical expertise in Azure and Databricks, who enjoys building robust pipelines and working in a collaborative, client-facing environment.

Key Responsibilities

• Develop solutions using Azure Databricks

• Implement Medallion Architecture (Bronze, Silver, Gold)

• Design ETL/ELT pipelines

• Build and maintain scalable data pipelines (batch & real-time)

• Develop data warehouse and lakehouse solutions

• Support BI, analytics, and AI workloads

• Optimise performance, cost, and reliability

• Implement CI/CD pipelines using Azure DevOps

• Collaborate with architects, analysts, and stakeholders

Mandatory Requirements

• Strong hands-on experience with Azure Databricks Data Engineering and Data Warehousing workloads

• Experience with Azure Data Factory

• Strong SQL and data warehousing expertise

• Experience with data modelling (Kimball, Inmon, and Data Vault)

• Experience with Power BI

• Strong programming skills (Python, SQL, scripting)

• Experience with CI/CD and DevOps practices

• Ability to produce technical documentation (HLD, LLD)

Desirable

• Experience in consulting / professional services

• Exposure to AI and advanced analytics solutions

• Exposure to Agentic AI systems built on top of Large Language Models

• Experience with real-time / streaming data

• Experience with Infrastructure as Code (Terraform or Bicep)

• Experience with Microsoft Fabric

• Exposure to the wider Databricks Data Intelligence Platform (e.g. Databricks Apps, Genie Code, Lakebase, Lakebridge, Delta Sharing)

• Experience in regulated industries

Soft Skills

• Strong problem-solving skills

• Collaborative and team-oriented mindset

• Clear communication and documentation skills

• Ability to work in fast-paced delivery environments

Contract Details

• Role Type: Permanent & Hybrid (2 days per week in office)

• Location: Guildford

• Clearance: Candidates must be eligible for UK Security Clearance (SC) or active SC