Platform Data Engineer

We are looking for a versatile and pragmatic Data Engineer to join our team. This is a hands-on role suited to someone who thrives across the full data stack, from ingestion and transformation through to serving reliable, well-structured data to consumers. You will be comfortable working across a range of technologies and environments, bringing both engineering rigour and a data-first mindset to everything you build.

You will work closely with data architects, analysts, and product teams to design and deliver scalable data pipelines, implement robust storage patterns, and help shape our data platform strategy, primarily on Azure, with exposure to wider cloud environments.

Key Technologies

·       Python

·       Azure

·       Databricks

·       Apache Spark

·       Delta Tables

·       Parquet

·       Unity Catalogue

Desirable

·       C#

·       DuckDB

·       AWS

Key Responsibilities

·       Design, build, and maintain scalable data pipelines using Apache Spark and Databricks, with a focus on reliability and performance.

·       Work with structured and semi-structured data in Parquet and Delta Table formats, applying appropriate partitioning and optimisation strategies.

·       Manage and evolve data assets within the Unity Catalogue, maintaining clear governance, lineage, and access controls.

·       Collaborate with architects and engineers to define and uphold data modelling and ingestion best practices.

·       Write clean, well-tested Python code; this is a core requirement of the role.

·       Troubleshoot and resolve data quality, pipeline, and performance issues across the data platform.

·       Contribute to infrastructure-as-code, CI/CD, and DevOps practices within an Azure-first environment.

·       Support the wider engineering team as needed; this role requires genuine flexibility across the stack.

Required Skills & Experience

Data Engineering

·       Solid, demonstrable experience in data engineering: pipeline design, data modelling, and working at scale.

·       Strong understanding of columnar storage formats, particularly Parquet, and experience with Delta Lake / Delta Tables.

·       Hands-on experience with Apache Spark, including DataFrame APIs, optimisation, and debugging.

·       Experience with Databricks, including notebook development, job orchestration, and cluster management.

·       Familiarity with Unity Catalogue or similar data governance and cataloguing tooling.

Programming

·       Highly proficient in Python (non-negotiable). You should be comfortable with idiomatic Python, testing, packaging, and working in collaborative codebases.

·       Comfortable reading and working across multiple technology stacks; you do not need to be an expert in everything, but you should be adaptable.

Cloud & Infrastructure

·       Practical Azure experience, ideally including Azure Data Factory, Azure Data Lake Storage, Synapse Analytics, or similar services.

·       Understanding of cloud-native design principles: scalability, cost-awareness, and security.

Engineering Practices

·       Experience working in Agile teams with version control (Git), code review, and CI/CD pipelines.

·       Good written and verbal communication, with the ability to articulate technical decisions to both technical and non-technical stakeholders.

Nice to Have

·       Experience with C# or .NET, useful for integrations with existing backend services.

·       Familiarity with DuckDB for lightweight, local analytical workloads or prototyping.

·       AWS experience, useful for cross-cloud projects and broadening platform perspective.

·       Experience with streaming data platforms such as Apache Kafka or Azure Event Hubs.

·       Exposure to dbt, Great Expectations, or similar data transformation and quality tooling.

What We Offer

·       A collaborative, engineering-led culture with high ownership and autonomy.

·       Opportunity to help shape the data platform from the ground up.

·       Competitive salary and benefits package.