Data Engineer

Join us as a Data Engineer and help build the data platform that will underpin a new standalone payments business.

 

As one of the first permanent engineers in the team, you will play a hands-on role in both shaping the platform and delivering the initial data products that run on it. Working with Databricks and AWS, you will design and build production-grade data pipelines, establish engineering standards and create the reusable patterns, automation and controls that will form the foundation of the platform.

 

This is not a role where platform engineering is separated from delivery. We believe the best platform capabilities emerge from solving real business problems. You will work closely with domain teams, building early data products, identifying common requirements and turning successful approaches into repeatable golden paths that make it easier for future teams to onboard and succeed.

 

As the platform evolves, your focus will increasingly shift from hands-on support towards self-service capabilities, reusable tooling and platform features that enable domain teams to build and operate trusted data products independently.

 

To be successful as a Data Engineer, you must have the following essential skills;

 

Production Databricks and data pipeline engineering

• Hands-on experience building production-grade Databricks platforms and reliable batch and streaming pipelines using Python, PySpark and SQL.

Software engineering, testing and automated delivery

• The ability to create modular, maintainable and reusable components, supported by automated pipeline testing, Git-based workflows and CI/CD deployment.

Secure and operable platform engineering

• Experience engineering systems for sensitive or regulated data, including identity and access control, encryption, secrets management, least privilege, monitoring and production incident resolution.

 

Some other highly valued skills may include;

Advanced Unity Catalog expertise

• Practical experience configuring lineage, permissions, metadata and governance through Unity Catalog.

Infrastructure-as-code and Databricks deployment automation

• Experience with Terraform, Databricks Asset Bundles and GitHub Actions or equivalent policy-controlled delivery tooling.

Platform product and self-service engineering

• Experience creating paved roads, golden paths, metadata-driven automation or internal developer platforms that enable domain teams to build compliant data products with less manual support.

 

You may be assessed on the key critical skills relevant for success in role, such as risk and controls, change and transformation, business acumen strategic thinking and digital and technology, as well as job-specific technical skills

 

The successful candidate will be based in London. Our offices are located at 1 Churchill Place and 7 Westferry Circus (new BPL office). We support a hybrid working pattern with 3 days per week office-based presence expected.