Senior Data Platform Engineer

 

The Role 

We are looking for a Databricks Platform SME to join Version 1’s Data & AI Practice, with strong Databricks expertise and the ability to take primary ownership of our Databricks platform. You will be responsible for Databricks administration, governance, entitlements, performance optimisation, and overall operational management, while contributing to scalable, cloud-native data solutions that enable analytics and AI at scale. 

This is a delivery-focused role for an experienced engineer who enjoys solving complex data challenges, takes ownership of platform outcomes, and applies strong engineering discipline in real-world environments.

What You’ll Be Doing 

• Partnering with business and technology stakeholders to build and enhance reliable, scalable data platforms  
• Taking primary ownership of the Databricks platform, including administration, governance, entitlements, optimisation, and operational management 
• Managing Databricks workspaces, users, groups, access controls, platform configuration, performance, reliability, and cost efficiency 
• Building and maintaining cloud‑native data solutions on AWS  
• Supporting production systems by troubleshooting issues and implementing continuous improvements 
• Ensuring solutions are production‑ready: secure, tested, observable, and cost‑efficient 
• Optimising data performance, reliability, and data quality in live environments 
• Designing, building, and operating batch and streaming data pipelines  
• Developing high‑throughput, low‑latency data processing systems  
• Orchestrating data workflows using Airflow or similar orchestration tools  
• Writing clean, efficient, and well‑tested code following software engineering best practices  
• Collaborating closely with product, analytics, and platform teams  

 

 

Qualifications

 

What You’ll Bring 

• 8+ years’ experience in hands‑on data platform engineering roles 
• Strong Databricks expertise, including platform administration, entitlements/access management, governance, performance optimisation, and operational management 
• Hands-on experience managing Databricks workspaces, clusters, jobs, notebooks, and production Spark workloads 
• Demonstrated ability to manage and optimize Databricks platform operations at scale 
• At least 2 Databricks platform implementations from design to production in the last 4 years. 
• Strong hands‑on coding skills in Python, Java, or similar languages, with strong SQL capability  
• Proven experience building and operating batch and streaming pipelines in production  
• Hands‑on experience using dbt for transformations and Airflow (or equivalent) for orchestration  
• Solid understanding of data modelling, data quality, and production data systems 
• Experience working with relational databases such as Postgres or SQL Server  
• Familiarity with CI/CD pipelines, testing frameworks, and monitoring / observability tooling

Nice to Have 

• Databricks Professional Certification 
• Experience enabling ML, AI, or GenAI workloads through strong data foundations 
• Exposure to modern BI and analytics tooling 
• Experience working in regulated or security‑conscious environments 
• Consulting or client‑facing delivery experience