Lead Data Engineer (London)

The AA has a purpose to Create confidence for drivers now and for the future.

The data team are helping to deliver this my making data our superpower, sitting on wealth of motoring & insurance data an opportunity has arisen for you to join us and lead our data engineering team to ensure we are able to unlock this power. 

As a Lead Data Engineer, you will provide technical guidance and leadership to our talented team of Data engineers working on our data pipelines and data lake house architecture. Performing this role, you will be central to the success of the Data teams drive to maximize our data power.

We are amid an exciting time for the data engineering team, so you will get the opportunity to define our data engineering approach and best practices, helping to make fundamental changes by implementing a modern data ecosystem that underpins our mission.

You will work closely with the broader business, supporting, challenging, and championing data and analytics for better decision-making.

Responsibilities
Guiding the team to consistently deliver high-quality technical outputs
Growing your team in terms of skills and personal growth
Showing an example with hands-on developments which are of high quality and tested thoroughly.
Establishing the data Lakehouse as a trusted, stable, and reliable source of insight for the business
Building analytics datasets that utilise our pipelines to provide actionable insights into customer understanding, operational efficiency, and other key business performance metrics.
You will possess strong organisational skills and attention to detail, including the ability to manage multiple tasks autonomously.
You strive for improvement in your work and that of others, proactively identifying issues and opportunities and process improvements.
Provide hands-on support to users reporting data incidents, helping the wider team triage and respond to user queries promptly.
Embedding technical architecture and documentation standards, governance, policies, processes, and procedures

Leadership

You will play a key role in our Leadership team so your technical capability will go hand in hand.
With the below
Operate with a large degree of ownership and a self-reliant mindset, able to decide how and when to involve senior stakeholders.
Demonstrate great leadership and relationship management that delivers results through effective teamwork and communication.
Take ownership of your development, proactively seeking out feedback to build self-awareness
Lead, coach, and develop members of the Data Engineering team, promoting best practices in development, and continually benchmarking against industry standards.

Conduct Rules

In line with SM&CR requirements, the jobholder must abide by the following conduct rules:
First Tier – Individual Conduct Rules
Rule 1.  You must act with integrity
Rule 2.  You must act with due care, skill and diligence
Rule 3.  You must be open and cooperative with the FCA, the PRA and other regulators
Rule 4.  You must pay due regard to the interests of customers and treat them fairly.
Rule 5.  You must observe proper standards of market conduct
Rule 6.  Act to deliver good outcomes for retail customers

Capability, Knowledge, and Experience

Technical Skills

Azure and its assorted services
Advanced SQL skills
Cloud data platforms such as Databricks. Snowflake, RedShift, Azure Synapse
Developing Spark/Databricks pipelines using Python
Event-driven architecture and data streaming
Implementing Data Lakehouse standards and best practice
Data Lakehouse design and modelling skills such as dimensional.
Agile delivery methodologies, Azure DevOps, and CI/CD pipelines.
Data Lakehouse and pipeline development best practices

Collaboration and Relationships

Work with the wider Data team and the business to build a seamless and well-integrated data ecosystem, aligned with The AA’s needs, quality standards and service commitments.
Become a trusted member of the Data team, building relationships across the wider business.
Demonstrate credibility and always have straightforward professional communication, being transparent and open about challenges and successes.
Manage a data engineering team building and supporting modern data engineering solutions.
Proven ability to balance the speed of delivery with the right long term technical solution, ensuring the platform is maintainable, scalable and secure 
Track record of enforcing best practice. Has the ability to combine rigour and discipline with a pragmatic approach to deliver the highest quality, cost effective solutions within set time-frames
Experience managing and leading teams to successful delivery as well as measuring and driving team performance 
Strong financial management experience of opex & capex
Interest and understanding of the external market, innovation and tech trends and how to best use these for the benefit of Data team 

Technical Assurance

Assure that all the technical designs and solutions developed within your Data Engineering team are fit for purpose and support the wider business and data strategy.
Look for continuous improvement of the data and analytics platform performance.
Best practices in data engineering are dear to your heart, but you are not dogmatic.
You have experience with numerous technology stacks and approach.
General

Education & Qualifications
Degree qualified; preferably but not essential.