Data Engineer II - Databricks and Python

Join our innovative Digital Intelligence team at J.P. Morgan, where we leverage cutting-edge technology to drive data-driven decision-making and enhance business performance. We are seeking a talented and motivated Databricks Data Engineer to join our team and contribute to our mission of transforming data into actionable insights.

As a Data Engineer II at JPMorgan Chase within the Digital Intelligence team, you will play a crucial role in designing, developing, and maintaining scalable data processing solutions using Databricks, Python, and AWS. You will collaborate with cross-functional teams to deliver high-quality data solutions that support our business objectives.

Job Responsibilities:

• Execute software solutions, design, development, and technical troubleshooting with the ability to think beyond routine or conventional approaches to build solutions or break down technical problems.
• Create secure and high-quality production code and maintain algorithms that run synchronously with appropriate systems.
• Produce architecture and design artifacts for complex applications while being accountable for ensuring design constraints are met by software code development.
• Gather, analyze, synthesize, and develop visualizations and reporting from large, diverse data sets in service of continuous improvement of software applications and systems.
• Proactively identify hidden problems and patterns in data and use these insights to drive improvements to coding hygiene and system architecture.
• Contribute to software engineering communities of practice and events that explore new and emerging technologies.
• Provide guidance to the immediate team of software engineers on daily tasks and activities.
• Set the overall guidance and expectations for team output, practices, and collaboration.
• Anticipate dependencies with other teams to deliver products and applications in line with business requirements.
• Manage stakeholder relationships and the team’s work in accordance with compliance standards, service level agreements, and business requirements.

Required Qualifications, Capabilities, and Skills:

• Formal training or certification on software engineering concepts and applied experience.
• Hands-on experience in data mapping, data architecture, and data modeling on Databricks.
• Extensive experience in AWS, design, implementation, and maintenance of data pipelines using Python, PySpark on Databricks.
• Proficient in Python and PySpark, able to write and execute complex queries to perform curation and build views required by end users (single and multi-dimensional).
• Strong understanding of front-end and back-end technologies, with a focus on creating seamless user experiences.
• Extensive experience in Databricks data engineering, data warehousing concepts, ETL processes (Job Runs, Data Ingestion and Delta Live Tables, Spark Streaming).
• Experienced in standing up and maintaining EC2/ECS instances, S3, Glue, and Lambda services.
• Experience in building Notebooks with complex code structures and debugging failed jobs.
• Proven experience in performance and tuning to ensure jobs are running at optimal levels and no performance bottleneck.
• Proven ability to deliver high-quality features into production systems in a rapid-paced, iterative development environment.

Preferred Qualifications, Capabilities, and Skills:

• Experience with machine learning and data science workflows.
• Familiarity with data visualization tools and techniques.
• Knowledge of data governance and security best practices.
• Experience in carrying out data analysis to support business insights.