Save time and effort sourcing top tech talent
Sign up for the chance to get matched to this role, and similar opportunities.
Position Description
We are seeking a passionate and innovative Data Engineer to join our team. The ideal candidate is a skilled technologist with a proven track record in designing, developing, and optimizing complex data solutions. If you are a passionate data engineer who thrives on challenges and enjoys working with cutting-edge technologies, we encourage you to apply.
Your future duties and responsibilities
• You will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams.
• The ideal candidate is an experienced modern data technologist with special focus on orchestrating various data solutions in cloud.
• You should be well-versed in optimizing data systems and building cloud optimized data solutions from ground up.
• You will support Operational Excellence Squad focused on Standardizing toolsets supporting the Enterprise Analytics platform and would ensure optimal data architecture is formulated and delivered.
• You must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products.
• The right candidate will be excited by the prospect of optimizing or even re-designing our company’s data architecture to support our next generation of products and data initiatives
Required qualifications to be successful in this role
• Bachelor’s in computer science or related field with at least 15+ years of relevant experience is desired.
• Candidate should also have strong hands-on experience using the following software/tools: Strong experience with AWS cloud services: EC2, EMR, EKS, Snowflake, Elastic-Search Experience with stream-processing systems: Storm, Spark-Streaming, Kafka etc. Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
• Experience with big data tools: Hadoop, Spark, Kafka, Kubernates etc.
• Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
• Experience with devops, data pipeline and workflow management tools: Concourse, Terraform, Luigi, Airflow, etc.
• Experience with data management tools: Collibra, Alation, BigID, Alex Solutions etc.
• Experience building and optimizing ‘big data’ data pipelines, architectures, and data sets.
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Strong analytic skills related to working with unstructured datasets.
• Experience in cloud services such as AWS EMR, EC2, EKS, Jupyter notebooks
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Build processes supporting data transformation, data structures, metadata management, dependency, and workload management.
• A successful history of manipulating, processing, and extracting value from large, disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Strong project management and organizational skills. Knowledge of agile methods is a plus.
• Experience leading a team of data engineers and supporting and working with cross-functional teams in a dynamic environment.
Sign up for the chance to get matched to this role, and similar opportunities.