Data Engineer II

We are looking for a Data Engineer to join the Data Engineer team. This is a pivotal role, responsible for supporting multiple Data Analysts and Data Scientists day-to-day and assisting various data-driven requests across the company. The position involves a wide variety of technical skills, from coding to infrastructure, from data analysis to database management. The ideal candidate will have experience working with large-scale data and building automated data ETL processes for large-scale data analysis. ODS or data warehousing experience would be ideal.

Key responsibilities include:

• Build and maintain database technologies, environments, and applications, seeking opportunities for improvements and efficiencies
• Should be familiar and have experience in cloud data platforms like Databricks and/or Cloudera, Iceberg/Delta tables
• Writes SQL statement procedures to optimize SQL execution and query development
• Effectively utilizes various technology like Spark (Scala, Python), Apache Kafka, Trino
• Manages the deployment of data solutions that are optimally standardized and database updates to meet project deliverables
• Leads database security posture, which includes proactively identifying security risks and implementing both risk mitigation plans and control functions
• Supports process improvement efforts to identify and test opportunities for automation and/or reduction in time to deployment
• Participating in the entire solution development cycle from requirement gathering to solution deployment.
• Working with multiple stakeholders across teams to fulfill ad-hoc investigations, including large-scale data extraction, transformation, and analyses.
• Supporting production code to produce comprehensive and accurate data artifacts.
• Promoting strategies to improve our data modeling, quality, and architecture.

All About You:

• Should have experience working with Databricks and/or Cloudera platforms
• Skilled in Spark framework (Scala/Python), Apache Kafka, Spark Streaming
• Highly skilled in SQL knowledge and relational databases.
• Experienced with developing and maintaining data pipelines for the extraction, transformation, and/or loading of large-scale data.
• Experienced with working with large data sets. You've implemented parallel processing and have dealt with the limitations and challenges of working with terabytes of data.
• Strong analytics skills related to working with unstructured datasets.
• Experienced with performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Experienced with database administration and query optimization with different databases.
• Experienced with at least one of the programming languages like Python
• Capable of effectively communicating requirements and solutions to audiences with different skill sets.
• Familiarity with the core requirements of data analysis and/or presentation.
• Focused, innovative, and self-sufficient problem solver who is willing to independently and creatively deep-dive into an issue, cut to the heart of a problem, and solve it.

Bonus Points if:

You have any prior experience with any of the following:

• AWS; S3, EMR, Ozone, Iceberg
• Python, Javascript, PLSQL, regex
• Jenkins, Datadog, Splunk, Dynatrace