Senior Data Engineer

Responsible for developing, testing, and implementing data engineering solutions to generate analytical and reporting solutions. Responsible for analyzing and preparing the data needed for data science-based outcomes. Also responsible for managing and maintaining metadata data structures besides providing necessary support for post-deployment related activities when needed. Design and build automations and incorporate low code-no code solutions to accelerate development cycle time.

Roles and Responsibilities:

• Design and build automations and incorporate low code-no code solutions to accelerate development cycle time.
• Leverage technical data dictionaries and business glossaries to analyze the datasets.
• Perform data profiling and data analysis for source systems and target data repositories.
• Understand metadata and the underlying data structures needed to standardize the data load processes.
• Develop data mapping specifications based on the results of data analysis and functional requirements.
• Perform a variety of data loads and data transformations using multiple tools and technologies.
• Build automated Extract, Transform and Load (ETL) jobs based on data mapping specifications.
• Validate the data mapping results and match them with the expected results.
• Leverage and implement Data Quality (DQ) rules provided.

Education Qualification:

Bachelor's degree in computer science or equivalent with a minimum of 5+ years of hands-on experience.

Desired Characteristics Technical Expertise:

• Ability to understand logical and physical data models, big data storage architecture, data modeling methodologies, metadata management, master data management, and data lineage techniques.
• Hands-on experience in AWS services, especially Redshift, Glue, RDS, Step Functions, S3, SageMaker, and Kinesis.
• Hands-on experience in programming languages like Python and Unix.
• Hands-on experience in writing SQL scripts for Oracle, MySQL, and PostgreSQL.
• Experience in handling both Online Transaction Processing (OLTP) and Online Analytical Processing (OLAP) data models.
• Experience with data warehousing, big data, near real-time (NRT), and Kafka.
• Exposure to unstructured datasets and ability to handle XML and JSON file formats.
• Exposure to Extract, Transform and Load (ETL) tools like Matillion or Informatica.
• Exposure or experience in Databricks.
• Experience using low code, no code tools like Microsoft/GitHub Copilot.

Domain Expertise:

• Exposure to handling enterprise ERP and CRM data.
• Knowledge of industrial applications in commercial, finance, or manufacturing settings.
• Exposure to finance and accounting data domains.

Leadership skills:

• Partner with other team members to understand the project objectives and resolve technical issues.
• Communicate project status or challenges in a clear and concise manner to cross-team members.
• Ability to drive technical solutions to meet business outcomes.