Gen AI Data Engineer - Azure AI Foundry

Responsibilities

- Developing and managing data pipelines using Apache Airflow and Azure Data Factory to support data integration and transformation processes
- Implementing data architecture development and database optimization strategies to enhance data infrastructure and performance
- Utilizing Amazon Web Services (AWS) and Microsoft Azure Databricks for scalable data engineering solutions
- Designing and maintaining data warehouses and data lakes to facilitate efficient data storage and retrieval
- Conducting data validation and quality checks to confirm data accuracy and reliability
- Building dimensional models and directed acrylic graphs (DAGs) to optimize data flow and analysis
- Collaborating with teams to develop data strategy and data anonymization techniques for secure data handling
- Performing database administration and security best practices to safeguard data assets
- Leveraging analytical thinking and creativity to address complex data challenges and drive innovation
- Engaging in query optimization and performance tuning to improve database efficiency and responsiveness

What You Must Have

- At least a Bachelor's degree
- At least 4years of experience
- Oral and written proficiency in English required

What Sets You Apart

- Preference for at least one of the following fields of study: Management Information Systems, Computer and Information Science, Systems Engineering, Electrical Engineering, Chemical Engineering, Industrial Engineering, Mathematics, Statistics, Mathematical Statistics
- Demonstrating proficiency in Amazon Web Services (AWS) and Microsoft Azure Databricks
- Utilizing Apache Hadoop and Apache Airflow for data processing
- Developing data architecture and data modeling strategies
- Implementing data anonymization and data security best practices
- Excelling in data pipeline and data integration techniques
- Navigating complex data engineering platforms and tools