Data Engineer, ShipTech Analytics

Job description:

Join ShipTech Analytics (STA) as a Data Engineer to build the data and analytics backbone powering Amazon's global transportation network. You'll contribute to our vision of building an intelligent, autonomous, and scalable performance management and decision support system across Amazon Transportation.

As a Data Engineer II, you'll work on building standardized data solutions that empower operations teams worldwide. You'll develop both near-real-time (NRT) streaming pipelines and batch processing pipelines that deliver critical insights for decision-making. You'll also build data tools that enable self-service analytics and automate operational tasks, while contributing to innovative AI-powered solutions delivering automated insights across the transportation network.

Key job responsibilities:

• Design and build scalable near-real-time (NRT) streaming and batch data pipelines supporting Amazon's global transportation network.
• Build data products and data tools that streamline the complete data lifecycle including data ingestion, processing, backfill, storage, vending, and operational support modules.
• Partner closely with stakeholders across operations and analytics teams to understand requirements, design solutions, and deliver datasets and data tools that enable faster, more informed decision-making.
• Own data quality and implement enhancements for datasets that enable operational excellence and improve customer experience.
• Collaborate with cross-functional teams to standardize analytics capabilities and build AI-powered automation.
• Leverage AWS cloud technologies to transform raw data into actionable metrics.
• Drive continuous improvement through code reviews, design discussions, operational best practices, and security compliance.

A day in the life:

You work backwards from what stakeholders need, clarifying ambiguities and building data products and tools. Throughout the day, you're writing code for data pipelines and building tools that handle ingestion, processing, storage, and vending alongside software engineers.

Some days you're in design discussions about data infrastructure, pipelines, and tools.

Beyond coding, you're documenting datasets, tools, and runbooks, participating in code reviews, and mentoring teammates. When on-call, you monitor pipeline health.

What makes the role interesting is the variety - you're partnering with stakeholders, TPMs, and SDEs building scalable data pipelines and tools that directly affect how Amazon's transportation network operates.

About the team:

ShipTech Analytics serves as the data and analytics backbone for Amazon's global transportation network. The team builds standardized data solutions that streamline analytics and enable faster decision-making for operations teams worldwide. Team members collaborate with cross-functional partners to deliver data products and tools.

The team is modernizing systems, building near-real-time architectures, and building AI-powered automation. Team members have opportunities to mentor others, prototype new ideas, and shape the future of data analytics. The team values collaboration across geographies with members in multiple locations working together to serve global operations.

Basic Qualifications:

• 3+ years of data engineering experience.
• Experience with data modeling, warehousing, and building ETL pipelines.
• Experience with SQL.
• Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS.
• Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets.
• Experience with big data technologies such as Hadoop, Hive, Spark, and EMR.

Preferred Qualifications:

• Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, Firehose, Lambda, and IAM roles and permissions.
• Experience with non-relational databases/data stores (object storage, document or key-value stores, graph databases, column-family databases).
• Knowledge of batch and streaming data architectures like Kafka, Kinesis, Flink, Storm, and Beam.