Python Software Developer

Job Description: Python / PySpark Developer

Experience: 710 Years
Locations: Pune, Chennai

Job Overview

We are looking for an experienced Python / PySpark Developer with strong expertise in Python programming, distributed data processing, and SQL. The ideal candidate should have hands-on experience developing scalable data processing solutions, building ETL pipelines, and handling large volumes of structured and semi-structured data.

Key Responsibilities

• Design, develop, and maintain data processing applications using Python and PySpark.
• Build and optimize ETL/ELT pipelines for large-scale data transformation and integration.
• Write complex SQL queries, joins, subqueries, window functions, and performance-optimized queries.
• Work with Apache Spark components, including DataFrames, Spark SQL, transformations, and actions.
• Perform data cleansing, validation, aggregation, and transformation.
• Troubleshoot data pipeline failures and resolve performance and data quality issues.
• Optimize Spark jobs by managing partitions, shuffles, caching, and execution plans.
• Integrate data from multiple sources, including relational databases, files, and cloud storage.
• Collaborate with data engineers, analysts, architects, and business stakeholders.
• Follow coding standards, testing practices, documentation, and deployment procedures.

Mandatory Skills

• Strong Python programming experience, including OOP concepts, functions, exception handling, and data structures.
• Hands-on experience with PySpark and Apache Spark.
• Strong SQL skills, including complex queries, joins, aggregations, and window functions.
• Experience developing and maintaining ETL/ELT data pipelines.
• Understanding of distributed computing and large-scale data processing.
• Experience with debugging, performance tuning, and data validation.

Good-to-Have Skills

• Experience with AWS, Azure, or GCP.
• Knowledge of cloud data services such as AWS S3, Glue, or EMR.
• Experience with Databricks and Delta Lake.
• Familiarity with Airflow or other workflow orchestration tools.
• Knowledge of Git, CI/CD, and automated testing.

Required Experience

• 7–10 years of overall IT experience, with relevant hands-on experience in Python, PySpark, and SQL.
• Experience working with large datasets and production data pipelines.
• Strong analytical, problem-solving, and communication skills.

Preferred Candidate Profile

Candidates should be proficient in Python, PySpark, and SQL, with the ability to develop reliable, scalable, and high-performance data processing solutions in an enterprise environment.