Lead Python Engineer

Role: Lead Python Engineer

Location: India, Remote

Experience: 7+ Years

Algoworks

www.algoworks.com

About the company

Algoworks is an award-winning artificial intelligence, engineering services and experience transformation firm with offices across the United States, Europe, South America and India. We bring together a global team of engineers, architects, designers, researchers and operators united by rigor, accountability and a commitment to delivering measurable results.

For over 20 years, Algoworks has partnered with Fortune 500 organizations across the Americas, Europe and Asia to define, build and run technology that drives meaningful business outcomes. Our work combines human-centered design, engineering excellence and AI-powered capabilities to solve complex challenges with clarity and precision. Innovation, particularly in the responsible application of AI, is embedded in how teams approach problem-solving and continuous improvement.

At Algoworks, growth is continuous and closely tied to impact. Teams collaborate across geographies and disciplines, strengthening outcomes through shared insight and collective expertise. The culture values transparency, open dialogue and an environment where every voice is heard and contribution is recognized.

Through collaboration, accountability and a focus on results, Algoworks operates at the intersection of technology and people, building not only advanced systems but strong global teams that elevate performance and create lasting impact.

Follow the video below to know about us! Clipchamp

Role overview

We are seeking a highly skilled Data Engineer with a strong background in Databricks and a passion for data engineering. The ideal candidate will have extensive experience in designing and implementing data solutions using various technologies, including PySpark, Spark SQL and Power BI. This role requires a proactive individual who can work independently and collaboratively in a fast-paced environment.

Key responsibilities:

• Design, develop and optimize scalable data pipelines using Python, PySpark, Spark SQL and Databricks.
• Build and manage data solutions using Azure Databricks, Delta Lake, Unity Catalog and Databricks Workflows.
• Develop complex SQL queries, views, data models and semantic layers for data warehousing and visualization.
• Integrate and manage data using Azure Data Lake, Azure Data Factory and other Azure services.
• Develop and maintain Power BI solutions, including dashboards, DAX measures and Power Query transformations.
• Implement data governance, security, compliance and data management best practices.
• Participate in code reviews, CI/CD processes, Agile ceremonies and end-to-end SDLC activities.
• Collaborate with cross-functional teams to understand business requirements and translate them into effective technical solutions.

Required skills and qualifications:

• 7+ years of hands-on experience in software development, with a strong emphasis on Python programming.
• Deep expertise in PySpark (Spark SQL, DataFrames and Datasets) for building and optimizing scalable data pipelines. (PySpark)
• Solid experience with Azure Databricks (Unity Catalog), including working with Delta tables, workflows and scripting in Python.
• Understanding of data governance, data management, security and compliance in cloud environments.
• Familiar with Azure cloud services, especially Azure Data Lake, Azure Data Factory,
• Experience with Power BI, including DAX and Power Query.
• Modeling data for Data Warehousing and Visualization projects through Semantic Layers.
• Skilled in writing complex SQL queries and building views involving joins across multiple tables.
• Proficient in tools like DevOps (project management) and version control and CI/CD pipelines.

Must have skills:

• Python
• PySpark and Spark SQL
• Azure Databricks and Unity Catalog
• Delta Lake / Delta Tables
• Azure Data Lake and Azure Data Factory
• Advanced SQL
• Power BI, DAX and Power Query
• Data Modeling and Semantic Layers
• Azure DevOps, Git and CI/CD

Good to have skills:

• Experience Azure AI, Azure Copilot Studio and Azure Fabric.
• Experience building workflows with Power Automate.
• EventHub or Kafka for streaming data ingestion.
• Proficient in designing and implementing real-time data streaming solutions.
• Experience across the Software Development Lifecycle (SDLC), with a strong grasp of Agile Scrum methodologies.

Desired attributes:

• Strong ownership and a proactive approach to problem-solving.
• Excellent collaboration and communication skills.
• Comfortable participating in mobbing sessions, code reviews and cross-team discussions.
• Ability to translate business requirements into practical technical solutions.
• Adaptable and willing to learn new technologies and tools.
• Strong attention to quality, scalability, security and performance.
• Ability to prioritize effectively and work independently in a fast-paced environment.

Interview process

2 rounds of discussion.

Skills

PythonDatabricks