Performance Engineer

Project Role : Performance Engineer Project Role Description : Diagnose issues that an in-house performance testing team has been unable to. There are five aspects to Performance Engineering: software development lifecycle and architecture, performance testing and validation, capacity planning, application performance management and problem detection and resolution. Must have skills : Data Engineering Good to have skills : Python (Programming Language), GitHub Minimum 3 year(s) of experience is required Educational Qualification : 15 years full time education

Summary: As a Performance Engineer, a typical day involves investigating and resolving complex performance issues that have not been identified by the internal performance testing team. The role encompasses a broad spectrum of activities including analyzing software development processes and architecture, validating performance through rigorous testing, planning for capacity needs, managing application performance, and swiftly detecting and addressing problems to ensure optimal system functionality. This position requires a proactive approach to identifying bottlenecks and collaborating with various teams to enhance overall system efficiency and reliability.

KEY RESPONSIBILITIES

Set up and configure Databricks workspaces, including cluster management, access controls, and integration with cloud identity and access management services Design and implement Medallion architecture (Bronze, Silver, and Gold layers) on cloud object storage using Delta Lake format, ensuring data quality and traceability at each layer Build and maintain ETL and ELT data pipelines using PySpark and Spark SQL within Databricks, covering ingestion, transformation, deduplication, normalisation, and standardisation Ingest data from multiple source systems including relational databases (Oracle, MySQL, SQL Server), flat files, and SaaS platforms such as Salesforce, into the Lakehouse Execute data migration from Oracle and other legacy systems, storing transformed outputs in the cloud storage Gold Layer using Databricks batch processing Orchestrate and automate data workflows using pipeline orchestration tools such as Apache Airflow, AWS Glue, or Azure Data Factory, ensuring timely, dependency-aware, and reliable data delivery Create and manage Linked Services, Datasets, and connection configurations for source and sink systems Monitor pipeline execution end-to-end, troubleshoot failures, implement corrective actions, and perform root cause analysis for recurring issues Optimise Spark jobs for performance, scalability, and cost efficiency — including partitioning strategy, caching, and resource configuration Ensure all data is stored in Delta format with appropriate schema enforcement, ACID compliance, and incremental load patterns Manage sensitive credentials and connection strings securely using a secrets management service (e.g., AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault) Maintain data quality standards by implementing validation and reconciliation logic within pipeline workflows Collaborate with data analysts, data scientists, and product teams across global delivery environments to understand data needs and deliver trusted datasets Maintain thorough documentation of pipeline designs, data flows, architecture decisions, and operational runbooks

Required Skills & Experience 3–4 years of professional experience in data engineering, with a significant and demonstrable portion of that time working on Databricks in a production environment Strong hands-on proficiency in PySpark and Python, with the ability to write clean, efficient, and maintainable transformation code Proven experience designing and implementing Medallion architecture (Bronze / Silver / Gold) on cloud object storage, with Delta Lake as the storage layer Solid understanding of Delta Lake capabilities: ACID transactions, schema evolution, time travel, and incremental ingestion patterns Hands-on experience migrating and extracting data from Oracle and other relational databases (MySQL, SQL Server) into a cloud-based Lakehouse Experience with batch data ingestion from SaaS platforms such as Salesforce Working knowledge of at least one pipeline orchestration tool — Apache Airflow, AWS Glue, Azure Data Factory, or similar — for scheduling and dependency management Familiarity with cloud-hosted relational databases and MySQL for structured data storage and maintenance Sound understanding of ETL and ELT design patterns and the trade-offs between them Proficiency in Git and GitHub for version control, branching, and collaborative code review Experience working within Agile or Waterfall delivery frameworks with structured sprint or milestone-based planning Strong analytical and troubleshooting skills ability to diagnose and resolve pipeline failures methodically Effective communication and collaboration skills with the ability to work across cross-functional and geographically distributed teams

GOOD TO HAVE

Experience with Databricks Unity Catalog for data governance, lineage tracking, and fine-grained access control Exposure to cloud secrets management integration (e.g., AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault) within Databricks secret scopes Background in legacy big data tooling — Hadoop, Hive, or Sqoop — particularly in the context of migration or modernisation projects Familiarity with advanced Spark optimisation techniques including broadcast joins, adaptive query execution, and partition tuning Basic understanding of data modelling concepts and dimensional or star schema design Exposure to DevOps practices for data pipelines, including CI/CD, automated testing frameworks, or infrastructure-as-code Databricks Certified Associate Developer for Apache Spark, or a relevant cloud data certification (AWS Certified Data Analytics, Microsoft Certified: Azure Data Engineer Associate, or equivalent)

EDUCATION

Bachelor s degree (B.Tech / B.E. or equivalent) in Computer Science, Information Technology, Data Science, or a related technical discipline is strongly preferred A Master s degree in Computer Science, Data Engineering, or a related field is an advantage but not mandatory Candidates with degrees in other engineering disciplines who can demonstrate strong, production-grade data engineering experience will also be considered Databricks Certified Associate Developer for Apache Spark, or a relevant cloud data certification (AWS Certified Data Analytics, Microsoft Certified: Azure Data Engineer Associate, or equivalent) are a distinct advantage

Additional Information:

• The candidate should have minimum 3 years of experience in Data Engineering.
• This position is based at our Mumbai office.
• A 15 years full time education is required.

15 years full time education

About Accenture Accenture is a leading global professional services company that helps the world’s leading businesses, governments and other organizations build their digital core, optimize their operations, accelerate revenue growth and enhance citizen services—creating tangible value at speed and scale. We are a talent- and innovation-led company with approximately 791,000 people serving clients in more than 120 countries. Technology is at the core of change today, and we are one of the world’s leaders in helping drive that change, with strong ecosystem relationships. We combine our strength in technology and leadership in cloud, data and AI with unmatched industry experience, functional expertise and global delivery capability. Our broad range of services, solutions and assets across Strategy & Consulting, Technology, Operations, Industry X and Song, together with our culture of shared success and commitment to creating 360° value, enable us to help our clients reinvent and build trusted, lasting relationships. We measure our success by the 360° value we create for our clients, each other, our shareholders, partners and communities. Visit us at   www.accenture.com  

Equal Employment Opportunity Statement

We believe that no one should be discriminated against because of their differences. All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, military veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by applicable law. Our rich diversity makes us more innovative, more competitive, and more creative, which helps us better serve our clients and our communities.