Custom Software Engineer

Project Role : Custom Software Engineer

Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs.

Must have skills : AWS Glue

Good to have skills : Python (Programming Language), PySpark

Minimum 7.5 Year(s) Of Experience Is Required

Educational Qualification : 15 years full time education

Summary:

As a Custom Software Engineer, you will engage in the development of custom software solutions that are designed to meet specific business needs. Your typical day will involve collaborating with cross-functional teams to design, code, and enhance various components across systems or applications. You will utilize modern frameworks and agile practices to ensure the delivery of scalable and high-performing solutions, while also addressing any challenges that arise during the development process. Your role will require a proactive approach to problem-solving and a commitment to continuous improvement in software development practices.

Roles & Responsibilities:

Build & Maintain Data Pipelines: Develop scalable, reliable ETL/ELT workflows using Python and AWS Glue (Jobs/Workflows), Lambda, and Step Functions where applicable.

Data Lake Engineering: Design and optimize S3-based data lakes (partitioning, lifecycle policies, storage classes) with robust metadata strategies using AWS Glue Data Catalog.

Security & Governance: Implement AWS IAM policies/roles, fine-grained access controls for S3, KMS-based encryption, and audit-ready logging (CloudTrail/CloudWatch).

Automation & CI/CD: Familiarity with Git, code reviews, and automated testing for data code. Package and deploy jobs/functions, maintain unit/integration tests, and set up CI/CD pipelines.

Serverless & Event-Driven: Build event-driven data processing with S3 events, Lambda triggers optimize for cost, concurrency, and resilience.

Data Quality & Observability: Implement validation, schema enforcement (e.g., Glue schemas), logging/monitoring, and alerting for pipelines and jobs.

Collaboration: Work closely with data platform, analytics, and governance teams to ensure reliability, scalability, and compliance.

(If Databricks): Develop notebooks/jobs, Delta Lake optimization (Z-ordering, OPTIMIZE/VACUUM), cluster/pool usage, and job orchestration.

Professional & Technical Skills:

• Must Have Skills: Proficiency in AWS Glue, Python (Programming Language), PySpark.
• Strong understanding of data integration and ETL processes.
• Experience with cloud computing services and architecture.
• Familiarity with agile methodologies and software development life cycle.
  
  

Additional Information:

• The candidate should have minimum 7.5 years of experience in AWS Glue, Python (Programming Language), PySpark.
• This position is based at our Bengaluru office.
• A 15 years full time education is required.