Bigdata Developer with PySpark

The Applications Development Intermediate Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities.

Responsibilities: Core Responsibilities Develop and maintain data pipelines:  Design, develop, and optimize scalable ETL (Extraction, Transformation, Loading) pipelines using PySpark to process large datasets. Coding and software engineering:  Write clean, efficient, well-documented code primarily in Python (PySpark) and Java, often utilizing frameworks like Spring Boot. Collaboration and communication:  Work with cross-functional teams, including senior developers, data engineers, analysts, and business partners, to understand data requirements and ensure seamless integration of solutions. Troubleshooting and optimization:  Debug and resolve data processing issues and performance bottlenecks in Spark applications and other big data technologies. Full lifecycle involvement:  Participate in the entire software development lifecycle (SDLC), from requirements analysis and design to testing, deployment, and operations, often using Agile/Scrum methodologies. Data integrity and quality:  Ensure data quality and integrity throughout the data lifecycle.  Key Technical Skills & Qualifications Programming Languages: Strong understanding of Core Java and Object-Oriented Programming (OOP) concepts. Proficiency in Python, specifically for PySpark development.

Big Data Technologies: Hands-on experience or familiarity with Apache Spark (PySpark), Hadoop, and related ecosystem components like Hive and Sqoop. S hould have knowledge of Linux OS and hands-on experience in writing shell scripts . Knowledge of distributed NoSQL databases (e.g., Elasticsearch, Cassandra, MongoDB) is a plus.

Databases & SQL: Basic knowledge of SQL and relational databases. Experience writing queries to validate and manipulate data.

Cloud Platforms: Familiarity with cloud services such as Amazon Web Services (AWS), Azure, or Google Cloud Platform (GCP).

Tools & Methodologies: Understanding of version control systems (e.g., Git). Experience with development and testing tools (e.g., JIRA, Confluence). Familiarity with DevOps practices and CI/CD pipelines is a bonus. 

Qualifications: 4-8 years of relevant experience in the Financial Service industry Intermediate level experience in Applications Development role Consistently demonstrates clear and concise written and verbal communication Demonstrated problem-solving and decision-making skills Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements

Education: Bachelor’s degree/University degree or equivalent experience

This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.

Job Family Group: Technology

Job Family: Applications Development

Time Type: Full time

Most Relevant Skills Please see the requirements listed above.

Other Relevant Skills For complementary skills, please see above and/or contact the recruiter.

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.   If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi .

View Citi’s EEO Policy Statement and the Know Your Rights poster.