Senior Python / PySpark Data Engineer
Saicon · Not specified · onsite · Posted 42d ago
Not disclosedFresherfulltimeSourced from
About a minute. You'll make an account on the way through.
About this role
We are seeking a Senior Python / PySpark Data Engineer with strong software engineering fundamentals and hands-on experience building scalable data solutions.
Required Skills
- Strong Python development experience with OOP, reusable code, classes, and functions
- Experience building and deploying Python packages (PyPI, wheel files)
- Expertise in Unit Testing and Integration Testing
- Strong CI/CD experience using GitHub Actions
- Solid PySpark knowledge, including:
- Performance tuning and optimization
- Partitioning and data ordering
- JDBC tuning and large-scale data ingestion
- Understanding of security best practices, access controls, and secure data access
Nice to Have
- Experience with enterprise data platforms
- Cloud and modern data engineering experience
What You'll Do
- Develop scalable Python and PySpark applications
- Build reusable frameworks and libraries
- Implement automated testing and CI/CD pipelines
- Optimize large-scale data processing workloads
- Collaborate with engineering teams to deliver secure, high-quality solutions
Sourced from foundit
Skills
secure data accessJDBC tuningData orderingLarge-scale data ingestionPerformance TuningUnit testingOOPoptimizationpartitioningGithub Actionssecurity best practicesAccess ControlsPysparkPythonIntegration TestingCi/Cd