Azure Databricks + Pyspark

Job Summary

Databricks requirements for Skillset we need:
•  6+ years of experience in Azure Datbricks,Pyspark Spark/Scala, SQOOP, Azure Databricks, Data Factory, Python and Kafka. Proficient in coding, testing, implementing, debugging, and documenting the complex Spark Jobs.
• Should have good understanding of Spark Architecture
• Good hands-on experience on setting up Kafka streams and consuming from Kafka streams
• Experience in RDBMS like Oracle, SQL Server, Netezza, DB2 etc.
• Should have good understanding of Data Architecture and Azure Cloud.
• Possess good knowledge in Unix and shell scripting.
• Involved in creating proper technical documentation in the work assignments.
• Understanding of the business needs and designs programs and systems that match the complex business requirements and records all the specifications that are involved in the development and coding process.
• He/she ensures that all the standard requirements have been met and is involved in performing the technical analysis.
• Understanding of QA and testing automation process.
• He/she is responsible for assisting the project manager by compiling information from the current systems, analyzing the program requirements, and ensuring that it meets the specified time requirements.
• He/she resolves moderate problems associated with the designed programs and provides technical guidance on complex programming.
• Understanding and knowledge of Agile would be preferred.