Big Data Processing Specialist
Job Description:
We are seeking a highly skilled and motivated individual to join our team as a Big Data Processing Specialist. In this role, the successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems using PySpark and Spark SQL.
Key Responsibilities:
* Design and develop scalable and efficient data pipelines using PySpark and Spark SQL.
* Develop and implement complex data models to support business needs.
* Collaborate with cross-functional teams to identify opportunities for process improvement and implement data-driven solutions.
* Ensure data quality and integrity through thorough testing and validation.