Job Title
Aubay Portugal is a multinational company with over 150 active partners operating in various sectors, including banking, insurance, telecommunications, services, energy, and transports.
This role involves designing, developing, and maintaining scalable and robust data pipelines to support migration from legacy platforms to modern data hubs.
1. Create data pipelines that are efficient and easy to maintain.
2. Transform large volumes of information using complex business rules.
3. Automate ingestion and orchestration processes.
4. Implement solutions using Spark (Scala), COS/S3 storage, and Parquet format.
5. Set up CI/CD pipelines, unit testing, and validation processes.
6. Prepare technical documentation.
7. Provide technical support and troubleshooting.
8. Mentor junior team members.
Requirements:
* Minimum 4 years of experience as a Data Engineer.
* Bachelor's degree in Computer Science or equivalent.
* Strong knowledge of Spark with Scala, Apache Airflow, Kafka/Event Streaming, and S3/COS.
* Solid experience with SQL and NoSQL databases.
* Familiarity with CI/CD tools.
* French and English fluency.
* Residency in Portugal — mandatory.