Job Information
Date Opened
18/08/2025
Job Type
Full time
Remote Job
Industry
IT Services
Job Description
This is a remote position.
We are looking for skilled Data Engineers to join our team. As a Data Engineer, you will be responsible for understanding business and technological challenges, develop data pipelines tackling those challenges, and ensuring their smooth deployment. You will as well be responsible of the applications of standard industry and within-the-company good practices, and the application and evolution of our various patterns.
Requirements
Key Responsibilities
Data Pipeline Development
* Design, build, and maintain scalable batch pipelines in Databricks using PySpark.
* Develop and support real-time data streams.
* Implement robust data transformations following the Medallion Architecture (Bronze, Silver, Gold).
* Optimize and troubleshoot performance of large-scale data jobs.
Platform & Infrastructure
* Manage and deploy infrastructure using Terraform and GitOps principles.
* Work within our Airflow orchestrator running on Azure Kubernetes Service (AKS).
* Apply best practices for CI/CD, monitoring, and error handling in production.
Collaboration & Communication
* Work closely with Data Architects to understand and implement proposed architectures.
* Coordinate with Project Managers and stakeholders to align on requirements and milestones.
* Participate in code reviews and share knowledge across the engineering team.
Testing & Documentation
* Define and perform tests to ensure data quality and pipeline reliability.
* Document technical specifications, workflows, and deployment processes.
Requirements
* Proven experience with Databricks and PySpark.
* Strong understanding of batch and streaming data processing.
* Experience with Azure Data Lake Storage Gen2 (ADLS).
* Solid experience with Airflow, preferably deployed in Kubernetes (AKS).
* Understanding of Medallion Architecture.
* Familiarity with data privacy, governance, and security standards.
* Working knowledge of Terraform and infrastructure-as-code.
If it sounds like you, share your CV with us and let's talk