Job Description:
We are seeking an experienced Big Data Architect to lead the design, implementation, and optimization of large-scale data platforms.
The ideal candidate is highly technical, autonomous, and capable of building end-to-end solutions while ensuring performance, scalability, and reliability across distributed environments.
Main Tasks and Responsibilities:
Design and implement scalable, resilient, and secure Big Data architectures.
Configure and manage distributed environments (on-premises or cloud) for large-scale data processing.
Develop and maintain data pipelines using Apache Airflow, including complex DAG design, monitoring, and CI/CD best practices.
Build and optimize batch and streaming applications using Apache Spark.
Drive technical decisions related to data storage, processing, governance, and system integration.
Collaborate with engineering, analytics, and business teams to ensure data quality and availability.
Produce clear documentation of architectures, data flows, and technical components.
Technical Skills:
Proven experience as a Big Data Architect or similar senior technical role.
Strong expertise in Apache Spark (PySpark, SQL, performance tuning).
Solid hands-on experience with Apache Airflow (DAGs, operators, sensors).
Deep understanding of distributed systems, clusters, containerization, and orchestration (Kubernetes is a plus).
Experience configuring Linux environments, networking, security, and automation.
Strong knowledge of Big Data technologies such as Hadoop, Hive, Kafka, Delta Lake, etc.
Experience working with cloud platforms (AWS, Azure, or GCP).
Strong analytical and problem-solving skills with a focus on scalable solution design.
Nice to have:
Experience with microservices-based architectures.
Familiarity with CI/CD tools (GitLab, Jenkins, Argo).
Knowledge of relational and NoSQL databases.
Language Skills:
English - Practice
Soft Skills:
Good communication skills
Strong analytical abilities, able to have the right synthesis and all possible details at same time required soft skills
Adaptability skills
Team spirit
Client Focus