SENIOR DATA ENGINEER (Lisbon office x1 day a week)
Fully remote in Portugal.
Xcede is delighted to be working with an organization dedicated to using state-of-the-art AI technologies to drive continuous improvements in products and business strategies. Their AI division, comprising teams focused on research, platforms, and product innovation, serves as the backbone of this mission. They deliver AI-powered tools for various purposes, including fraud detection, personalized gaming recommendations, AI-enabled co-pilots, and CRM enhancements.
Our client cultivates teams that are autonomous, collaborative, and self-directed. As a member of these teams, you’ll work alongside Data Scientists, ML Engineers, Data Engineers, Product Analysts, Agile Delivery Leads, and Product Owners to develop impactful AI solutions.
As a Senior Data Engineer, your work will focus on designing and implementing robust data infrastructure to transform raw data into high-quality features used in machine learning models.
Responsibilities:
1. Build and optimize scalable pipelines for processing large datasets in distributed production environments.
2. Create Python-based microservices that deliver processed data and features for production use.
3. Develop and maintain internal systems supporting CI/CD workflows, experimental tracking, and data versioning.
4. Apply data quality measures to ensure accuracy and reliability in data workflows.
Requirements:
* Expertise in developing software with Python in large-scale, high-performance environments.
* Proficiency in designing and managing models for structured and unstructured datasets.
* Experience with Apache Kafka for distributed data streaming applications.
* Familiarity with cloud-based ecosystems and infrastructure solutions.
* Knowledge of distributed system design and architecture.
* Hands-on experience with Apache Spark and PySpark for data processing.
Preferred Qualifications:
* Experience leveraging the Databricks platform for data and analytics tasks.
* Experience with real-time and streaming data processing technologies.
* Familiarity with NoSQL databases like Neo4j, Redis, etc.
#J-18808-Ljbffr