Job Opportunity:
We're seeking an accomplished Data Engineer to join our dynamic international team in Porto, Portugal.
This role offers the opportunity to work on high-volume data processing pipelines using cutting-edge technologies such as Apache Spark and Scala in a Unix environment.
Key Responsibilities:
* Develop, maintain, and optimize big data processing jobs using Spark and Scala.
* Write efficient and scalable SQL queries for data extraction, transformation, and analysis.
* Collaborate with cross-functional teams to deliver reliable and performant data solutions.
* Ensure robustness of data pipelines in a Unix-based infrastructure.
* Write and maintain Shell scripts to support automation and data operations.
Requirements:
To succeed in this position, you'll need:
* Proven experience with Apache Spark and Scala in production environments.
* Solid background in SQL for Big Data (large-scale datasets, complex queries, performance tuning).
* Familiarity with Shell scripting and Unix operating systems.
* Ability to work collaboratively in a distributed international team.
By joining our team, you'll have the opportunity to drive business success through innovative data-driven solutions.
At Smart Consulting, we foster a culture of transformation, adaptability and innovation. We're passionate about harnessing technology to drive success.
Our diverse team is dedicated to delivering exceptional results. If you're driven by a passion for technology and data, we'd love to hear from you.