Job Title: DevOps Engineer
We are seeking a highly skilled DevOps Engineer to join our IT team.
* Key Responsibilities:
* Design and maintain Azure data infrastructure using IaC tools to ensure scalability and reliability.
* Administer and optimize Databricks workspaces, clusters, jobs, and overall platform performance to meet business requirements.
* Automate deployments and CI/CD pipelines for reliable data pipeline releases, minimizing downtime and ensuring consistency.
* Develop automated testing for data quality, pipeline validation, and infrastructure checks to identify and resolve issues promptly.
* Implement monitoring and alerting with Azure Monitor, Log Analytics, and Databricks tools to provide real-time insights and proactive support.
* Optimize cloud costs, cluster performance, and large-scale Spark workloads to improve efficiency and reduce expenses.
* Ensure strong security practices including IAM, encryption, network security, and secret management to protect sensitive data and applications.
About the Role:
This is an exciting opportunity for a motivated and detail-oriented individual to collaborate with IT, security, and business teams to provide platform guidance and troubleshooting support.