Position Overview
My client is hiring a Senior DataOps Engineer to design, automate, and operate reliable, secure data pipelines across hybrid environments. You'll apply DevOps principles to data engineering, ensuring high‑quality datasets power AI, BI, and operational use cases.
Key Responsibilities - Build & manage CI/CD pipelines for data workloads
- Implement automated testing, schema validation & data contracts
- Enforce metadata, lineage & governance policies
- Monitor pipeline health, performance & cost (dashboards, alerts)
- Support incident response & SLA management
- Collaborate with Data, BI & AI teams to deliver production‑ready data
- Apply RBAC, masking, tokenisation for sensitive data (PII/PHI)
- Maintain lineage, cataloging & quality KPIs
- Optimize performance (partitioning, caching, cost efficiency)
Tech Stack - Databricks
- Spark
- Delta Lake
- ADF
- Airflow ...