Position Overview
Job Description
- Design and build scalable, cloud-native data platforms from greenfield to production
- Implement near-real-time ingestion pipelines using event-driven patterns
- Define and enforce platform standards, including Data Lake / Lakehouse principles, medallion architecture, and data contracts
- Refactor and optimise existing Spark and PySpark scripts for performance and maintainability
- Introduce best practices for code quality, testing, and CI/CD across data pipelines
- Drive adoption of AI tooling and agentic workflows within the data engineering team
- Ensure data quality, observability, and reliability across all pipelines and platforms
- Develop self-service tooling and microservices to simplify platform usage for other teams
Qualifications
- 5+ years of professional experience in Data Engineering
- Strong Python and SQL development skills for pipeline development and optimisati...