N
source · wttj·req · jb_e142c4d72b·listed 4h ago
Databricks Lead Engineer
NTT·London, United Kingdom·Hybrid·Full-time
Sourced listing · wttjNo salary disclosed
compensation · estimating
Estimating…
Sign up to see our estimate based on role, location, and seniority.
source · estimate pending
Summary
the pitchJoin a leading company in the field of data engineering and analytics as a Databricks Lead Engineer. In this role, you will be responsible for client engagement and delivery, data pipeline development, and collaboration with various stakeholders. You will have the opportunity to innovate and improve data engineering practices while contributing to the growth of the practice. The ideal candidate will have extensive experience in data engineering, pipeline development on Databricks, and a strong understanding of data governance and compliance frameworks.
Role
posted by company- Experience leading or mentoring teams of engineers to deliver high-quality scalable data solutions
- Hands-on expertise across the data lifecycle: ingestion, transformation, modelling, governance, and consumption
- Strong problem-solving, analytical, and communication skills
- Proven experience in data engineering and pipeline development on Databricks and cloud-native platforms
- Strong consulting values with ability to collaborate effectively in client-facing environments
- Exposure to AI/ML workloads desirable
- Understanding of data governance, security, and compliance frameworks
- Familiarity with Databricks Workflows and other orchestration tools
- Familiarity with medallion architectures, data lakehouse principles and distributed data processing
- Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow)
- Experience with version control tools (GitHub, Bitbucket) and CI/CD pipelines
- Strong SQL and Python (or equivalent language) skills for data manipulation and automation
- Hands-on experience with cloud platforms (AWS, Azure, GCP)
- Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent
- Knowledge of data modelling methodologies (star schemas, Data Vault, Kimball, Inmon)
- Preferred: BSc/MSc in Computer Science, Data Engineering, or related field
- Databricks certifications (Data Engineer Professional) highly desirable
- Education: University degree required
- Experience: Minimum 5–8 years in data engineering, data warehousing, or data architecture roles, with at least 3+ years working with Databricks
Key responsibilities
- Engagement with clients to understand their data needs and deliver high-performing, scalable, and secure data pipelines.
- Development of data pipelines, both batch and streaming, and architecture of Databricks and Lakehouse solutions.
- Collaboration with cross-functional teams, including Solution Architects, Data Engineers, and client stakeholders, to ensure successful adoption of Databricks-based solutions.