Lead Data Engineer

Job Title:                            Lead Data Engineer

Location:                           Scotland (flexible to either Glasgow, Perth, Aberdeen or Inverness)

Employment Type:         Permanent

THE ROLE

A forward-thinking energy company with a strong commitment to a low-carbon future, is looking for a Lead Data Engineer to join their dynamic team. We are looking for a Lead Data Engineer who will be responsible for the technical management and development of the data platform, overseeing the planning and implementation of technical features and functions. They will also be instrumental in maintaining and expanding the Data Model as the company grows, contributing to the implementation of Data Governance principles.

RESPONSIBILITIES

As a Lead Data Engineer, you will collaborate closely with the Lead Data Architect and Data Management teams on data models, governance, and security. You will also work with business stakeholders and IT teams to deliver technical capabilities, data pipelines, and models efficiently into production.

• Serving as the Subject Matter Expert (SME) and technical owner for the data platform, overseeing its development, including planning and implementing technical features and functions supporting the data product lifecycle and roadmap.
• Leading a predominantly 3rd-party service-based data engineering capability, employing a professional lifecycle approach that encompasses analysis, planning, design, development, testing, deployment, and maintenance.
• Leading the evaluation of business requirements entering the data engineering team to ensure appropriateness and feasibility within any solution engineering.
• Leading the design, engineering and improvement of enterprise-level data solutions.
• Collaborating with other data professionals to optimize models and algorithms for data quality, security, and governance.

KEY REQUIREMENTS

• Excellent technical computing, analysis, design, and development skills to a proven professional level.
• Experience in designing, building, and operating analytics solutions using Azure cloud technologies.
• Demonstrable use of specific Azure data-related resources, including Data Factory, Databricks, Storage Account (Data Lake / Delta Lake), SQL Database, and SQL Data Warehouse (Dedicated SQL Pool).
• The ability to lead and mentor a data engineering team, taking responsibility for designing and building new data models, ETL pipelines, engineering robust solution components, undertaking testing/QA, producing standards documentation, and leading best practices using Azure DevOps.