Lead data engineer

Band 8a · Data engineer · 551 job evaluation points

UK GDaD PCF level: Lead data engineer

Most often performed at Civil Service grade G7.

A lead data engineer is responsible for the design and implementation of numerous complex data flows to connect operational systems, data for analytics and business intelligence (BI) systems.
At this role level, you will:
recognise and share opportunities to reuse existing data flows between teams
be responsible for the build of data-streaming systems
co-ordinate teams and set best practice and standards
apply knowledge of systems integration to your work
champion data engineering across government

Contains public sector information from the UK Government Digital and Data Profession Capability Framework (https://understand-digital-data-roles-skills.service.gov.uk/), licensed under the Open Government Licence v3.0. © Crown copyright.

Responsibilities

Lead the design and running of the organisation's data platform and its many data flows.
Set engineering standards for pipelines, testing, security, and documentation.
Plan data integration with health and care partners, including shared care records and regional data services.
Make sure data flows that support direct care are resilient and have clinical safety cases.
Coordinate data engineering teams and line manage engineers.

Skills

Skill	Expected level	What this level means
Communicating between the technical and non-technicalUK GDaD PCF	Practitioner (3)	You can: listen to and interpret the needs of technical and non-technical stakeholders, and manage their expectations manage active and reactive communication support or host difficult discussions within the team or with diverse senior stakeholders
Data analysis and synthesisUK GDaD PCF	Practitioner (3)	You can: understand and help teams to apply a range of techniques for data profiling source system analysis from a complex single source bring multiple data sources together in a conformed model for analysis
Data compliance and securityUK GDaD PCF	Expert (4)	You can: advise senior stakeholders on data security, ethical or procedural risks improve organisational awareness of data compliance and procedures lead, guide and mentor teams in implementing secure data practices and maintaining compliance
Data development processUK GDaD PCF	Expert (4)	You can: establish cross-organisational data solutions that include all aspects of the data development life cycle define and promote good practices for creating repeatable, reliable and reusable data solutions
Data innovationUK GDaD PCF	Practitioner (3)	You can: test and evaluate the feasibility of unfamiliar and emerging technologies, or familiar technologies in new data contexts share what you learn with the organisation, explaining potential benefits, risks, and practical considerations recommend, design and implement innovative data solutions based on organisation objectives, user needs, and operational constraints
Data integration designUK GDaD PCF	Expert (4)	You can: establish cross-organisational data integration standards and design patterns guide teams in designing secure and interoperable systems and services
Data modellingUK GDaD PCF	Expert (4)	You can: understand the concepts and principles of data modelling and can produce relevant data models work across government and industry, recognising opportunities for the reuse and alignment of data models in different organisations design the method to categorise data models within an organisation
Metadata managementUK GDaD PCF	Practitioner (3)	You can: design an appropriate metadata repository suggest changes to improve current metadata repositories understand a range of tools for storing and working with metadata advise less experienced members of the team about metadata management
Problem managementUK GDaD PCF	Practitioner (3)	You can: ensure that the right actions are taken to investigate, resolve and anticipate problems co-ordinate the team to investigate problems, implement solutions and take preventive measures
Programming and build (data and analytics engineering)UK GDaD PCF	Practitioner (3)	You can: lead the design, code, testing and deployment of secure, resilient and maintainable solutions continuously improve the codebase and reliability of solutions create automation to improve the software development life cycle work with others to implement standards and good practice to ensure security, testability and maintainability of solutions
Understanding health and care servicesThis reference	Practitioner (3)	You can: analyse how a service fits into care pathways across organisations work with clinicians, care staff, and patients to shape digital services explain the effect of digital decisions on care, safety, and staff workload
Information governance and data protectionThis reference	Practitioner (3)	You can: lead data protection impact assessments and information sharing agreements advise teams on lawful basis, consent, confidentiality, and retention investigate incidents and recommend improvements
Health data interoperabilityThis reference	Practitioner (3)	You can: design and build integrations using FHIR, HL7 version 2, and messaging patterns write and profile FHIR resources and implementation guides resolve complex mapping and data quality issues between systems
Clinical terminology and classificationThis reference	Working (2)	You can: find and use the right codes for a data item or form use terminology browsers and reference sets
Pseudonymisation and disclosure controlThis reference	Practitioner (3)	You can: design pseudonymisation and linkage methods that keep identifiers separate from analysis data assess the re-identification risk of a data set or publication and choose suitable controls advise teams on safe settings, such as trusted research environments
Clinical risk managementThis reference	Working (2)	You can: take part in hazard workshops and contribute to a hazard log follow the clinical risk management process for your work provide evidence for a clinical safety case, such as test results
People managementThis reference	Practitioner (3)	You can: line manage a team, setting objectives and running appraisals support wellbeing and manage attendance, performance, and conduct plan the team's development and succession

Typical qualifications and experience

Extensive experience of leading data engineering for complex systems.

Band 8a outline

Knowledge: Expert knowledge of a discipline and its management.
Autonomy: Interprets organisational policy for a service; sets the team's direction.
Scope: A service area or a discipline across the organisation.
Leadership: Manages a team, or leads a discipline without line management.
Accountability: A service area, its staff, and its budget.

Job evaluation (illustrative)

Factor	Level	Level summary	Points
Communication and relationship skills	5	Gives highly complex or contentious information needing developed persuasion and negotiation, presents to large groups, or communicates where there are significant barriers to acceptance.	45
Knowledge, training, and experience	7	Highly developed specialist knowledge across the range of the specialism (master's level or equivalent).	196
Analytical and judgemental skills	5	Judgements on highly complex facts or situations needing analysis, interpretation, and comparison of options.	60
Planning and organisational skills	4	Plans a broad range of complex activities or strategies, adjusting them as needed.	42
Physical skills	2	Skills gained through practice, such as standard keyboard use or driving.	15
Responsibility for patient and client care	1	Helps patients, clients, or relatives during incidental contact.	4
Responsibility for policy and service development	4	Implements policy and develops policy or services for a service or several areas.	32
Responsibility for financial and physical resources	2	Handles cash or valuables, looks after equipment used by others, controls stock, or authorises small payments.	12
Responsibility for people	3	Manages a group of staff day to day, teaches core or specialist training, or gives core HR advice.	21
Responsibility for information resources	5	Designs and develops major information systems, or runs information systems for several services as the main duty.	34
Responsibility for research and development	2	Regularly does research, development, trials, or equipment and system testing as part of the job.	12
Freedom to act	5	Works within general policies, and in most cases decides how to interpret them.	45
Physical effort	1	Sitting, standing, and walking, with little physical effort.	3
Mental effort	4	Frequent prolonged concentration, or occasional intense concentration.	18
Emotional effort	1	Rare exposure, or occasional indirect exposure, to distressing circumstances.	5
Working conditions	2	Occasional unpleasant conditions, frequent travel or outdoor work, or near-continuous screen use on most days.	7
Total (Band 8a: 540 to 584)			551