Data engineer
Data engineers build and run the pipelines and platforms that move health and care data from clinical and operational systems into places where it can be analysed and used safely. They design data flows, integrate sources that use different standards, and make sure data is accurate, timely, secure, and only available to the people who should see it.
- Family
- Data
- UK GDaD PCF role
- Data engineer (Data)
- ESCO occupations
- data engineer (ISCO-08 2511)
- On GitHub
- Role document · Role data
Levels
| Band | Role level | UK GDaD PCF level | Civil Service grades | Job evaluation points |
|---|---|---|---|---|
| 6 | Data engineer | Data engineer | HEO/SEO | 421 |
| 7 | Senior data engineer | Senior data engineer | SEO/G7 | 477 |
| 8a | Lead data engineer | Lead data engineer | G7 | 551 |
| 8b | Head of data engineering | Head of data engineering | G7/G6 | 589 |
In a digital health care organisation
- Source systems include electronic patient records, laboratory, imaging, and pharmacy systems, which use standards such as HL7 version 2, HL7 FHIR, SNOMED CT, and ICD.
- Pipelines often carry identifiable patient data, so engineers build in pseudonymisation, access control, and audit from the start.
- Some data feeds support direct care, such as alerts or patient lists, so a failed or late pipeline can affect patients and needs clinical safety assessment.
- Data often needs linking across care settings, which depends on reliable patient matching and consistent identifiers.
UK GDaD PCF role description
A data engineer develops and constructs data products and services, and integrates them into systems and business processes.
ESCO occupations and skills
ESCO lists the skills and knowledge that employers usually need (essential) or may need (optional) for each occupation. Use them as a checklist for development conversations.
data engineer
Data engineers develop the architecture needed to process, manage, and store large amounts of data which will be used by data scientists for analysis. They design the infrastructure and maintain data pipelines and warehouses to leverage data for strategic advantage.
Essential skills and knowledge (23)
Skills and competences
- create data sets
- design database in the cloud
- develop data processing applications
- establish data processes
- implement data warehousing techniques
- manage ICT data architecture
- manage data
- manage quantitative data
- manage research data
- perform dimensionality reduction
- process data
- store digital data and systems
- use data processing techniques
- use databases
Knowledge
- cloud technologies
- computer science
- data analytics
- data models
- data storage
- data warehouse
- database management systems
- unstructured data
Other
Optional skills and knowledge (5)
Skills and competences
Knowledge
Contains ESCO v1.2.1 data (https://esco.ec.europa.eu/). © European Union. Reuse authorised under Commission Decision 2011/833/EU.