Data engineering

Quoted from the UK GDaD PCF.

Levels

1. Awareness

You can:

• explain why you need engineering support to design and provide products for the organisation
• explain why you need to cleanse and prepare data for use in data science products
• explain different architectures, including cloud and on-premise, and data manipulation and transformation tools

2. Working

You can:

• identify the appropriate engineering support to design and provide products for the organisation
• recognise the processes and tools for cleansing and preparing data, and put reusable processes and checks in place
• use different architectures, including cloud and on-premise
• use data manipulation and transformation tools

3. Practitioner

You can:

• work with data engineers and data scientists to design and provide products for the organisation

4. Expert

You can:

• help to identify the data engineering requirements for any data science product, while working with data engineers and data scientists to design and deliver those products into the organisation effectively
• understand the need to cleanse and prepare data before including it in data science products and can put reusable processes and checks in place
• understand a broad range of architectures, including cloud and on-premise, and data manipulation and transformation tools deployed within the organisation, and can use these tools appropriately and help others use them

Contains public sector information from the UK Government Digital and Data Profession Capability Framework (https://understand-digital-data-roles-skills.service.gov.uk/), licensed under the Open Government Licence v3.0. © Crown copyright.

Closest ESCO skills

• data engineering (exact match)
• establish data processes (narrow match)

Contains ESCO v1.2.1 data (https://esco.ec.europa.eu/). © European Union. Reuse authorised under Commission Decision 2011/833/EU.

Role levels that expect this skill