{
"cells": [
{
"cell_type": "markdown",
"id": "beb97ca4",
"metadata": {},
"source": [
"# Study Permissions Dataset"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "e3b8790c",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"text/markdown": [
">Last modified: 14 Nov 2025"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import sys\n",
"import os\n",
"sys.path.append(os.path.abspath('../../../../scripts/'))\n",
"from data_doc_helper import UKLLCDataSet as DS, last_modified\n",
"API_KEY = os.environ['FASTAPI_KEY']\n",
"ds = DS(\"study_permissions\")\n",
"last_modified()"
]
},
{
"cell_type": "markdown",
"id": "5b3e81e6",
"metadata": {},
"source": [
"<div style=\"background-color: rgba(0, 178, 169, 0.3); padding: 5px; border-radius: 5px;\"><strong>UK LLC has created a study permissions dataset which serves as the UK LLC denominator or 'spine'.</strong></div> "
]
},
{
"cell_type": "markdown",
"id": "fe073008",
"metadata": {},
"source": [
"## 1. Summary"
]
},
{
"cell_type": "markdown",
"id": "c938a7b1",
"metadata": {},
"source": [
"The **study permissions** dataset, formerly know as the **denominator** dataset, incorporates all participants across all LPS. Each project in the TRE is automatically provisioned a file named 'study_permissions_v0000_YYYYMMDD' (formerly known as 'CORE_denominator_file1_v0000_YYYYMMDD'). The date in the dataset name relates to the \"File 1\" [**freeze**](../../../../ukllc_key_facts/Sample/UKLLC_sample.md) (permissions) to which each project is fixed. Participant permissions are correct and applied to each dataset as of this date.\n",
"\n",
"The dataset **enables researchers to**:\n",
"* calculate linkage rates (when joined to linked datasets)\n",
"* contextualise LPS participants within UK LLC in relation to the LPS datasets. \n",
"\n",
"When considering NHS England and Wales linkage permission flags (*nhs_e_linkage_permission* and *nhs_w_linkage_permission*), you need to also consider the *national_opt_out* flag. When *national_opt_out* is set to \"0\", this means that the participant has set **explicit consent** with their LPS. Where *national_opt_out* is set to \"1\", this means that the participant has flowed for linkage under [**Section 251 support**](https://www.hra.nhs.uk/about-us/committees-and-services/confidentiality-advisory-group/confidential-patient-information-and-regulations/). Participants who flow under **Section 251** are filtered by linked data owners for **national data opt-outs** (NDOOs)- the health records of any section 251 participant who has set an NDOO will not flow to UK LLC. Note, however, that exclusion of participants whose *national_opt_out* flag is set to \"1\", cannot be inferred to mean that they had set an NDOO. NHS data owners do not notify UK LLC of which participants have set NDOOs and there are other reasons for participants' health data not flowing, such as failed linkage due to quality of personal identifiers."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "fb0becc7",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"text/html": [
"<style type=\"text/css\">\n",
"#T_7b3c9 th {\n",
" text-align: left;\n",
"}\n",
"#T_7b3c9_row0_col0, #T_7b3c9_row0_col1, #T_7b3c9_row1_col0, #T_7b3c9_row1_col1, #T_7b3c9_row2_col0, #T_7b3c9_row2_col1, #T_7b3c9_row3_col0, #T_7b3c9_row3_col1, #T_7b3c9_row4_col0, #T_7b3c9_row4_col1, #T_7b3c9_row5_col0, #T_7b3c9_row5_col1, #T_7b3c9_row6_col0, #T_7b3c9_row6_col1, #T_7b3c9_row7_col0, #T_7b3c9_row7_col1, #T_7b3c9_row8_col0, #T_7b3c9_row8_col1, #T_7b3c9_row9_col0, #T_7b3c9_row9_col1, #T_7b3c9_row10_col0, #T_7b3c9_row10_col1, #T_7b3c9_row11_col0, #T_7b3c9_row11_col1 {\n",
" text-align: left;\n",
"}\n",
"</style>\n",
"<table id=\"T_7b3c9\" style=\"font-size: 14px\">\n",
" <thead>\n",
" <tr>\n",
" <th id=\"T_7b3c9_level0_col0\" class=\"col_heading level0 col0\" >Dataset Descriptor</th>\n",
" <th id=\"T_7b3c9_level0_col1\" class=\"col_heading level0 col1\" >Dataset-specific Information</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td id=\"T_7b3c9_row0_col0\" class=\"data row0 col0\" >Name of Dataset in TRE</td>\n",
" <td id=\"T_7b3c9_row0_col1\" class=\"data row0 col1\" >UKLLC_study_permissions</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_7b3c9_row1_col0\" class=\"data row1 col0\" >Citation (APA)</td>\n",
" <td id=\"T_7b3c9_row1_col1\" class=\"data row1 col1\" >UK Longitudinal Linkage Collaboration. (2025). <i>UK LLC Managed: Study Permissions Dataset - Dated 07 Apr 2025.</i> UK Longitudinal Linkage Collaboration (UK LLC). <a href=\"https://doi.org/10.71760/ukllc-dataset-00460-01\" rel=\"noopener noreferrer\" target=\"_blank\">https://doi.org/10.71760/ukllc-dataset-00460-01</a></td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_7b3c9_row2_col0\" class=\"data row2 col0\" >Download Citation</td>\n",
" <td id=\"T_7b3c9_row2_col1\" class=\"data row2 col1\" > <a href=\"https://api.datacite.org/application/vnd.citationstyles.csl+json/10.71760/ukllc-dataset-00460-01\" rel=\"noopener noreferrer\" target=\"_blank\">Citeproc JSON</a> <a href=\"https://api.datacite.org/application/x-bibtex/10.71760/ukllc-dataset-00460-01\" rel=\"noopener noreferrer\" target=\"_blank\">BibTeX</a> <a href=\"https://api.datacite.org/application/x-research-info-systems/10.71760/ukllc-dataset-00460-01\" rel=\"noopener noreferrer\" target=\"_blank\">RIS</a></td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_7b3c9_row3_col0\" class=\"data row3 col0\" >Series</td>\n",
" <td id=\"T_7b3c9_row3_col1\" class=\"data row3 col1\" > <a href=\"https://guidebook.ukllc.ac.uk/docs/ukllc_managed_data/ukllc_data\">UK LLC Managed</a></td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_7b3c9_row4_col0\" class=\"data row4 col0\" >Owner</td>\n",
" <td id=\"T_7b3c9_row4_col1\" class=\"data row4 col1\" >UK Longitudinal Linkage Collaboration</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_7b3c9_row5_col0\" class=\"data row5 col0\" >Temporal Coverage</td>\n",
" <td id=\"T_7b3c9_row5_col1\" class=\"data row5 col1\" >Unknown - Unknown</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_7b3c9_row6_col0\" class=\"data row6 col0\" >Keywords</td>\n",
" <td id=\"T_7b3c9_row6_col1\" class=\"data row6 col1\" ></td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_7b3c9_row7_col0\" class=\"data row7 col0\" >Participant Count</td>\n",
" <td id=\"T_7b3c9_row7_col1\" class=\"data row7 col1\" >332157</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_7b3c9_row8_col0\" class=\"data row8 col0\" >Number of variables</td>\n",
" <td id=\"T_7b3c9_row8_col1\" class=\"data row8 col1\" >15</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_7b3c9_row9_col0\" class=\"data row9 col0\" >Number of observations</td>\n",
" <td id=\"T_7b3c9_row9_col1\" class=\"data row9 col1\" >326421</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_7b3c9_row10_col0\" class=\"data row10 col0\" >Specific Restrictions to Data Use</td>\n",
" <td id=\"T_7b3c9_row10_col1\" class=\"data row10 col1\" >None</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_7b3c9_row11_col0\" class=\"data row11 col0\" >Build a Data Request</td>\n",
" <td id=\"T_7b3c9_row11_col1\" class=\"data row11 col1\" > <a href=\"https://explore.ukllc.ac.uk/\" rel=\"noopener noreferrer\" target=\"_blank\">https://explore.ukllc.ac.uk/</a></td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n"
],
"text/plain": [
"<pandas.io.formats.style.Styler at 0x150f3489400>"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ds.info_table()"
]
},
{
"cell_type": "markdown",
"id": "799ad0f1",
"metadata": {},
"source": [
"## 2. Variables"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "4c6731f3",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"text/html": [
"<style type=\"text/css\">\n",
"#T_3171f th {\n",
" text-align: left;\n",
"}\n",
"#T_3171f_row0_col0, #T_3171f_row0_col1, #T_3171f_row1_col0, #T_3171f_row1_col1, #T_3171f_row2_col0, #T_3171f_row2_col1, #T_3171f_row3_col0, #T_3171f_row3_col1, #T_3171f_row4_col0, #T_3171f_row4_col1, #T_3171f_row5_col0, #T_3171f_row5_col1, #T_3171f_row6_col0, #T_3171f_row6_col1, #T_3171f_row7_col0, #T_3171f_row7_col1, #T_3171f_row8_col0, #T_3171f_row8_col1, #T_3171f_row9_col0, #T_3171f_row9_col1, #T_3171f_row10_col0, #T_3171f_row10_col1, #T_3171f_row11_col0, #T_3171f_row11_col1, #T_3171f_row12_col0, #T_3171f_row12_col1, #T_3171f_row13_col0, #T_3171f_row13_col1, #T_3171f_row14_col0, #T_3171f_row14_col1 {\n",
" text-align: left;\n",
"}\n",
"</style>\n",
"<table id=\"T_3171f\" style=\"font-size: 14px\">\n",
" <thead>\n",
" <tr>\n",
" <th id=\"T_3171f_level0_col0\" class=\"col_heading level0 col0\" >Variable</th>\n",
" <th id=\"T_3171f_level0_col1\" class=\"col_heading level0 col1\" >Description</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td id=\"T_3171f_row0_col0\" class=\"data row0 col0\" >study_id_e</td>\n",
" <td id=\"T_3171f_row0_col1\" class=\"data row0 col1\" >None</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row1_col0\" class=\"data row1 col0\" >cohort</td>\n",
" <td id=\"T_3171f_row1_col1\" class=\"data row1 col1\" >Longitudinal Population Study participant belongs to</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row2_col0\" class=\"data row2 col0\" >ukllc_status</td>\n",
" <td id=\"T_3171f_row2_col1\" class=\"data row2 col1\" >Overall UK LLC permission </td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row3_col0\" class=\"data row3 col0\" >nhs_e_linkage_permission</td>\n",
" <td id=\"T_3171f_row3_col1\" class=\"data row3 col1\" >Permission to link to NHS England records</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row4_col0\" class=\"data row4 col0\" >nhs_ni_linkage_permission</td>\n",
" <td id=\"T_3171f_row4_col1\" class=\"data row4 col1\" >Permission to link to NHS records in Northern Ireland</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row5_col0\" class=\"data row5 col0\" >nhs_s_linkage_permission</td>\n",
" <td id=\"T_3171f_row5_col1\" class=\"data row5 col1\" >Permission to link to Scottish NHS records</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row6_col0\" class=\"data row6 col0\" >nhs_w_linkage_permission</td>\n",
" <td id=\"T_3171f_row6_col1\" class=\"data row6 col1\" >Permission to link to Welsh NHS records</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row7_col0\" class=\"data row7 col0\" >geocoding_permission</td>\n",
" <td id=\"T_3171f_row7_col1\" class=\"data row7 col1\" >Permission to geocode and link to records with postcode/UPRN</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row8_col0\" class=\"data row8 col0\" >zoesymptomtracker_permission</td>\n",
" <td id=\"T_3171f_row8_col1\" class=\"data row8 col1\" >Permission to link to Zoe Sympton Tracker data</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row9_col0\" class=\"data row9 col0\" >national_opt_out</td>\n",
" <td id=\"T_3171f_row9_col1\" class=\"data row9 col1\" >LPS is using opt-out and opt-out to be applied to participant</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row10_col0\" class=\"data row10 col0\" >avail_from_dt</td>\n",
" <td id=\"T_3171f_row10_col1\" class=\"data row10 col1\" >None</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row11_col0\" class=\"data row11 col0\" >llc_join_date</td>\n",
" <td id=\"T_3171f_row11_col1\" class=\"data row11 col1\" >Date participant join UK LLC</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row12_col0\" class=\"data row12 col0\" >hmrc_linkage_permission</td>\n",
" <td id=\"T_3171f_row12_col1\" class=\"data row12 col1\" >None</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row13_col0\" class=\"data row13 col0\" >dwp_linkage_permission</td>\n",
" <td id=\"T_3171f_row13_col1\" class=\"data row13 col1\" >None</td>\n",
" </tr>\n",
" <tr>\n",
" <td id=\"T_3171f_row14_col0\" class=\"data row14 col0\" >dfe_linkage_permission</td>\n",
" <td id=\"T_3171f_row14_col1\" class=\"data row14 col1\" >None</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n"
],
"text/plain": [
"<pandas.io.formats.style.Styler at 0x150f34f1a90>"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ds.variable_table()"
]
},
{
"cell_type": "markdown",
"id": "8a469c49",
"metadata": {},
"source": [
"## 3. Version History"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "41fd9910",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"# ds.version_history()"
]
},
{
"cell_type": "markdown",
"id": "81e0d149",
"metadata": {},
"source": [
"TBC"
]
},
{
"cell_type": "markdown",
"id": "bedf5551",
"metadata": {},
"source": [
"## 4. Useful Syntax"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "29e9d5bb",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [
{
"data": {
"text/markdown": [
"Below we will include syntax that may be helpful to other researchers in the UK LLC TRE. For longer scripts, we will include a snippet of the code plus a link to Git where you can find the full scripts."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"ds.useful_syntax()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "jupbook",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.2"
}
},
"nbformat": 4,
"nbformat_minor": 5
}