.. _modify-metadata-label: Metadata Handling ======================= Within a Project, there are two types of metadata: - **Project-level Metadata**: This metadata is configured at the Project level and applies to each session (Subject ID/Session ID). - **File-level Metadata**: This metadata applies to the files within a session. In this section, you will learn how to programmatically modify any metadata available in the active Project. To begin, log in and activate a Project following the steps in :ref:`login-label`. Project-level Metadata ------------------------- To retrieve the metadata available at Project-level, use: .. code-block:: python p_metadata = project.metadata_parameters where :code:`p_metadata` is a dictionary in which each key corresponds to a different metadata field available. Further, each value of this dictionary is itself another dictionary with following keys: - :code:`title`: Title of the metadata parameter. - :code:`id`: Identification code of the metadata parameter (this code corresponds to the defined :code:`PROJECTMETADATAKEY` in :ref:`data-asearch-label`) - :code:`type`: Type of the metadata parameter. - :code:`visible`: Boolean determining if the parameter is visible in the QMENTA Platform or not. - :code:`readonly`: Boolean determining if the parameter is editable via the QMENTA Platform or not. - :code:`mandatory`: Boolean determining if the parameter must be defined upon data upload. - :code:`description`: Description of the metadata parameter. - :code:`options`: List of values the metadata parameter can optionally take with type: single/multiple-option. - :code:`order`: Order in which the parameter is shown in the QMENTA Platform. An example :code:`p_metadata` output taken from Project with only one parameter is: .. code-block:: python { 'gender': { 'title': 'Gender', 'id': 'gender', 'type': 'single_option', 'visible': 1, 'readonly': 1, 'mandatory': 0, 'description': '', 'options': [ {'value': 'male', 'label': 'M'}, {'value': 'female', 'label': 'F'}, {'value': 'unknown', 'label': 'UN'} ], 'order': 1 } } .. note:: When a new Project is created in the QMENTA Platform, it will not yet have any of these configurable Project-level metadata parameters. .. note:: Project-level metadata parameters can only be configured via the QMENTA Platform. In addition to configurable Project-level metadata parameters, there are four metadata parameters that are included in every Project within the QMENTA Platform upon Project creation. - :code:`age_at_scan`: Age of the subject on the date the acquisition was performed. - :code:`date_at_scan`: Date in which the acquisition was performed. - :code:`qa_status`: Quality Check (QC) status of the session. - :code:`qa_comments`: Quality Check (QC) comments of the session. If the data uploaded to a session is DICOM, the fields :code:`age_at_scan` and :code:`date_at_scan` are automatically populated by the Medical Image Data analysis during upload. Now, using the methods already explained in :ref:`data-search-label` you can obtain the Project-level metadata parameters for the sessions available in the active Project. An example is shown below: .. code-block:: python # Define the Subject ID. subject_name = "0001" # Retrieve Subject Information subjects = [subject for subject in project.subjects_metadata if subject["patient_secret_name"] == subject_name] where :code:`subject_name` corresponds to the Subject ID and :code:`subjects` is a list of the sessions available for the Subject ID. Particularly, each session in the list is a dictionary which contains the available Project-level metadata such as: - :code:`age_at_scan`. - :code:`date_at_scan`. - :code:`md_gender`. - :code:`qa_status`. - :code:`qa_comments`. .. note:: Metadata data parameter :code:`md_gender` was defined in the example above. Note that the :code:`md_` is automatically added to the identification code. To visualize all the information available for the session you can simply: .. code-block:: python print(subjects[0]) An example of a Project in the QMENTA Platform is shown next. .. figure:: figs/modify_metadata/1.webp Example Project-level metadata available in a Project in the QMENTA Platform. Project-level Metadata Modification ---------------------------------------------------------------- In order to modify the value of a given Project-level parameter of a session (Subject ID/Session ID), you have to first retrieve the parameters from the session, then modify them as needed, and subsequently apply the update to the parameters of the session, as indicated next: .. code-block:: python # Define the Subject ID and Session ID # to unequivocally identify a session. subject_name = "0001" ssid = "1" # Retrieve the Session Information session = [subject for subject in project.subjects_metadata if subject["patient_secret_name"] == subject_name and subject["ssid"] == ssid] patient_id = session[0]["_id"] age_at_scan = session[0]["age_at_scan"] # Modify the desired parameters tags = list(set(subject["tags"] + ["new tag"])) # Add new tag to the subject. metadata = {"gender":"male"} # Update Gender metadata. # Apply the change project.change_subject_metadata(patient_id, subject_name, ssid, tags, age_at_scan, metadata) If you now retrieve the metadata information of the session once again it will have been updated. .. note:: Any metadata can be changed except for the :code:`patient_id`. .. _file-metadata-label: File-level Metadata ---------------------- The data available within a session also have metadata. It can be accessed by identifying the ID of the storing unit (i.e., Container ID) and then retrieving the metadata as follows: .. code-block:: python subject_name = "0001" ssid = "1" # Container ID Identification container_id = project.get_subject_container_id(subject_name, ssid) # Files Metadata Extraction files_metadata = project.list_container_files_metadata(container_id) where :code:`files_metadata` is a list of dictionaries each belonging to a different file in the container. Each dictionary has the following keys: - :code:`name`: Name of the file. - :code:`size`: Size of the file in bytes. - :code:`tags`: List of tags associated with the file. - :code:`metadata`: Dictionary with further information about the file. .. note:: DICOM Series are stored compressed into a ZIP file. This last dictionary further contains the following keys: - :code:`info`: Metadata file information extracted by the Medical Image Data analysis. - :code:`format`: File format. - :code:`protocol`: File name without the file extension. - :code:`hidden`: Boolean indicating if the file is visible within the container. - :code:`modality`: Modality of the file. An example of all the metadata file information extracted during the upload process for a file can be visualized as follows: .. code-block:: python print(f"File Name: {files_metadata[0]['name']}) print(f"File Tags: {files_metadata[0]['tags']}) print(f"File Modality: {files_metadata[0]['metadata']['modality']}) print(f"File Metadata: \n\n{files_metadata[0]['metadata']['info']}") .. note:: The extraction of this metadata file information is heavily dependent on the type of file. Particularly, on the header of the file. As such, while DICOM Series have a large set of metadata available, others such as NIfTI or PNGs barely have any metadata. An example of the file-level metadata is shown next. .. figure:: figs/modify_metadata/2.webp Example File-level metadata. See the list of DICOM tags and their values available on the right-hand side of the image. The tags available depend on the file. File-level Metadata Modification ----------------------------------- Amongst all the metadata belonging to a file, only the File Modality and Tags can be manually modified. See the example below where we have updated the modality and tags of the file in the first position of :code:`files_metadata`.: .. code-block:: python subject_name = "0001" ssid = "1" # Container ID Identification container_id = project.get_subject_container_id(subject_name, ssid) # Get File Names files_metadata = project.list_container_files_metadata(container_id) # Show the name of the file to modify print(f"File Name: {files_metadata[0]['name']}) # Modify the modality and tag of the first file of the list. new_modality = "DWI" new_tags = files_metadata[0]['tags'] + ["modified"] project.change_file_metadata(container_id, files_metadata[0]['name'], new_modality, new_tags) .. _qc-subjects-label: Quality Check (QC) Status ---------------------------- The Quality Check (QC) Status facilitates the process of assuring the quality of uploaded data in terms of: - Adherence to the acquisition protocol. - Image quality. The QC status related metadata (:code:`qa_status` and :code:`qa_comments`) have specific methods to read and modify it. .. code-block:: python from qmenta.client.Project import QCStatus # Reading the QC status related metadata subject_name = "0001" ssid = "1" qc_status, qc_comments = project.get_qc_status_subject(subject_name=subject_name, ssid=ssid) # Modifying the QC status related metadata. patient_id = project.get_subject_id(subject_name, ssid) qc_comments = "The quality of the images available in this session are correct." qc_status = QCStatus.PASS project.set_qc_status_subject(patient_id, qc_status, qc_comments) where :code:`qc_comments` is a string and :code:`qc_status` is a value class :class:`QCStatus` which can be either of: - :code:`PASS` - :code:`FAIL` - :code:`UNDETERMINED` The QC Status is visualized in the QMENTA Platform as follows: .. figure:: figs/modify_metadata/3.webp Example appearance of the different values of the QC Status in the QMENTA Platform. .. _paa-label: Protocol Adherence Automation Analysis ----------------------------------------- In the QMENTA Platform the quality of the uploaded data can be checked in several ways and the result of the inspection can be stored within the QC Status. The Protocol Adherence Automation analysis is an in-house analysis implemented in the QMENTA Platform that allows the comparison of the uploaded data of a session (i.e., Subject ID/Session ID) to a set of predefined rules. If the rules are aligned with the imaging acquisition protocol of the study, the Protocol Adherence Automation Analysis will facilitate quick identification of protocol deviations. See `attached an example set of predefined rules `_ specifically designed to assess the adherence of the data to the ADNI 4 protocol. Upon execution of the analysis, the QC Status and QC Comments of the session are updated with automatically extracted information. Particularly, if the imaging session is in adherence with the acquisition protocol the QC Status will be set to :code:`PASS`, or to :code:`FAIL` otherwise. Further, the QC Comments are automatically updated with the summary information of the decision. Execution of the Protocol Adherence Automation analysis requires retrieval of the session of interest's Container ID. The analysis can then be started as follows: .. code-block:: python subject_name = "0001" ssid = "1" # Container ID Identification container_id = project.get_subject_container_id(subject_name, ssid) # Start analysis analysis_id = project.start_analysis( script_name='qmenta_protocol_adherence_automation', version='1.9', settings={"input":container_id}, analysis_name = f"Protocol Adherence Automation (v.{version})", ) .. note:: This is an example of how to programmatically start an analysis within the QMENTA Platform. For further details read section :ref:`launch-analysis-label`. An example result of the Protocol Adherence Automation analysis. .. figure:: figs/modify_metadata/4.webp Example result of the Protocol Adherence Automation analysis. You can see the Project is configured with several rules and while the :code:`DTI` and :code:`DTI GFM AP` rules pass the :code:`3D_T1` rule fails. Hence, the overall QC Status is set to :code:`FAIL`. Protocol Adherence Rules ++++++++++++++++++++++++++++++ The set of protocol adherence rules predefined in the active Project can be obtained via: .. code-block:: python description = project.get_project_pa_rules("Path to file: Rules.json") The method above downloads the rules applied by the Protocol Adherence Automation analysis and stores them in the specified path. Further, it outputs :code:`description` which contains a brief description of the rules. After downloading the rules, you can modify them manually, and set them again in the same or other Project with the following command: .. code-block:: python project.set_project_pa_rules("Path to file: Rules.json", description) .. note:: The above command only sets the Protocol Adherence rules in the active Project. Next Steps ---------- Explore the following sections in detail: - :ref:`launch-analysis-label` – Execute analyses. - :ref:`download-results-label` – Retrieve and store analysis outputs.