HD4DP v1HD4DP v1
Please be advised that HD4DP v1 will soon be phased out. This application will be replaced by HD4DP v2.
The exact date when the project will be available in HD4DP v2 will be communicated soon.
General description of the application HD4DP v1General description of the application HD4DP v1
HD4DP v1 ("healthdata.be for data providers") is a local application accessible via web browser developed by healthdata.be (Sciensano) and installed at the site of the data provider (healthcare organizations). This application allows the data provider to extract data from the primary system and map that data into one or more data collections (if possible).
Clinical information that cannot be provided by the primary systems can be provided manually in the ‘prefilled’ form view of the Data Collection Definition (DCD).
The (form view of the) Data Collection Definitions (DCDs) are managed centrally via a metadata catalogue.
User manual of the application HD4DP v1User manual of the application HD4DP v1
An extensive user manual, for both participating healthcare organizations and their IT service providers, is available on this docs.healthdata.be portal: https://docs.healthdata.be/hd4dp-v1.
Request account for HD4DP v1Request account for HD4DP v1
To access the HD4DP v1 application, you need an account, a username and a password. If you do not have an account, you can request one by following the steps below. If you have an account, follow the "sign in" instructions at the end of this page.
- Open the link for the HD4DP v1 application. You will see this screen:
- Click on the Request account link on the login page.
- Fill out the request form:
- Enter a username, first name, last name and e-mail address
- Select the organization and data collection(s)
- Fill in the Requester e-mail field if a person requests an account for a third person
- Submit the request user the Request button
- Confirmation e-mails are sent to the person for whom the account was requested as well as the requester, if the field "Requester e-mail" was filled.
- The approval or rejection for the user account is confirmed by mail. Depending on your organization, this could take a few hours.
- This action will be done by an HD4DP v1 administrator in your organization
- The confirmation mail will include all the necessary information to log in
Logging in to the HD4DP v1 application is done in 3 steps:
- Enter your username and password in the appropriate fields
- Select the correct organization
- Click on "Log in"
Create a registration in HD4DP v1Create a registration in HD4DP v1
The Data Collection view leads a user to the existing registers. A user can start collecting data from this view.
Creating and submitting a registration consists of 4 steps:
- Select the register for creating a registration.
- Click on the "New registration" button.
- Save the registration temporarily if needed. The status of the record will be "Open" for saved registrations.
- Submit the registration if no validation errors present. The status of the registration will change to "Sending".
- When the record has been processed, the status will change to "Submitted"
As soon as a registration is sent, the "status confirmation" column will show "Pending".
- It shows "OK" once the registration has arrived, and "NOK" if the registration did not arrive within 48 hours. If the status is "NOK", the software will automatically try to resend the registration up to 10 days after the initial submission.
- For the statuses "Sending","Corrections needed" and "Approved" the status confirmation will be empty because the registration has not yet been submitted.
- A registration can be "Reopened" if needed as long as the status of the registration is "Submitted".
Change a registration in HD4DP v1Change a registration in HD4DP v1
A user can modify and complete a registration in four steps:
- Select the register from which you wish to modify one or more registration(s).
- Click on the registration to be changed, and complete the form in full. The status of the record is then:
- Open for registrations saved manually or by uploading a csv file
- Corrections needed for registrations with errors
- Save registration temporarily if necessary. Record status becomes Open for stored registrations.
- Send the registration if there are no more validation errors.
- Registration status changes to Sending
- When the record is processed, the status changes to Submitted.
Once a registration is sent, the message Pending appears in the status confirmation column.
- When the registration is received, the message OK appears; if the registration is not received within 48 hours, the message NOK appears. With the status "NOK", the software will automatically try to send the registration up to 10 days after the initial submission.
- For the statuses Sending, Corrections needed and Approved, the status confirmation is empty because the registration has not yet been sent.
- A registration can be Reopened if necessary, as long as the status of the registration is "Submitted".
Delete a registration in HD4DP v1Delete a registration in HD4DP v1
Only registrations with status Open or Corrections needed can be deleted.
The following steps are required to delete the registration:
- Select the record and version for which you wish to delete a registration
- Select the registration(s) you wish to delete
- Select the Actions button and select the Delete registrations option
Technical manual of the application HD4DP v1Technical manual of the application HD4DP v1 Adelaide.DAmore Tue, 04/18/2023 - 11:16
HD4DP v1 csv uploadHD4DP v1 csv upload
The upload center is introduced to make the upload of multiple registrations more performant and user friendly.
The link to the upload center can be reached by the main menu:
If uploads are processed, an overview with the name of the uploade file, the status and a visual representation of the status, the number of records that were processed, the way of uploading, the user and the upload date is saved in the main screen of the upload center is given to the user a quick and visual impression of the data uploaded in bulk by using the upload center:
In the right upper corner two buttons are shown.
- Configuration: the default settings for a register can be set by any user that is authorized to participate in a register.
Following options are available:
- Conflict mode: what need to be done in case of conflict
- Conflict master: is the data from the new registration of from the record in the database saved in case of conflict
- Ignore duplicate records
- Autosubmit: send the data automatically for processing in HD4RES
- Validation: what need to be done after validation the registration:
- None: only the registrations without errors will be saved
- Save on validations: Save the registrations when there are validation errors upon uploading
- Validate all: Only perform commit if non of its records have (errors or) validation errors
4 steps need to be executed for a new upload. When the titel bar of a separate step turns green, orange or red and the icon of the step in the overview, the step has respectively ended with success, with warnings or errors.
Step 1 - Select file
The first step is to select the CSV file with the data to be uploaded. The default settings for a register can be found in the Configurations on the main screen of the upload center.
Be sure to deselect the "Use default configurations" when changing the default settings:
For the BCR register, the possibility exists to upload two files and link them during the process:
- Both documents will be validated separately. By using the link "Click here to see the linked upload" the user can switch between the linked documents. Refresh of push F5 on you to see the status of the upload.
After uploading succesfully, the title bar of this step will turn green.
Step 2 - Validity Check
Validation checks are executed for every record.
Step 3 - Upload
The file is being uploaded and the result is shown in this step per line of data added:
Step 4 - Finalize
The result is shown visually for all registrations and for all registrations in a CSV file in reports after finalization:
Succes report is a CSV file with all registrations who were processed succesfully.
In case of errors, a link to a detailed error report is shown too:
Please find below the movie that guides you through the functionalities of the Upload Center:
Create a csv for HD4DP v1Create a csv for HD4DP v1
Processing registrations in bulk is done by uploading CSV files. These files are plain text files which can contain multiple registrations at once and are extracted from your primary system.
The first step is to create the file correctly.
Using a CSV editor
While Excel is a fine tool to view CSV’s, we do not recommend it to edit CSVs. Instead use notepad++ or any other text editor. Here are a few risks you should be aware of when editing a CSV in Excel. Excel will interpret the content, which may lead to changes:
- Leading zeros disappear in fields that are recognized as numeric fields
- Entries like 3-9 can become March-9
- The only accepted date format DD/MM/YYYY can be modified (e.g. To DD/MM/YY)
- The decimal separator can differ from that in HD4DP, a semicolon wil lead to a correct upload
- When saving a file as .csv, Excel uses the default field separator. HD4DP only accepts CSV with a semicolon as separator. This default setting can be adapted in the properties of your computer.
- CSV encoding must be in UTF-8.
Setting up the document
Every column in the CSV file needs to be recognized as a field of the register by the HD4DP application. Therefor each column in the file must be identical by the technical name of the field in the register.
Tip: Downloading (manually) entered data from HD4DP will guide you in formatting a CSV file and may help during the development of the CSV extraction from the primary systems.
The Data Collection Definition (DCD) specifications for a register and its fields are defined and documented on this documentation portal.
Each field in the form can be completed through a value in a CSV file.
An example field is “Date of last follow-up”, shown in the screenshot below.
This field is of the type “date” and is required (*). Within the technical documentation of this data collection, this is shown as follows:
To include this field in a CSV upload file, it is sufficient to create a column with the name “date_of_last_followup” and populate it with the appropriate data i.e. a date in the format dd/mm/yyyy.
Fields can be required, read-only and computed (automatically calculated). Fields can also have a default value.
This information is present in the detailed technical description for each data collection.
- The column separator is the semicolon (;)
- The decimal separator for numbers is the comma (,)
- The date format is dd/mm/yyyy
Basic content types
Depending on the type of the field, a different representation of the data is expected. The table below describes the different basic types and the rules on how to provide the content for these types.
|Content type||Expected format/content|
|choice||code from choice list|
|number||number (decimal separator = ,)|
|patientID||SSIN number. If the person does not have a SSIN, leave this field blank.|
|questionnaire||code from questionnaire answer list|
|text||● free text|
● if a binding reference list is used: a code from the reference list
● if a non-binding reference list is used: a code from the reference list or free text
|attachment||● expected format/content: Name of the file that must be attached (e.g. protocol.txt).|
● expected extension: .txt
● file must be stored in the same folder as the folder that is used for the CSV-upload
Advanced content types
Other than these simple types, more complex data structures can be used, as shown in the table below. Each of these types is explained in more detail below the table.
|Content type||CSV column name||Expected format|
|fields within fieldset||fieldset_label|field_label||depending on the field type|
|list (1 field)||list_label|field||value1|value2|etc|
|list (block of fields)||list_label|0|field1|
|depending on the field type|
|nested fields below choice or multichoice||choice_label|nested_item||depending on the field type|
A fieldset is a collection of fields, as shown in the image below:
Anthropometry is the title of the fieldset, and this fieldset contains two fields, weight and height. Fieldsets fieldsets do not have a number cfr. image below - Anthropometry.
Sections do not have an impact on the CSV file, whereas fieldsets do. The title of the fieldset must be included in the field name column as follows: fieldset_label|field_label.
E.g. for the two Anthropometry fieldset fields weight and height below, the correct CSV column headers are: anthropometry|weight en anthropometry|height.
A list is also a collection of fields, like a fieldset, but with the additional property that the collection of fields can be repeated.
An example is shown in the image below: “Birthdays of the biological children for this patient” is a list. One list item consists of two fields, “Child birth month” and “Child birth year”. For each child, a list item can be added.
The CSV column names consist of the list header label and the field label (as for fieldsets), together with a counter to distinguish the different list items. The correct CSV column names for the two list items below are:
Please note that for every line, the numbers should increment, starting from 0 (|00|,|01|, .. is ok, |01|, |03|, ... is not). You can't have blank values for |00| and filled values for a higher number.
Please note that the numbering requires a stable format, meaning the number of characters used by the number has to be constant. You can't have one record using |00| and another using |0|. Generally we advice to use a string length of 2 digits.
For lists consisting of 1 field a simplified implementation is possible. The CSV column header only consists of the list header label and multiple values are provided in the one column, separated by a pipe (|).
E.g. for the list in the image below, the CSV column header is diagnosis_orphacode and the content of the column is 562|702. This is the example of a text field with a reference list: providing the codes of the reference list is sufficient.
Nested fields are fields that only become available when specific options are selected in the form. An example is shown below: the field “Specify” only becomes available if the checkbox “Other” is marked. These fields also have a combined CSV column header, consisting of the choice list label and the field label. For the example below, the correct CSV column header is hence base_of_diagnosis|specify.
When a CSV is prepared and put in the provisioning folder, it can contain references to attachments for data collections that specifically allow this.
These references are relative paths to the file location. If such a reference is present in the CSV file, the attachment content is uploaded and linked to the created registration. The attachment is then available in the HD4DP client as well as in the HD4RES client when the registration is submitted.
The maximum file size for attachments is 6 MB.
If a data collection permits you to send attachments you should have the column name to use in the CSV. If not, you should be able to find it at https://www.healthdata.be/dcd/#/collections or you can contact Support.Healthdata@sciensano.be.
Add the column name to the header of the CSV and add the file names as values in the column.
Put the CSV file in the correct provisioning folder (organization sub folder, then in the register sub folder), along with a “picture.png” file of your choice. The application picks the CSV file and creates a new registration.
Open the registration and verify the attachment has correctly been uploaded.
CSV download and upload for stable dataCSV download and upload for stable data
Before starting a new data collection based on stable data of another data collection, you need to finalize the data collection which serves as the starting point of your new data collection cf. article "Finalize the data collection"
To start the new data collection select the new version in the Data collection section. In this case v3.
Click the button "Go to the participation page":
- Click on ‘Participate’:
- The Start date in the Participation status will be updated and access to the new version is completed and the user can return to the Data Collection section:
- Once the previous data collection is finalized and the new one has been started, you can download data from the first version and add those to the next version.
- Go to the Registrations tab:
- Download the data by using the Download button "CSV Download All (Stable Data)":
When everything is downloaded, you can start uploading by Using the upload center.