Introduction

This document is written for IT staff / system engineers of data providers and therefore assumes technical knowledge. It acts as a guide through the on-boarding process of HD4DP v2 and covers installation of the server, user configuration, network configuration and remote access.

The order of steps in this document should be respected during execution.

Overview

HD4DP v2 consists of a modular application stack, which allows healthdata.be to seamlessly upgrade individual elements.

An HD4DP v2 deployment comprises of following components:

• Form.io component
• MongoDB
• PostgreSQL
• Nextgen Connect

As it is the case in HD4DP 1.0, an Encryption Module with a connection to the eHealthBox is still required for HD4DP v2 and must be provided by the data provider.

Network configuration

IP

The HD4DP server needs to be accessible via domain names in DNS, and must have a static IP in your private network.

DNS

The application stack of HD4DP v2 requires four domain names pointing to the IP of the locally installed HD4DP v2 server. Use the following names in your DNS:

• nextgenconnect.hd4dp.<yourdomain.be>
• hd4dp.<yourdomain.be>
• metabase.hd4dp.<yourdomain.be>
• admin.hd4dp.<yourdomain.be>

Firewall

The following connections should be possible in the firewall flow:

• To and from (a) machine(s) in your IT department on port 22 for initial configuration and local support.
• To and from the Encryption Module server. The protocol and ports depend on your local EM implementation. Contact your EM vendor if more information is necessary.
• Reachable by your staff who uses HD4DP, on ports 80 and 443 for HTTP(s) traffic.
• To and from the LDAP server (this is not mandatory if you are not using LDAP to authenticate) (port 389 by default)

The healthdata.be proxy server is used as a gateway to the internet for the security of HD4DP servers. The configuration of this proxy server will be provided to you by healthdata.be at a later date.

Server installation

To install the application stack of HD4DP v2, healthdata.be requires a fresh installed operating system, specifically Ubuntu Server 18.04 LTS.

Please use these instructions even if you have previous experience with installing this operating system, as its configuration is specific for healthdata.be.

These instructions assume that the network configuration described in the previous section is completed.

Instructions

HD4DP v2 requires a (virtual) machine running Ubuntu Server 18.04 LTS.

We assume knowledge of loading a .iso file onto a (virtual) machine. Healthdata.be can’t provide instructions for this, as the environment of your center is unknown. Should you have any trouble, however, please contact Healthdata.be support so that we can help out.

Please find the installation steps below.

Installation steps

1. Download the .iso file from the link below.
   Download Ubuntu Server 18.04 LTS
2. Create a new (virtual) machine with Linux Ubuntu 64 bit as the OS family
3. When prompted, select the .iso file downloaded in step 1.
4. After some time, you will be prompted to select a system language. Select English.
5. “Keyboard configuration”
   Select your preferred keyboard layout and press enter
6. “Network Connections”
   Highlight the network interface and press enter. Navigate as follows:
   Edit IPv4 -> Manual -> enter the network details -> save -> Done
7. Proxy IP -> Leave default/empty.
8. “Configure Ubuntu Archive Mirror” -> leave default
9. “File system Setup” -> Use An Entire Disk
10. Proceed until “Confirm destructive action” -> press continue. The installation process starts, this can take several minutes.
11. In the meantime, create the user for Healthdata.
    username = healthdata,
    Password = choose a secure password and communicate it to healthdata.be.
12. Mark “Install OpenSSH server”. This will be used for remote access. “Import SSH Identity” -> no -> done
13. “Featured Server Snaps” -> Select nothing and press Done.
14. Wait until installation is finished.

Configuration steps

Connecting to the server

Log into the machine with the healthdata.be user created in the previous section.

Instructions (from a Windows machine):

1. Install the tool Putty and open the application.
2. On the configuration screen, enter the following (replace cursive text with the appropriate values)
   • Host Name: healthdata@server_private_ip
   • Port: 22
   • Connection type: SSH
3. Click Open. Enter the password (you will not see text as you type, you can paste into putty by right-clicking in the terminal).
4. You should now be logged in and see a prompt  “healthdata@server_name:~$”

Administrator account for internal use

An administrator account for internal use can be created on the HD4DP v2 server.

The configuration of remote access (described below) should not happen on this account, but on the Healthdata.be account.

The internal account can later be used to install and configure OS monitoring software and antivirus software by the internal IT team. For more information, see the section on Antivirus and Monitoring.

(Text with a gray background should be entered as a command in the terminal of the server)

Create the user:

            sudo adduser <username>

Add the user to the sudo group

            sudo usermod -aG sudo <username>

Installation and configuration of the software stack

Healthdata.be support will instruct you when to execute the next step, which is to enable remote access so that Healthdata.be can execute the software installation and configuration.

Backups

The configuration of the HD4DP v2 server is administered by healthdata.be and does not require backups.

HD4DP v2 regularly dumps its databases automatically to the /backup directory on the server. A network storage should be mounted at this location.

Please fill out the infrastructure sheet with the required credentials, domain name/url, protocol… to connect to the network drive. The connection will then be configured by healthdata.be.

Patching and Updates

Healthdata.be configures HD4DP v2 servers to automatically receive recommended security updates. The choice for Ubuntu 18.04 is motivated by the long-term support for this version. Security flaws are rare in this distribution, and security updates are quick and often don’t require a system reboot.

If the IT department of your organization prefers to manage patches, this is possible but not encouraged. Please use the account for internal use created in Section 3 for this purpose.

Antivirus and Monitoring

Most data providers will want to manage their own antivirus and OS monitoring on all machines in their network. Installation of such software on the HD4DP v2 server is allowed, but healthdata.be should be informed about all extra software installed on the server. Additionally, healthdata.be will not provide support for the installation of this software.

Contact information

The HD4DP v2 Infrastructure Sheet contains information that healthdata.be needs in order to start the insallation of the HD4DP 2.0 Software at your organization.

Below you can find the description of the necessary information:

SERVER CONNECTION

Healthdata.be performs its installation and support tasks remotely (using VPN or remote port forwarding via SSH). Please provide the required credentials.

• Type of connection (VPN / Remote port forwarding via SSH)
• Link (IF VPN)
• Username, token, other (if VPN)
• Password (if VPN)³
• Public SSH Key (if remote port forwarding)

³ For security reasons, we advise to communicate passwords to us either by phone, or via a link using a secret-sharing service such as onetimesecret.com.

SERVER MACHINE

• Server Name
• Internal IP-Address
• Ram (in GB)
• CPU (number of CPU's and number of cores)
• Disk space (in GB)
• Username: Healthdata
• Password ³

³ For security reasons, we advise to communicate passwords to us either by phone, or via a link using a secret-sharing service such as onetimesecret.com.

ATTACHED DRIVE FOR BACKUPS

HD4DP 2.0 regularly performs data dumps for backup purposes. Please provide connection information to a network share volume.

• Link / IP address
• Path
• Username
• Password ³

³ For security reasons, we advise to communicate passwords to us either by phone, or via a link using a secret-sharing service such as onetimesecret.com.

USER MANAGEMENT

HD4DP can either connect to a LDAP server or use its own application database for performing user authentication and management. Please check the user management mechanism you want to use.

• LDAP user management : Yes / No
• Application user management : Yes / No

LDAP configuration (Optional)

If you chose ‘LDAP user management’ as user management mechanism, please provide the following information that allows us to connect to your LDAP system.

• Connection URL
• Username
• Password³

³ For security reasons, we advise to communicate passwords to us either by phone, or via a link using a secret-sharing service such as onetimesecret.com.

SOFTWARE CONFIGURATION

Encryption Module interface

HD4DP communicates with the Encryption Module (EM) either using the file system interface or by calling a REST web service. Please choose which interface HD4DP should use for its communication with the Encryption Module.

Note: if the encryption module is not yet purchased (or developed), HD4DP can already be installed; the EM can then be configured in HD4DP once it is available. Please note that HD4DP 1.x and HD4DP 2.0 cannot use the same EM.

• REST web service
• File system

REST web service interface

If you chose to communicate with the Encryption Module using a REST interface, please provide the web service URLs that should be used by HD4DP for its communication with EM.

• "Outgoing flow URL: Example: http://host:8080/encryptionmodule/send"
• "Incoming flow URL : Example: http://host:8080/encryptionmodule/receive"

File system interface

• "Incoming directory: Directory where HD4DP checks for incoming files"
• "Incoming directory: Directory where HD4DP writes outgoing files"
• "Incoming directory: Directory to which HD4DP moves successfully processed files"
• "Incoming directory: Directory to which HD4DP moves unsuccessfully processed files"

This documentation details the necessary adaptations to be performed in order to allow the necessary technical accesses and smooth operation of the different healthdata.be platforms and interfaces.

The file is available for download below.

Documentation for System 2 System API on Architecture 2.0

Description of the service

API is the acronym for Application Programming Interface, which is a software intermediary that allows two applications to talk to each other.

In this case, the API is used to have the system of the Data Providers communicate with the system of HealthData.

The S2S API is a unified API that will allow clients (Data Providers) to submit DCDs data to HD4DP2.0 in fully automated way.

End point information (per DCD) + examples

HOW TO: Upload data using System 2 System

Steps To Upload data

1. IT services of data providers must setup their systems to be able to communicate with HD4DP v2 System 2 System API

A prerequisite to be able to use the Health Data's System 2 System API, is that the IT services of the hospitals must have the following in place before the systems can communicate:

• The endpoint/URL is protected by credentials for which the support services need to be contacted.
• End-to-end API process to submit DCD registrations in a fully automated manner.
• Support for searching submitted DCD registrations.

2. Prepare the JSON file (example file in this section)

To send DCD registrations to Health Data, the file must be in a .json file format.

• At the Data Provider's side, we must foresee a way to extract the JSON file from the electronic patient files and/or other local databases.

• Author group, Author and Coauthor:
  • When the Author group, Author and Coauthor has been left out in the json file, the default Author group, Author and Coauthor will be used automatically.
  • When the desired Author group, Author and Coauthor are specified in the json file, the following fields TX_AUTHOR_GR, TX_AUTHOR and TX_COAUTHOR must be added to the json file with their values respectively.
    
    Example:

TX_AUTHOR_GR;TX_AUTHOR;TX_COAUTHOR
Test group;test@sciensano.be;test@sciensano.be

Note:
The Author group, Author and Coauthor must exist and are well configured at the back-end of the system. TX_AUTHOR_GR can be a string that identifies the Author group to which this Author belongs. Commonly, the first name and last name are used to identify the TX_AUTHOR_GROUP. Be sure to avoid leading and trailing spaces when entering the Author group value.

• Similarly, to submit a draft, the only thing to add is the key STATUS (all upper case) with the value "draft" to the request, as described in Submission of drafts.

• Adding separators to a NISS number:
  It is not necessary to necessary to add separators in a NISS number when uploading a file using S2S API. You can fill the NISS number out both with or without seperators. E.g.: 85.04.02-169.32 or 85040216932.
  
  Example:

• Make sure the name of the JSON file has the correct format:
  HD_DCD_submjson_HDBPnumber_HDBPabbreviation_versionnumber_versionreleasedate

So for NSIH SSI the format would be:
HD_DCD_submjson_HDBP0032_

EXAMPLES:

• Please find below an example file for NSIH SSI:
  • HD_DCD_submjson_HDBP00

Disclaimer: The example files above are only provided as a guideline and do not contain real life data.

3. Uploading the JSON File

• The IT department of the Data Provider must provide a manner in their system to send the API requests containing DCD registrations in a JSON file format, the correct end-points must be addressed.

• In this case the end-point to upload the json file will be:

4. Validate the JSON Upload

4.1 Validation of the S2S API Upload via the response:

Verify in the same way the request was sent, that the returned response is containing a valid Business key.

If a valid Business key has been provided, the registration upload via System 2 System API was succesful.

4.2 Validation of the System 2 System API via HD4DP 2.0:

Step 1: Open the web application HD4DP 2.0.

Step 2: Select the concerned organization in the dropdown list and click on Volgende (Next)

Step 3: Fill in the username and password, that has been provided by your IT Department or Healthdata team, and click on Log in to access the HD4DP 2.0 application.

Step 4: Navigate in the menu on the left-hand side panel to the desired study program:

<Screenshot of the menu item on the left-hand side panel>

Step 5: Check that the uploaded registration(s) is/are displayed in the overview table:

<Screenshot of registration on right-hand side panel>

Documentation for CSV Upload on Architecture 2.0

Description of the service

The CSV Upload functionality gives the possibility to import multiple parameters from a set of patients in one time into HD4DP 2.0. The csv file is based on an extract of the electronic patient files and/or other local databases.

Currently there is no user interface in HD4DP 2.0 for uploading csv files. If a data provider wants to upload a csv file, the file has to be dropped at a specific location. These files will be picked up and processed periodically. The files should be “final”, meaning that no application is writing to them. The pick-up location will be identical for all registries.
Pre-registry handling will be based on a naming-convention of the csv file.

HOW TO: Upload data using CSV Upload

Steps To Upload data

1. Prepare the csv file (example file in this section)

• Extract the csv file from the electronic patient files and/or other local databases.

• Author group, Author and Coauthor:
  • When the Author Group, Author and Coauthor have been left out in the csv file, the default Author group, Author and Coauthor will be used automatically.
  • When the desired Author Group, Author and Coauthor are specified in the csv file, the following headers TX_AUTHOR_GR, TX_AUTHOR and TX_COAUTHOR must be added to the csv file with their values respectively.
    
    Example:

TX_AUTHOR_GR;TX_AUTHOR;TX_COAUTHOR
Test group;test@sciensano.be;test@sciensano.be

Note:
The Author group, Author and Coauthor must exist and are well configured at the back-end of the system. TX_AUTHOR_GR can be a string that identifies the Author group to which this Author belongs. Commonly, the first name and last name are used to identify the TX_AUTHOR_GROUP. Be sure to avoid leading and trailing spaces when entering the Author group value.

• Similarly, add the field name 'STATUS' in capitals in an additional column. Add the value 'draft' in case a manual submission of the record is requested.
  If not, the record will be submitted without manual intervention.

• Adding separators to a NISS number:
  It is not necessary to add separators in a NISS number when uploading a file using CSV Upload. You can fill out the NISS number both with or without separators. E.g.: 85.04.02-169.32 or 85040216932.
  
  Example:

• Make sure the name of the csv file has the correct format:
  HD_DCD_submcsv_HDBPnumber_HDBPabbreviation_versionnumber_versionreleasedate

So for NRC Influenza the format would be:
HD_DCD_submcsv_HDBP0333_

EXAMPLES:

• Please find below an example file for NRC Influenza:
  • HD_DCD_submcsv_HDBP0333_

Disclaimer: The example files above are only provided as a guideline and do not contain real life data.

2. Uploading the csv file

Step 1: Open the sftp tool like WinSCP

Step 2: Get the credentials (Host name, Port number, User name and Password) from the IT department of the Data Provider, to log on to the sftp server located on the Data Provider side. See section Data transfer via an SFTP client in the documentation of the Online Acceptance Environment.

Step 3: Fill in the credentials into the Login screen and click on Login to be able to access the different upload folders:

Note: a warning might be given, just click on Update

Now the CSV Upload folder structure is displayed on the right-hand side panel. You will notice draftcsv folders and submcsv folders for the diverse HD business projects. Historically, a directly submitted csv file would end up in the submitted folder, whereas a csv file featuring ‘draft’ in the status field would have ended up in the draft folder. At this stage there is no real distinction between both folders, so you can save the csv file in either of them.
Attention: If you wish to create a draft to work on at a later stage, don’t forget to add ‘draft’ in the status column of the csv file.

Step 4: Select the project folder <project folder name> and open it by double-clicking on it:

<Screenshot of Project with DCD folders>

Step 5: Double-click on the DCD folder to open it:

<Screenshot of DCD folders>

Step 6: Now go to the folder on the left-hand side panel where the csv file to be uploaded is located:

<Screenshot of DCD folders>

Step 7: Drag the csv file to be uploaded from the left-hand side panel into the folder on the right-hand side panel:

<Screenshot of file on right-hand side panel>

Step 8: Wait until the polling system of the CSV Uploader has picked up the CSV file and has processed it.
Once the csv file has been processed it will disappear from the folder, when we refresh the page manually!

<Screenshot of DCD folder where the file has disappeared after a refresh>

3. Validate csv upload

Once the csv file has been processed 3 folders will be created (if they haven't been created already) in the DCD folder:

• ARCHIVE (after a csv file has been processed, the original csv file will be saved in this folder)
• RESULT (when the csv file is placed in this folder, it means that the csv file has been processed, a file will be created or (or appended, if the file already existed) with the result of the upload of the csv file).
  All the errors that are described in this file are business related, which means that they are technically correct, but in violation with the business rules or contain wrong values for that field.
• ERROR (when the csv file is placed in this folder, it means that a technical error has occurred like the csv file contained erroneous formatting. The csv file won't get processed and an error file will be created with the errors and reason why the csv file couldn't be processed)

3.1 Validation of the csv upload via sftp tool:

Step 1: Double-click on the ERROR folder to open it, click on the refresh button and verify that there is no error file present.

Step 2: Return to the DCD folder. Now double-click on the RESULT folder to open it, click on the refresh button and verify that the result file is present.

Step 3: Double-click on the result file to open it.

<Screenshot of result file of the DCD>

Step 4: If there are multiple records in the result file, scroll to the entry of the current csv upload by looking at the upload date (Started at dd/mm/yyyy hh:mm).
Verify the result file that the upload was successful by searching for the word SUCCESS and having a look at the Status. This Status must contain: Success;Success Count:1;Error Count:0

<Screenshot of result file of the DCD>

3.2 Validation of the csv upload via HD4DP 2.0:

Step 1: Open the web application HD4DP 2.0.

Step 2: Select the concerned organization in the dropdown list and click on Volgende (Next)

Step 3: Fill in the username and password, that has been provided by your IT Department or Healthdata team, and click on Log in to access the HD4DP 2.0 application.

Step 4: Navigate in the menu on the left-hand side panel to the desired study program:

<Screenshot of menu navigation to the DCD>

Step 5: Check that the uploaded registration(s) is/are displayed in the overview table.

<Screenshot of DCD Overview screen>

The person with the login for the local database of "HD4DP v2 local" has access to all the data stored in the database. This means that the personal data of the patients will be VISIBLE to that user.

"data_collection_name" in local database

• "data_collection_name" =

SELECT * from local_dwhmessage WHERE data_collection_name = 'add data_collection_name' and created_on > current_date - interval '15' day;

SELECT * from local_dwhmessage as ldm left join local_dwhmessage_key_value as ldmkv on ldmkv.msg_document_id = ldm.document_id WHERE ldm.data_collection_name = 'add data_collection_name';

SELECT * from local_dwhmessage as ldm left join local_dwhmessage_key_value as ldmkv on ldmkv.msg_document_id = ldm.document_id left join local_dwhmessage_key_value_plus as ldmkvp on ldmkvp.key_value_id = ldmkv.id WHERE ldm.data_collection_name = 'add data_collection_name';

This documentation is being updated regularly. We try to provide as correct, complete and clear as possible information on these pages. Nevertheless, if you see anything in the documentation that is not correct, does not match your experience or requires further clarification, please create a request (type : request for information) via our portal (https://sciensano.service-now.com/sp) or send us an e-mail via support.healthdata@sciensano.be to report this documentation issue. Please, do not forget to mention the URL or web address of the page with the documentation issue. We will then adjust the documentation as soon as possible. Thank you!

De inhoud van deze pagina is alleen beschikbaar in het EN. Selecteer de EN knop om de pagina te openen.

De inhoud van deze pagina is alleen beschikbaar in het EN. Selecteer de EN knop om de pagina te openen.

De inhoud van deze pagina is alleen beschikbaar in het EN. Selecteer de EN knop om de pagina te openen.

De inhoud van deze pagina is alleen beschikbaar in het EN. Selecteer de EN knop om de pagina te openen.

Before starting a new data collection based on stable data of another data collection, you need to finalize the data collection which serves as the starting point of your new data collection cf. article "Finalize the data collection"

To start the new data collection select the new version in the Data collection section. In this case v3. 

Click the button "Go to the participation page":

• Click on ‘Participate’:

• The Start date in the Participation status will be updated and access to the new version is completed and the user can return to the Data Collection section:

• Once the previous data collection is finalized and the new one has been started, you can download data from the first version and add those to the next version. 

• Go to the Registrations tab:

• Download the data by using the Download button "CSV Download All (Stable Data)":

When everything is downloaded, you can start uploading by Using the upload center.