HD4DP v2 Online Acceptance Environment

Last updated: 2024-02-16 11:00

Introduction

To support development and validation of data transfers using S2S API or CSV upload, a central Online Acceptance Environment is available for the IT services or IT partners of data providers. It is meant to replace the locally installed acceptance environments at the side of the data providers. With the Online Acceptance Environment the three types of data transfer can be tested and validated: data transfer via an API platform, via an SFTP client and via manual input in the study form.

In order to keep the acceptance environment light, it is rebuilt once a week (every Saturday), automatically removing all data that have been entered for testing. The data are stored locally and will not be sent to Healthdata.be infrastructure. The testing is limited to the upload to the HD4DP v2 application.

Application URLs and port

The Online Acceptance Environment can be found on https://hd4dp.acceptance.healthdata.be, which is a publically accessible URL.

On the homepage you are requested to select your organization from the drop-down list in order to proceed.

Log in with the credentials you have received upon request. The username is test@sciensano.be for all users.

Since the list of organizations to choose from is limited, you might not find your organization in it. In that case we advise you to request your credentials through our service portal at https://sciensano.service-now.com/sp via the Request something tab and subsequently the Request for Information box.

When requesting an account for this acceptance environment you will receive 3 different types of credentials:
⦁ credentials to log in to the front end of the online acceptance environment
⦁ credentials to use the API (-> authorization tab in Postman)
⦁ credentials for the SFTP server you use to test the CSV Upload

Once logged in, the layout looks very familiar: to the left you will find the navigation panel with all running projects and projects that have passed the user acceptance testing (UAT) phase. Note that the list of projects featuring in our Online Acceptance Environment is not filtered out for the organization you have selected.

The data transfer methods

As mentioned above, the Online Acceptance Environment enables the testing of the uploads for the following three types of data transfer. They are described in order of preference underneath:

Data transfer via an API platform

The data are extracted directly from the EPD systems and sent to HD4DP v2 local using S2S API before they are sent to healthdata.be. This transfer method requires the use of an API development platform, such as Postman (freely available).

The endpoint (URL) to send your payload to for testing is

https://hd4dp.acceptance.healthdata.be/proxy/api/dcd/payload/submit

This endpoint is to be completed with some parameters, such as the ID of an organization, the ID of a dcd, the version number:

https://hd4dp.acceptance.healthdata.be/proxy/api/dcd/payload/submit?organization-id=6&dcd-id=18&Version=1

Click on the Send button to post the payload. A succesful submission is indicated with the status message “202 Accepted”. This can also be checked visually in the front end of the Online Acceptance Environment.

The field Data source in the top selection bar indicates whether the data were transferred via S2S API, CSV Upload or manually with HD4DP.

In the production environment records sent through API are sent directly to the healthdata.be infrastructure (status “Submitted” in the Progress field).

Next to posting payloads (POST) you can also retrieve information (GET). Examples of such "calls":

  • The call https://hd4dp.acceptance.healthdata.be/proxy/api/organization (see below) will return an organization id:
  • The call https://hd4dp.acceptance.healthdata.be/proxy/api/dcd/menu/structure?organization-id=6 (see below) will return the menu structure with all projects your organization is registered for.

More information about the API data transfer can be found at https://docs.healthdata.be/documentation/hd4dp-v2-health-data-data-providers/hd4dp-v2-s2s-api

Data transfer via an SFTP client

The csv upload is the second type of data transfer. The data are transferred to an SFTP server and subsequently picked up by the healthdata.be system. This transfer method requires the use of an SFTP client, such as WinSCP (freely available).

The Login window will look somehow as follows:

Herein, you enter Host name (sftp.acceptance.healthdata.be) and Port number (2220).

Credentials for the SFTP folder are shared together with the Front-end and API credentials as described in the section “Navigate to and access the Online Acceptance Environment”.

When logging in to WinSCP, you will need to navigate to the correct csv folder : csv/<project>/<dcd>. Here you need to drag and drop the csv you want to upload from the left panel to the right panel. The CSV file will now be picked up by the polling system of the CSV Uploader, which checks for new CSV files every minute.

The folders Archive and Result will only be created after the first CSV file has been uploaded for testing.

The Result folder shows a log file containing CSV Upload reports. The status Error Count shows technical errors such as incorrect name, code …

CSV files that were uploaded in Architecure 1 can be reused in Architecture 2. Prerequisite for this is the addition of necessary fields that are typical for Architecture 2, e.g.:

  • Author Group (TX_AUTHOR_GR) with the value "Test group"
  • Author (TX_AUTHOR) with the value "test@sciensano.be"
  • Coauthor (TX_COAUTHOR) with the value "test@sciensano.be"

To further facilitate the process of reusing CSV files a mapping table with old and new CSV names is provided.

Next to adding fields, you can also leave out fields, which is indicated in the log file reports as a warning.

Once again the back-end process can be checked in the study forms on the front-end interface. You want to refresh the window to update to the newest status.

More information about the CSV data transfer can be found at https://docs.healthdata.be/documentation/hd4dp-v2-health-data-data-providers/hd4dp-v2-csv-upload.

Manual input in the study form

The third data transfer method is the form entry, carried out manually. For this you can use a common browser such as Google Chrome. This method can also be used to validate data sent via S2S API or CSV Upload.