Upload a CSV file

There are 3 ways to upload CSV files with data into HD4DP v1:

• Manually
  • In de HD4DP application
  • In the specific folders on the server (provisioning)
• Automatically
  • in the specific folders on the server (fasttrack)

The file can be maximum 6 MB. 

When using the manual upload by using te application or putting files on the server (provisioning), the registrations still need to be manually submitted.

When using the fasttrack method, submission is done automatically.

Manually in the HD4DP application

Data can be uploaded by using the same approach as we use for the stable data.

More information can be found in the article "CSV download and upload for stable data".

Manually in the application folders

HD4DP allows to import files from the folders used by the application.

Save the created CSV file in the correct register folder of your organization or sub-organisation. The folders of the sub-organizations are to be find in the subfolders of the main organization.

When uploading a CSV, a check is performed for each record if any corresponding record is already available in the application. The check for corresponding data is based on the unique id. If a record is already available in the application a merge takes place between the existing record and the new CSV. In case of conflicts, the values of the new CSV are implemented.

The uploaded registrations will be automaticallyuploaded in the HD4DP application.

They will need to be manually validated and submitted in the application.

Automatically in the application folders

The goal of the CSV fast track is to automatically transmit complete valid CSV files to healthdata without human intervention. The following topics must be taken into account when using the CSV fast track. HD4DP allows to automatically import files from dedicated folders. On the server where HD4DP is installed, create a new folder next to the provisioning and stable data folders.

After creating the fast track folder, you must configure the location of the dedicated folder in HD4DP by adapting the configuration parameter "FAST_TRACK_DIRECTORY" in the admin configuration panel. 

Subfolders contributing to the organization and data collections, will  automatically be created according to a specific folder structure. 

In this video a more profound explanation about how to set up the fast-track is given:

When a CSV file is placed in the folder of a data collection, several actions can occur: 

• If the file could not be manipulated (e.g. an invalid CSV or a data collection that has not yet started) the file is moved to ARCHIVE_ERROR
• If the file was successfully read but only partially valid, the file will be split accordingly.
  • e.g: line1 is valid, line2 ignored because of duplicate record, line3 contains an error
    • a CSV file containing line1 will be placed in SUCCESS
    • a CSV file containing line2 will be placed in IGNORE
    • a CSV file containing line3 will be placed in ERROR and an extra column is added which explains why the CSVs raised an error.
    • the original CSV will be moved to ARCHIVE
• If the file is successfully read and validated, it is moved entirely to both the SUCCESS folder and the ARCHIVE folder.