Bulk Data Import

Bulk Data Import is an operation designed to upload large FHIR datasets to the FHIR server. Bulk Data Import works much like Bulk Data Export. Though this operation is not defined in the FHIR specification, they share similar principles.

Import Request

To import FHIR resources into the FHIR Server:

1. Upload the file, containing newline-delimited JSON, where each line is a single resource.

2. Make a post request to the server with a file path specified in the body.

curl --location --request POST 'https://example-develop.edenlab.dev/fhir/$import' \
--header 'Content-Type: application/json' \
--data-raw '{
    "filepath": "/import/file1.ndjson",
    "range": {
    "start": 1032,
    "end": 5000
    }
}'

Headers

• Prefer (string, required): Specifies whether the response is immediate or asynchronous. The header is set to respond-async (based on ).

{"id":"5c41cecf-cf81-434f-9da7-e24e5a99dbc2","name":[{"given":["Brenda"],"family":["Jackson"]}],"gender":"female","birthDate":"1956-10-14T00:00:00.000Z","resourceType":"Patient"}
{"id":"3fabcb98-0995-447d-a03f-314d202b32f4","name":[{"given":["Bram"],"family":["Sandeep"]}],"gender":"male","birthDate":"1994-11-01T00:00:00.000Z","resourceType":"Patient"}
{"id":"945e5c7f-504b-43bd-9562-a2ef82c244b2","name":[{"given":["Sandy"],"family":["Hamlin"]}],"gender":"female","birthDate":"1988-01-24T00:00:00.000Z","resourceType":"Patient"}

ifNoneExist

The ifNoneExist parameter is designed to check whether the resource with a given identifier already exists in the database or not using the FHIR search. If it does, the procedure omits the resource. This way, it is possible to ensure that no duplicates of existing resources are created during the import procedure.

{"ifNoneExist":"identifier=http://www.ohiogastro.com/procedure|2399234","resource":{"code":{"coding":[{"code":"99231","display":"SUBSEQUENT HOSPITAL CARE","system":"http://www.ama-assn.org/go/cpt"}]},"identifier":[{"system":"http://www.ohiogastro.com/procedure","value":"2399234"}],"location":{"reference":"Location?identifier=http://www.ohiogastro.com/location|14"},"meta":{"profile":["http://hl7.org/fhir/us/core/StructureDefinition/us-core-procedure"],"source":"http://www.ohiogastro.com"},"performedDateTime":"2022-03-29","performer":[{"actor":{"reference":"Practitioner?identifier=http://www.ohiogastro.com/practitioner|1104392281"}}],"resourceType":"Procedure","status":"completed","subject":{"reference":"Patient?identifier=http://www.ohiogastro.com/patient|982947"}}}
{"ifNoneExist":"identifier=http://www.ohiogastro.com/procedure|2399235","resource":{"code":{"coding":[{"code":"XRSFO","system":"http://www.ama-assn.org/go/cpt"}]},"identifier":[{"system":"http://www.ohiogastro.com/procedure","value":"2399235"}],"location":{"reference":"Location?identifier=http://www.ohiogastro.com/location|20"},"meta":{"profile":["http://hl7.org/fhir/us/core/StructureDefinition/us-core-procedure"],"source":"http://www.ohiogastro.com"},"performedDateTime":"2022-04-11","performer":[{"actor":{"reference":"Practitioner?identifier=http://www.ohiogastro.com/practitioner|1720160914"}}],"resourceType":"Procedure","status":"completed","subject":{"reference":"Patient?identifier=http://www.ohiogastro.com/patient|982948"}}}

Range

A user is able to start import from a specific line and end import to a specific line. In order to do this, the user needs to add parameter “range“ and put the number of rows to start - range.start and end - range.end

"range": {
    "start": 1032,
    "end": 5000
    }

Conditional References

When processing import, the Kodjin FHIR Server performs the next steps:

• check all references for search URIs
• For search URIs, use the search to locate matching resources
  • if there are no matches or multiple matches, the transaction fails, and an error is returned to log import
  • if there is a single match, the procedure replaces the search URI with a reference to the matching resource

Notes

• The import process supports referential integrity. Therefore, if the input data contains invalid references or if some resources fail to be imported, the state of the FHIR store will not violate referential integrity.

• If some resources fail to be imported, such as due to parsing errors, successfully imported resources aren't rolled back. For example, if 5 of 100 resources fail to import, the remaining 95 resources are imported into the store.

Bulk Data Delete Request

After a bulk data request has been started, a client may send a DELETE request to the URL provided in the Content-Location header to cancel the request.

Endpoint

DELETE [polling content location]

Bulk Data Status Request

After a bulk data request has been started, the client may poll the status URL provided in the Content-Location header.

Endpoint

GET [polling content location]

Responses

Status: 202 Accepted 
 {
    "id": "a91451b9-6678-43c8-beb1-1aa7577301cb", 
    "status": "in-progress", 
    "fileUrl": "https://example.com/fhir/$import", 
    "processed": { 
        "total": 350, 
        "imported": {
            "count": 350
        },
        "errors": {
            "count": 0
        }
    }, 
    "createdAt": "2022-09-23T08:42:21.532662101Z", 
    "updatedAt":"2022-09-23T08:42:28.247404672Z"
 }

Status: 500 Internal Server Error
Content-Type: application/json
{
    "resourceType": "OperationOutcome",
    "id": "1",
    "issue": [
        {
            "severity": "error",
            "code": "processing",
            "details": {
                "text": "An internal timeout has occurred"
            }
        }
    ]
}

Status: 200 OK
Expires: Mon, 22 Jul 2019 23:59:59 GMT
Content-Type: application/json
{
    "id": "a6d1255e-d976-4dd4-a944-1472156e77af",
    "status": "completed",
    "fileUrl": "https://example.com/fhir/$import",
    "processed": {
        "total": 195059,
        "imported": {
            "fileUrl": "https://example.com/fhir/$import",
            "count": 195059
        },
        "errors": {
        "count": 0
        }
    },
    "createdAt": "2022-09-22T11:43:52.666574356Z",
    "updatedAt": "2022-09-22T12:12:18.730391797Z"
}