The FHIR delete operation performs a "logical" delete. This means that data is not physically removed from the database.
For example, suppose a Patient resource with ID 123 is created (via an HTTP POST /Patient and subsequently deleted (via an HTTP DELETE Patient/123). This will cause a second version of the Patient/123 resource to be created with version Patient/123/_history/2 that is marked as deleted.
This patient will no longer appear in search results, and attempts to read the resource (using an HTTP GET Patient/123) will fail with an "HTTP 410 Gone" response.
The original content of the resource is not destroyed however. It can still be found using two FHIR operations:
GET Patient/123/_history/1GET Patient/123/_historyNote that HTTP 410 Gone responses will include a Location header containing the fully qualified resource ID as well as the version ID. For example:
410 Gone
Location: http://example.org/fhir/Patient/123/_history/12
In this example, we can see that the deleted version of the resource is version 12. This means that the last non-deleted version is version 11, and this could be accessed using a version-specific read to the following URL:
http://example.org/fhir/Patient/123/_history/11
A common problem – frequently in test systems but sometimes in production systems, too – is cleaning up batches of interdependent data.
For example, suppose you have a CDR containing a patient resource with the ID Patient/A. Suppose this CDR also contains Encounter/1 and Encounter/2, as well as Observation/3 and MedicationAdministration/4, and all of these resources have a reference to the resource Patient/A. We will call these resources the child resources.
If you try to DELETE Patient/A (using a standard FHIR DELETE operation), this request will be denied, assuming that there is a resource link between the child resources and the patient (a resource link is a field that is indexed with an active SearchParameter).
This will result in an HTTP 409 Conflict with a response like the following:
{
"resourceType": "OperationOutcome",
"issue": [
{
"severity": "error",
"code": "processing",
"diagnostics": "Unable to delete Patient/1 because at least one resource has a reference to this resource. First reference found was resource Observation/2 in path Observation.subject.where(resolve() is Patient)"
}
]
}
If you want to force a delete of Patient/A, you have several options:
You can manually delete all of the child resources before trying to delete the Patient.
You can disable referential integrity checking in the CDR by changing the Enforce Referential Integrity on Delete setting on the FHIR Storage module. This means that any delete will be permitted as long as the user has appropriate permissions to actually perform the delete, even if other resources still have references left. This can be confusing for other users, since it leaves resources with invalid references, but there are valid cases for doing so.
You can use a transactional delete
You can use cascading deletes
The FHIR Transaction operation can be used to delete multiple resources at the same time.
This is useful if you have chains or collections of resources to delete at ones, but also can be used to delete circular references.
To delete multiple resources in a transaction, POST a Bundle such as the following to the root of your FHIR endpoint.
{
"resourceType": "Bundle",
"type": "transaction",
"entry": [
{
"request": {
"method": "DELETE",
"url": "Organization/1"
}
},
{
"request": {
"method": "DELETE",
"url": "Organization/2"
}
}
]
}
By default, Smile CDR will block the deletion of a resource if any other resources have indexed references to the resource being deleted.
For example, suppose the resource Patient/123 has been saved in the repository, and a second resource Observation/456 is then saved as well, where the Observation.subject reference is a reference to the Patient.
In this situation, attempts to delete Patient/123 will be blocked unless resources with references to this resource are deleted first (or are deleted as a part of the same transaction in the case of a transactional delete).
You can disable this referential integrity checking globally using the Enforce Referential Integrity on Write configuration property on the FHIR Storage module.
You can also selectively disable referential integrity checking by configuring the system to disable referential integrity checking for specific paths. This is done using the Enforce Referential Integrity on Write.
The value for this setting is a set of one or more FHIRPath expressions, each one on a new line. For example, this setting could be set to Observation.subject in order to allow the deletion described above to proceed.
With cascading deletes enabled, a user can perform the delete on Patient/A (per the example above) and all of the child resources will be deleted as well.
In order to perform a cascading delete, three things must occur:
First, the Cascading Deletes Enabled setting must be enabled on the FHIR Storage module.
Then, the user performing the operation must have the FHIR_DELETE_CASCADE_ALLOWED permission, as well as a specific permission allowing the child resource to be deleted. For example, you might grant the user the FHIR_DELETE_CASCADE_ALLOWED and FHIR_ALL_DELETE permissions.
Finally, to perform a cascaded delete, the client HTTP request must include either a special URL parameter (_cascade) or a special header to indicate that a cascading delete is desired.
The following example shows a delete using a URL parameter:
DELETE /Patient/123?_cascade=delete
The following example shows a delete using an HTTP header:
DELETE /Patient/123
X-Cascade: delete
Note that depending on the number of child resources linked to a given resource, the cascading delete can potentially take several seconds or even minutes to complete and consume considerable memory and processing resources during that time, which may have unexpected impacts on Smile CDR. To minimize the duration and impact of cascading deletes, the actual deletes are performed in batches of a pre-configured size. If after completing 10 passes of batched deletes, there are still child resources remaining, the deletes will be rolled back and Smile CDR will return an error similar to:
Requested delete operation stopped before all conflicts were handled. May need to increase the configured Maximum Delete Conflict Query Count.
If this type of error occurs after attempting a Cascading Delete request, the batch size can be increased by increasing the configured value of Delete Child Resource Count.
In some cases, it is desirable to truly delete data. This might be because it was truly entered in error and should not be seen, because it represents a privacy concern to leave it in place, or because your solution does not require long term retention of stale data.
The $expunge operation is a powerful operation that can physically delete old versions of resources, deleted resources, or even all data in the database.
Note that this operation is globally disabled by default as it is potentially dangerous. Change the Expunge Operation Enabled setting to enable it.
| Name | Type | Usage | Default | Permission Required |
|---|---|---|---|---|
limit |
Number | This parameter specifies the maximum number of entries (resource versions and/or resources) that will be deleted in a single batch before exiting. | 1000 | N/A |
expungeDeletedResources |
Token (boolean value) |
If set to true, deleted resources will be
expunged (including all previous versions of the resource).
|
false | FHIR_EXPUNGE_DELETED |
expungePreviousVersions |
Token (boolean value) |
If set to true, non-current versions of
resources will be expunged.
|
false | FHIR_EXPUNGE_PREVIOUS_VERSIONS |
expungeEverything |
Token (boolean value) |
If set to true, current versions of
resources will also be expunged.
|
false | FHIR_EXPUNGE_EVERYTHING |
The $expunge operation can be invoked against a single resource instance, or even an individual version of a resource instance. If invoked at the instance level (shown below), previous versions of the resource may be deleted (if expungePreviousVersions is set to true) and the current version may be deleted (if the resource is deleted and expungeDeletedResources is set to true).
POST [base]/Patient/123/$expunge
Content-Type: application/fhir+json
{
"resourceType": "Parameters",
"parameter": [
{
"name": "limit",
"valueInteger": 1000
},{
"name": "expungeDeletedResources",
"valueBoolean": true
},{
"name": "expungePreviousVersions",
"valueBoolean": true
}
]
}
The $expunge operation can also be invoked at the instance version level (shown below). This can be used to expunge an individual version of a resource without affecting other versions.
POST [base]/Patient/123/_history/2/$expunge
Content-Type: application/fhir+json
{
"resourceType": "Parameters",
"parameter": [
{
"name": "expungeDeletedResources",
"valueBoolean": true
}
]
}
The $expunge operation can be invoked at the type level. In this mode, all resources of a given type will be processed with the same rules as at the instance level.
POST [base]/Patient/$expunge
Content-Type: application/fhir+json
{
"resourceType": "Parameters",
"parameter": [
{
"name": "expungeDeletedResources",
"valueBoolean": true
},{
"name": "expungePreviousVersions",
"valueBoolean": true
}
]
}
The $expunge operation can be invoked at the system level. In this mode, all resources on the server will be processed with the same rules as at the instance level.
POST [base]/$expunge
Content-Type: application/fhir+json
{
"resourceType": "Parameters",
"parameter": [
{
"name": "expungeDeletedResources",
"valueBoolean": true
},{
"name": "expungePreviousVersions",
"valueBoolean": true
}
]
}
The following operation will delete all data, including non-deleted resources.
In order to delete all data, two settings need to be enabled on the Storage Module:
POST [base]/$expunge
Content-Type: application/fhir+json
{
"resourceType": "Parameters",
"parameter": [
{
"name": "expungeEverything",
"valueBoolean": true
}
]
}
Note that this activity will also remove all default search parameters. In order to reload these default search parameters, simply restart Smile CDR after this operation and they will be loaded automatically.
If you need to quickly delete all data associated with a set of resources, for example in a test environment, you can combine the DELETE and $expunge operations into a single step. Smile CDR offers two ways to do this. You can either call a usual DELETE with a special parameter _expunge=true or you can POST a $delete-expunge operation. Both of these ways result in starting a Delete Expunge Batch Job that deletes and expunges the requested details in the background.
In order to perform a Delete Expunge, three settings need to be enabled on the Storage Module:
Note that unlike the $expunge operation, the Delete Expunge operations can only be invoked at the type level. If there is a need to delete and expunge a single resource, this can be accomplished using the _id=search parameter.
A user must have both FHIR_DELETE_ALL_OF_TYPE permission and FHIR_EXPUNGE_EVERYTHING permission to DELETE with _expunge=true. See Delete Expunge Parameters below for possible parameters on this operation.
Here are examples of performing delete expunge using the DELETE method:
DELETE [base]/Observation?status=cancelled&_expunge=true
DELETE [base]/Patient?_id=123&_expunge=true
A user must have FHIR_DELETE_EXPUNGE permission to call this operation. See Delete Expunge Parameters below for possible parameters on this operation.
Here is an example of performing delete expunge with a POST to /$delete-expunge:
POST [base]/$delete-expunge
Content-Type: application/fhir+json
{
"resourceType": "Parameters",
"parameter": [ {
"name": "url",
"valueString": "Observation?subject.active=false"
}, {
"name": "url",
"valueString": "Patient?active=false"
}, {
"name": "batchSize",
"valueInteger": 1000
}, {
"name": "cascade",
"valueBoolean": false
} ]
}
In the above example, the server starts a batch job to delete and expunge all matching resources in batches of 1000 resources at a time. The server deletes the urls in order. In this example, the server would first delete and expunge all matching Observation resources and then delete and expunge all matching Patient resources. The newest resources are deleted and expunged first. The batch job tracks incremental progress, and can be safely restarted if the server is stopped before the operation completes.
| DELETE URL Parameters | $delete-expunge Parameters | Cardinality | Type | Usage | Default |
|---|---|---|---|---|---|
| Any search parameters on the HTTP request URL are applied to the deletion search. | url |
1..* | string |
A search URL that will be searched. Any matching results will be
deleted and expunged. Values must be in the form
[resourceType]?[parameters].
|
|
_cascade |
cascade |
0..1 | boolean |
If set to false (which is the default), when a resource is
being deleted, an error will occur if other resources contain indexed references
to the resource being deleted. If set to true, those other
resources will be deleted as well.
|
false |
_maxRounds |
cascadeMaxRounds |
0..1 | integer |
If set, specifies the maximum number of cascade rounds that will be applied.
If you only expect to cascade one level deep, setting this to 1
avoids accidentally cascading farther than anticipated. This should be set to
the lowest number possible.
|
20000 |
| N/A | batchSize |
0..1 | integer | The number of resources that will be deleted in a single batch. This number must not be too large or memory issues may occur. In most cases the default is good enough. | 20000 |
The DELETE _expunge=true and $delete-expunge operations create the same type of batch job. Batch jobs are stopped and restarted on the Batch Job Management page.
The $delete-expunge operation is partition aware. The operation is performed only on the partition that was included in the request and the job is only started if the user is allowed to access that partition.
The delete expunge batch job is optimized to delete the resource records as quickly as possible. This means usual checks (authorization callbacks, logging, auditing etc.) are skipped when performing batch delete expunges. If you require normal checks, then you should use the normal DELETE followed by $expunge operation.
The only check that is made before deleting the resources is referential integrity: resources are not delete expunged if there are other resources that refer to them. The urls to delete expunge need to be ordered so that the child resources are removed before the parent resources are removed.
The resources are removed in batches, with all database records associated to those resources deleted in a single transaction. The batch size indicates the number of resources to delete together in single transaction. So for example if the batch size is set to 100 and there are 15 records per resource, that results in removing 1500 records in a single transaction. Larger batch sizes performs faster but require more memory.
If the DELETE ?_expunge=true syntax is used to trigger the delete expunge, then the batch size is determined by the value of Expunge Batch Size) property.