ETL Import: Source
The ETL Import: Source configuration category includes the following configurable options:
Async Mode Batch Size
Asynchronous Mode Enabled
Async Mode File Size
Import Source
Mapping Script (Text)
Mapping Script (File)
Worker Row Hashing Function (Text)
Worker Row Hashing Function (File)
Worker Thread Count
Synchronous Mode: Continue On Failure
Enable Transaction Log
|
|
Async Mode Batch Size |
|
|
|
| POSITIVE_INTEGER | |
| Controls the number of rows to process as a single batch. | |
|
|
100
|
|
|
|
|
Asynchronous Mode Enabled |
|
|
|
| BOOLEAN | |
| If enabled, data will be imported asynchronously. See Asynchronous Mode for more information. | |
|
|
false
|
|
|
|
|
Async Mode File Size |
|
|
|
| POSITIVE_INTEGER | |
| Controls the maximum number of lines that will be placed into a single file. Larger values may improve performance, but will require more memory as each file will be loaded into memory while it is being processed. | |
|
|
1000
|
|
|
|
|
Import Source |
|
|
|
| ENUM | |
| Values |
|
| This setting defines the format for files being imported. | |
|
|
CSV
|
|
|
|
|
Mapping Script (Text) |
|
|
|
| JAVASCRIPT | |
This is the script that processes each received row. It must have at least one function, with signature function handleEtlImportRow(inputMap, context). See ETL Importer module for details on how this function works.
|
|
|
|
function handleEtlImportRow(inputMap, context) {
// Mapping goes here
}
|
|
|
|
|
Mapping Script (File) |
|
|
|
| Resource Path | |
This is the filepath to the script that processes each received row. The script must have at least one function, with signature function handleEtlImportRow(inputMap, context). See ETL Importer module for details on how this function works. Values should be prefixed with file: or classpath:.
|
|
|
|
| (no default) | |
|
|
|
Worker Row Hashing Function (Text) |
|
|
|
| JAVASCRIPT | |
| If set, this script is used to provide an appropriate hash for each individual row in order to ensure that rows are processed in an appropriate order. This setting may be left blank if in-order processing of individual rows is not necessary. | |
|
|
| (no default) | |
|
|
|
Worker Row Hashing Function (File) |
|
|
|
| Resource Path | |
| This is the filepath to the script that is used to provide an appropriate hash for each individual row in order to ensure that rows are processed in an appropriate order. This setting may be left blank if in-order processing of individual rows is not necessary. | |
|
|
| (no default) | |
|
|
|
Worker Thread Count |
|
|
|
| POSITIVE_INTEGER | |
This setting configures the number of worker threads that will be allocated to processing rows of the CSV file. If set to 1, the file will be processed in a single-threaded manner.
|
|
|
|
4
|
|
|
|
|
Synchronous Mode: Continue On Failure |
|
|
|
| BOOLEAN | |
| If enabled, in synchronous mode any errors in processing will not abort processing but will instead result in a log line being written and processing will continue. | |
|
|
false
|
|
|
|
|
Enable Transaction Log |
|
|
|
| BOOLEAN | |
| If enabled, FHIR Search/Read/Create/Update/Delete operations executed from within the Javascript environment will generate transaction logs. Note that this can have a significant impact on performance and storage requirements. | |
|
|
false
|
|
|