On this page:
   35.21    ETL Import: CSV Properties 35.23    FHIR Binary Storage   

35.22ETL Import: Source

 

The ETL Import: Source configuration category includes the following configurable options:

  • Async Mode Batch Size

  • Asynchronous Mode Enabled

  • Async Mode File Size

  • Import Source

  • Mapping Script

  • Worker Row Hashing Function

  • Worker Thread Count

  • Synchronous Mode: Continue On Failure

  • Enable Transaction Log

35.22.1Property: Async Mode Batch Size

 
Property Name Async Mode Batch Size
Property Key
Property Type POSITIVE_INTEGER
Description Controls the number of rows to process as a single batch.
Applies to Modules
  • ETL Importer
Default Value 100
Example Property
module.[MODULE_ID].config.asynchronous_mode.batch_size = 100

35.22.2Property: Asynchronous Mode Enabled

 
Property Name Asynchronous Mode Enabled
Property Key
Property Type BOOLEAN
Description If enabled, data will be imported asynchronously. See Asynchronous Mode for more information.
Applies to Modules
  • ETL Importer
Default Value false
Example Property
module.[MODULE_ID].config.asynchronous_mode.enabled = false

35.22.3Property: Async Mode File Size

 
Property Name Async Mode File Size
Property Key
Property Type POSITIVE_INTEGER
Description Controls the maximum number of lines that will be placed into a single file. Larger values may improve performance, but will require more memory as each file will be loaded into memory while it is being processed.
Applies to Modules
  • ETL Importer
Default Value 1000
Example Property
module.[MODULE_ID].config.asynchronous_mode.file_size = 1000

35.22.4Property: Import Source

 
Property Name Import Source
Property Key
Property Type ENUM
Values
  • CSV
Description This setting defines the format for files being imported.
Applies to Modules
  • ETL Importer
Default Value CSV
Example Property
module.[MODULE_ID].config.import_source = CSV

35.22.5Property: Mapping Script

 
Property Name Mapping Script
Property Key
Property Type JAVASCRIPT
Description This is the script that processes each received row. It must have at least one function, with signature function handleEtlImportRow(inputMap, context). See ETL Importer module for details on how this function works.
Applies to Modules
  • ETL Importer
Default Value function handleEtlImportRow(inputMap, context) { // Mapping goes here }
Example Property
module.[MODULE_ID].config.mapping.script = function handleEtlImportRow(inputMap, context) {
   // Mapping goes here
}

35.22.6Property: Worker Row Hashing Function

 
Property Name Worker Row Hashing Function
Property Key
Property Type JAVASCRIPT
Description If set, this script is used to provide an appropriate hash for each individual row in order to ensure that rows are processed in an appropriate order. This setting may be left blank if in-order processing of individual rows is not necessary.
Applies to Modules
  • ETL Importer
Default Value (no default)
Example Property
module.[MODULE_ID].config.parallelism.hashing.script = 

35.22.7Property: Worker Thread Count

 
Property Name Worker Thread Count
Property Key
Property Type POSITIVE_INTEGER
Description This setting configures the number of worker threads that will be allocated to processing rows of the CSV file. If set to 1, the file will be processed in a single-threaded manner.
Applies to Modules
  • ETL Importer
Default Value 4
Example Property
module.[MODULE_ID].config.parallelism.threadcount = 4

35.22.8Property: Synchronous Mode: Continue On Failure

 
Property Name Synchronous Mode: Continue On Failure
Property Key
Property Type BOOLEAN
Description If enabled, in synchronous mode any errors in processing will not abort processing but will instead result in a log line being written and processing will continue.
Applies to Modules
  • ETL Importer
Default Value false
Example Property
module.[MODULE_ID].config.synchronous_mode.continue_on_failure = false

35.22.9Property: Enable Transaction Log

 
Property Name Enable Transaction Log
Property Key
Property Type BOOLEAN
Description If enabled, FHIR Search/Read/Create/Update/Delete operations executed from within the Javascript environment will generate transaction logs. Note that this can have a significant impact on performance and storage requirements.
Applies to Modules
  • ETL Importer
Default Value false
Example Property
module.[MODULE_ID].config.transaction_log.enabled = false
   35.21    ETL Import: CSV Properties 35.23    FHIR Binary Storage