Smile CDR v2024.08.PRE

On this page:

46.38 ETL Import: CSV Properties 46.40 FHIR Binary Storage

46.39.1ETL Import: Source

The ETL Import: Source configuration category includes the following configurable options:

Async Mode Batch Size
Asynchronous Mode Enabled
Async Mode File Size
Import Source
Mapping Script (Text)
Mapping Script (File)
Worker Row Hashing Function (Text)
Worker Row Hashing Function (File)
Worker Thread Count
Synchronous Mode: Continue On Failure
Enable Transaction Log

46.39.2Property: Async Mode Batch Size

Property Name	Async Mode Batch Size
Property Key
Property Type	POSITIVE_INTEGER
Description	Controls the number of rows to process as a single batch.
Applies to Modules	ETL Importer
Default Value	`100`
Example Property	`module.[MODULE_ID].config.asynchronous_mode.batch_size = 100`

46.39.3Property: Asynchronous Mode Enabled

Property Name	Asynchronous Mode Enabled
Property Key
Property Type	BOOLEAN
Description	If enabled, data will be imported asynchronously. See Asynchronous Mode for more information.
Applies to Modules	ETL Importer
Default Value	`false`
Example Property	`module.[MODULE_ID].config.asynchronous_mode.enabled = false`

46.39.4Property: Async Mode File Size

Property Name	Async Mode File Size
Property Key
Property Type	POSITIVE_INTEGER
Description	Controls the maximum number of lines that will be placed into a single file. Larger values may improve performance, but will require more memory as each file will be loaded into memory while it is being processed.
Applies to Modules	ETL Importer
Default Value	`1000`
Example Property	`module.[MODULE_ID].config.asynchronous_mode.file_size = 1000`

46.39.5Property: Import Source

Property Name	Import Source
Property Key
Property Type	ENUM
Values	`CSV`
Description	This setting defines the format for files being imported.
Applies to Modules	ETL Importer
Default Value	`CSV`
Example Property	`module.[MODULE_ID].config.import_source = CSV`

46.39.6Property: Mapping Script (Text)

Property Name	Mapping Script (Text)
Property Key
Property Type	JAVASCRIPT
Description	This is the script that processes each received row. It must have at least one function, with signature `function handleEtlImportRow(inputMap, context)`. See ETL Importer module for details on how this function works.
Applies to Modules	ETL Importer
Default Value	`function handleEtlImportRow(inputMap, context) { // Mapping goes here }`
Example Property	`module.[MODULE_ID].config.mapping.script = function handleEtlImportRow(inputMap, context) { // Mapping goes here }`

46.39.7Property: Mapping Script (File)

Property Name	Mapping Script (File)
Property Key
Property Type	Resource Path
Description	This is the filepath to the script that processes each received row. The script must have at least one function, with signature `function handleEtlImportRow(inputMap, context)`. See ETL Importer module for details on how this function works. Values should be prefixed with `file:` or `classpath:`.
Applies to Modules	ETL Importer
Default Value	(no default)
Example Property	`module.[MODULE_ID].config.mapping.script.file =`

46.39.8Property: Worker Row Hashing Function (Text)

Property Name	Worker Row Hashing Function (Text)
Property Key
Property Type	JAVASCRIPT
Description	If set, this script is used to provide an appropriate hash for each individual row in order to ensure that rows are processed in an appropriate order. This setting may be left blank if in-order processing of individual rows is not necessary.
Applies to Modules	ETL Importer
Default Value	(no default)
Example Property	`module.[MODULE_ID].config.parallelism.hashing.script =`

46.39.9Property: Worker Row Hashing Function (File)

Property Name	Worker Row Hashing Function (File)
Property Key
Property Type	Resource Path
Description	This is the filepath to the script that is used to provide an appropriate hash for each individual row in order to ensure that rows are processed in an appropriate order. This setting may be left blank if in-order processing of individual rows is not necessary.
Applies to Modules	ETL Importer
Default Value	(no default)
Example Property	`module.[MODULE_ID].config.parallelism.hashing.script.file =`

46.39.10Property: Worker Thread Count

Property Name	Worker Thread Count
Property Key
Property Type	POSITIVE_INTEGER
Description	This setting configures the number of worker threads that will be allocated to processing rows of the CSV file. If set to `1`, the file will be processed in a single-threaded manner.
Applies to Modules	ETL Importer
Default Value	`4`
Example Property	`module.[MODULE_ID].config.parallelism.threadcount = 4`

46.39.11Property: Synchronous Mode: Continue On Failure

Property Name	Synchronous Mode: Continue On Failure
Property Key
Property Type	BOOLEAN
Description	If enabled, in synchronous mode any errors in processing will not abort processing but will instead result in a log line being written and processing will continue.
Applies to Modules	ETL Importer
Default Value	`false`
Example Property	`module.[MODULE_ID].config.synchronous_mode.continue_on_failure = false`

46.39.12Property: Enable Transaction Log

Property Name	Enable Transaction Log
Property Key
Property Type	BOOLEAN
Description	If enabled, FHIR Search/Read/Create/Update/Delete operations executed from within the Javascript environment will generate transaction logs. Note that this can have a significant impact on performance and storage requirements.
Applies to Modules	ETL Importer
Default Value	`false`
Example Property	`module.[MODULE_ID].config.transaction_log.enabled = false`

46.38 ETL Import: CSV Properties 46.40 FHIR Binary Storage