Content only available in english

Importer Properties Template

When creating an importer, feel free to copy-paste this into the recordm-importer.properties file to use as a base. Values already filled are either the default or the most probably value (such as the case for internal urls). Properties that are NOT commented out are mandatory.

Secret values (tokens, passwords) belong in a separate recordm-importer-secrets.properties file — see the secrets template below and How to move importer secrets to a secrets file.

## MANDATORY: URL for the RecordM where we want to create/update definitions' instances 
recordm.url=http://localhost:40280

## MANDATORY: URL for the ES server where we want to search
elasticsearch.url=http://localhost:9200

## OPTIONAL: Specifies whether to wait for the created instances to be indexed, default: false 
#es.sync-refresh.on-create = false 

## MANDATORY: recordm.token — goes in recordm-importer-secrets.properties (see the secrets template below).
## It may also be set here, but a value in the secrets file wins over this one.

## OPTIONAL: The format of the date fields
# date.format=yyyy-MM-dd

## OPTIONAL: The format of the date fields with hours in the .xlsx files
#datetime.format=yyyy-MM-dd HH:mm:ss

##OPTIONAL: Maximum capacity of the Camel SEDA queue (i.e., the number of messages it can hold)
#camel.queueSize=100

##OPTIONAL: Camel number of concurrent threads processing
#camel.concurrentConsumers=1

##OPTIONAL: the http port where the importer should listen, defaults to 40285 - cannot have more than one importer listening on the same port
#camel.http.port=40285

##OPTIONAL: Camel option to only read a file when it has read permission.
#It prevents processing files that are still being modified.
#(e.g.: when copying a file to the import directory, the import will only take place after the file is fully copied) 
#camel.readLock=changed

##OPTIONAL: Camel interval (in ms) to check if it was permisison to read a file.
#camel.readLockCheckInterval=1000

##OPTIONAL: The name of the only Definition we want to import
##When this property is available all the files are imported for this Definition regardless of their names
#definition=Definition to Import

##OPTIONAL: The names of the key fields (primary and secondary) for each Definition on the property "definitions.withKeyFields"
#definition.Definition_2.keyFields=Field X,Field Y,...

##OPTIONAL: The re-mapping from fields of the record being imported to fields of Definition 2, for each Definition on the property "definitions.withKeyFields"
#definition.Definition_2.fieldsTranslation=Column X->Field X,Column Y->Field Y,...

##OPTIONAL: flag to indicate if updates should be ignored or not; default is false, i.e. process everything
## Can improve performance a lot when we just want to insert new records and we don't want or cannot filter the source
#ignoreUpdates=false

## OPTIONAL: some text to use when identifying this particular importer config
## normally used to fill the sourceField when creating new instances.
#identifier=Some text that identifies this configuration

## MANDATORY (HTTP OR DATABASE): the cron expression we should use to schedule the imports
## Uses the Quartz syntax - https://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html
cron= * * * * * ? *

############ EXCEL PROPERTIES ##################

##OPTIONAL (EXCEL): The number of rows to skip in excel, until the row with the headers
#excel.skipRows=0

##OPTIONAL (EXCEL): The Excel files encoding
#files.encoding=iso-8859-1

############# HTTP PROPERTIES ##################

## MANDATORY (HTTP) URI to consume 
# Can be replaced by a `uri.groovy` file.
http.source-uri=

############ DATABASE PROPERTIES ###############

## MANDATORY (DATABASE) Java Driver for the type of database in question 
datasource.driver=oracle.jdbc.OracleDriver

## MANDATORY (DATABASE) URL to access the database from
datasource.url=

## MANDATORY (DATABASE) Credentials for authentication
## (both may instead go in recordm-importer-secrets.properties — see the secrets template below)
datasource.username=
datasource.password=

## MANDATORY (DATABASE): the query used to select the data to import
# Can be replaced by the name of a `.sql` file in the importer directory
query=
# query=SELECT * FROM TABLE
# query=very-important-query.sql

## OPTIONAL (DATABASE): a query to use during DB connection healthcheck, defaults to "select 1+1"
#healthcheck.query=

Secrets template

Secrets live in recordm-importer-secrets.properties, next to recordm-importer.properties, and use the same key names as the main file. Any property may be moved there; a value in the secrets file wins over the same key in the main file. Secrets are never logged (startup logs list only the key names) and are not part of the properties header handed to groovy scripts — scripts read them with secret().

## MANDATORY: Token to use on RecordM and ES requests
## See: https://learning.cultofbits.com/en/developers/developer-howtos/generate-intemporal-token
recordm.token=<token>

## OPTIONAL: auth data for external apps in $extRef fields (see recordm-importer.properties for the format)
#extRef.external_app_name.auth=<cookie|header|query_param>:<name>=<value>

## OPTIONAL (DATABASE): source database credentials
#datasource.username=
#datasource.password=

## OPTIONAL (HTTP): http4 credentials and custom auth headers
#http4.authUsername=the_username
#http4.authPassword=the_password
#http.header.an-auth-key=<value>

See How to move importer secrets to a secrets file for the migration steps.