Connection Profile 3.5

Connection Profile

The Connection Profile is a properties file that looks like this:

app.instance=dev00000
app.username=datapump.agent
app.password=******
app.agent=demo

reader.instance=dev00000
reader.username=datapump.reader
reader.password=******

database.url=jdbc:mysql://localhost/sndm
database.schema=sndm
database.username=admin
database.password=******

The connection profile is specified with the -p command line option.

java -jar sndm-3.n.n-mysql.jar -p profile -y yamlfile

ServiceNow Connections

Beginning with release 3.5, the loader uses two separate connections for communicating with the ServiceNow instance. If you have installed and are using the DataPump App, then it is recommended that you use two separate user accounts.

• reader — These credentials are used to read data records from the instance. This account should be granted snc_read_only role, plus additional roles as required to access the records being exported.
• app — These credentials are used to connect to the DataPump App, but are not used to read data. This account should be granted x_108443_sndml.daemon role. This account is unnecessary if you are using the command line (YAML) loader.

If you are NOT using the DataPump App, then you only need to specify a reader account. However, you must insure that the account has read access to the following ServiceNow tables in order to create new SQL tables in the database:

• sys_db_object
• sys_dictionary
• sys_glide_object

Database Connection

Database username, password and schema are required unless you are using SQLite. For SQLite you only need to provide a URL. The value of datamart.schema will vary based on the DBMS.

• For MySQL, datamart.schema should be the name of the database
• For Oracle, datamart.schema should be the name of the user
• For Microsoft SQL Server, datamart.schema should be dbo
• For SQLite, datamart.schema should be main or left blank

Variable Substitution

The loader will interpolate environment variables using the notation ${varname}. For example, if the environment variable SNDML_CONFIG has been defined, then you can do something like this:

loader.metrics_folder=${SNDML_CONFIG}/metrics

If the entire property value is enclosed in backticks, then it will be passed to Runtime.exec() for evaluation. For example, if you have the LastPass CLI installed in /usr/local/bin, then you may be able to do something like this:

reader.password=`/usr/local/bin/lpass show --password my.servicenow.reader.password`
app.password=`/usr/local/bin/lpass show --password my.servicenow.datapump.password`
database.password=`/usr/local/bin/lpass show --password my.database.password`

List of Properties

Alternate names are provided for some properties in order to maintain backwards compatibility with earlier versions of the code. The list of property default values and alternate names is maintained in the file property_names.xml.

Property Name	Alternate Name	Notes / Description
reader.instance	servicenow.instance	Required. Name or URL of the ServiceNow instance from which data records will be retrieved. You can specify an instance name (e.g. dev12345) or a URL (e.g. https://dev12345.service-now.com)
reader.username	servicenow.username	Required.
reader.password	servicenow.password	Required.
reader.domain	servicenow.domain	For use in domain separated instances. This can be a single sys_id or a comma separated list of domains. Refer to Domain Separation.
reader.pagesize	servicenow.pagesize	Number of records to be processed at a time if not overridden at the table level. Default is 200.
app.agent	daemon.agent	Required if using the DataPump app.
app.scope	daemon.scope	Optional. Default is x_108443_sndml.
app.instance	servicenow.instance	Required if using the DataPump app. Name or URL of the instance where the app is installed.
app.username	servicenow.username	Required if app.instance is specified.
app.password	servicenow.password	Required if app.instance is specified.
dict.instance		Name or URL of the instance from which dictionary information will be obtained. This is optional and the default is to use reader.instance. This property is not applicable if using the DataPump app (i.e. if app.instance is specified.)
dict.username		Required if dict.instance is specified.
disct.password		Required if dict.instance is specified.
database.url	datamart.url	Required. JDBC URL for the target database. Refer to the documentation for your selected JDBC driver.
database.username	datamart.username	Required unless database is SQLite. User name for the target database.
database.password	datamart.password	Required unless database is SQLite.
database.dialect	datamart.dialect	Specifies a section in the templates file. If not specified, the loader will infer the correct section using the drivers element of the template and the JDBC prefix of the datamart URL.
database.schema	datamart.schema	Recommended. See notes above.
database.templates	datamart.templates	The name of an XML file to use for generating SQL statements. If not specified, the loader will use sqltemplates.xml which is embedded in the JAR. For information refer to Templates.
database.autocreate	datamart.autocreate	Causes tables in the SQL database to be created if they do not already exist. Defaults is "true".
database.timezone		This is normally not specified. If omitted then "GMT" will be used. If set to "default" or "local" then the time zone of the Java VM will be used.
database.warn_on_truncate		Causes a warning to be written to the log whenever a field is truncated when written to the database. Default is "true".
loader.metrics_folder		The name of a directory where metrics files are located. If not specified, metrics files are assumed to reside in the current working directory. For information refer to Metrics.
daemon.interval		Recommended if using --daemon. Interval in seconds. Default is 60.
daemon.continue		Controls whether or not the daemon will continue running after encountering an IO error in communication with ServiceNow or the database. If “true” then it writes an error to the log and goes back to sleep. If “false” then it will immediately abort.
server.port		Required if using --server.
server.backlog		HTTP Server socket backlog. Default is 0.
server.threads	daemon.threads	Maximum number of jobs that can be processed concurrently. Additional jobs will wait until a thread becomes available. Applicable for --daemon or --server. Default is 4.
server.pidfile	daemon.pidfile	If specified, then the process ID will be written to this file. This is used in conjunction with a server or daemon. If you want to shut down the server then read the PID number from this file and send a kill signal to the PID.
server.shutdown_seconds	daemon.shutdown_seconds	Seconds that the daemon or server will wait before aborting worker threads. Default is 30.
server.heartbeat		Applicable if using --server. Interval in seconds that the HTTP server will send a heartbeat message to the scoped app confirming that it is still listening and that the app session is still active. Default is 300. Set to 0 to disable the heartbeat.