Operational Phases

This page describes the operational phases that are executed by the benchmark software. One, some or all of the phases can be enabled and will run in the order listed below.

• loadOntologies - populate the RDF database with required ontologies (from reference knowledge). Required phase, can be done manually by uploading all .ttl files located at : /data/ontologies
• loadReferenceDatasets - populate the RDF database with required reference data (from reference knowledge). Required phase, can be done manually by uploading all .ttl files located at : /data/datasets
• generateCreativeWorks - generate the data used for benchmarking. Data is saved to files of defined size (generatedTriplesPerFile) and total number of triples (datasetSize). Requires phases : loadOntologies, loadDatasets.
• loadCreativeWorks - load generated data from previous phase into RDF database, using N-Quads serialization format. Some databases may not be able to run this phase, in that case generated data can be loaded manually.
• generateQuerySubstitutionParameters - Controls generation of query substitution parameters which later can be used during the warmup and benchmark phases. For each query a substitution parameters file is created and saved into 'creativeWorksPath' location. If no files are found at that location, queries executed during warmup and benchmark phases will use randomly generated parameters. Requires phases : loadOntologies, loadDatasets, generateCreativeWorks, loadCreativeWorks.
• validateQueryResults - validate correctness of results for editorial and aggregate operations against a validation dataset. Requires phases : loadOntologies, loadDatasets.
• warmUp - runs the aggregation agents for warmupPeriodSeconds seconds, results are not collected. Requires phases : loadOntologies, loadDatasets, generateCreativeWorks (optional, in case it has been already done), loadCreativeWorks (optional, in case it has been already done), generateQuerySubstitutionParameters.
• runBenchmark - runs the benchmark for benchmarkRunPeriodSeconds seconds, results are collected. Editorial and aggregation agents are run simultaneously. Requires phases : loadOntologies, loadDatasets, generateCreativeWorks (optional, in case it has been already done), loadCreativeWorks (optional, in case it has been already done), generateQuerySubstitutionParameters.
• runBenchmarkOnlineReplicationAndBackup - benchmark is measuring performance under currently ongoing backup process. Verifies that certain conditions are met such as milestone points at which backup has been started. Requires additional implementation of provided shell script files (/data/enterprise/scripts) for using vendor's specific command for backup. Requires phases : loadOntologies, loadDatasets, generateCreativeWorks, loadCreativeWorks, generateQuerySubstitutionParameters. Also making a full backup prior to running the benchmark for later restore point.
• checkConformance - runs tests for conformance to the OWL2-RL rules. Requires phase : loadOntologies.. To be run independently on an empty repository (using OWL2-RL rule-set). Required phase : loadOntologies. No data generation and loading is required.