Testing

Testing

On this page, we collect information about how to test the Vitruv framework and applications for it.

Applications

For testing applications, the ViewBasedVitruvApplicationTest base class is provided. It uses JUnit 5 and requires the implementation of at least one method:

• getChangePropagationSpecifications: Returns the ChangePropagationSpecifications developed for the applications, e.g., using the Reactions language.

It provides the complete infrastructure for tests by creating a test projects with a virtual model (V-SUM). Views can then be generated with the createSelector method of the VirtualModel and the associated API for creating views.

Note that there is also an old implementation of an application test class called VitruvApplicationTest. It does not use the views API of the VirtualModel but creates a local resource set to mimic the behavior of views. This resource set can be considered a set of views, which can be modified and whose changes can be propagated to the virtual model by predefined methods, in particular:

• resourceAt: Returns or creates a resource, i.e., a view, at the specified path
• from: Returns the root element of the specified type from the specified resource or from the resource at the specified path
• record: Records changes performed on the specified objects or any contained objects within the specified lambda
• saveAndPropagate: Saves the changes performed in previous record calls and propagates them to the virtual model
• propagate': Is a convenience method for a combined record and saveAndPropagate For further utility methods see the class documentation. For their usage, take a look at the existing tests, such as the ''SimpleChangesTests'' for the reactions language.

Important: Note that the old VitruvApplicationTest class should not be used anymore when writing new tests.

Test runtime projects

For each test case, two projects get created: one for the project itself and one for the V-SUM information. These projects are placed within the JUnit workspace folder created by Eclipse for the workspace of the JUnit plug-in tests. By default, the projects will be deleted after the test was finished successfully. To retain the project files even if the tests were successfull or to also delete them even if they failed, you can set the system property ''vitruv.retainTestProjects'' to the values ALWAYS and NEVER. For example, specifying -Dvitruv.retainTestProjects=ALWAYS would keep all project files generated during the test run.

*Note: The system property testWorkspacePath, which we have previously used to specify a fixed folder for the workspace to not have it been placed somewhere in the system's temporary files, has been removed.

Debug Output

For debug outputs, we use the Apache Log4j logger. Within a Reactions specifications, a logger can be addressed as logger in each code block.

The granularity of the debug output can be specified for a logger by calling the setLevel(Level) method on it. When using a ViewBasedVitruvApplicationTest, the logger is automatically managed. The debug level can be specified by defining the VM argument vitruv.logLevel. For example, specifying the parameter -Dvitruv.logLevel=DEBUG as a VM argument, the logger will output any logging information that is on debug level or above (warning, error etc.).

Change Visualization

The tools.vitruv.testutils.changevisualization project provided in the Vitruv-Change repository contributes a visualization of changes specified and processed in the Vitruv projects. It provides both a persistable data model as well as a UI to visualize the changes. The UI can be started standalone as a Java application running the StandaloneChangeVisualization class.

Data model

The ChangeVisualizationDataModel can monitor changes to a ChangeableModelRepository, such as a VirtualModel. It can, for example, be attached to a VirtualModel of test cases. To this end, simply create such a ChangeableModelRepository in the test and register it at the virtualModel provided in our application test classes before the test starts. The ChangeableModelRepository can then save the recorded changes to a file, or a ChangeVisualizationUI can be attached and directly shows the changes in a UI.

UI

The ChangeVisuzalizationUI provides a UI for a ChangeVisualizationDataModel. It represents all changes of the underlying data source. It is both possible to let the data model monitor a ChangeableModelRepository and see the changes in the UI on-the-fly, or to load a persisted file that was recoreded by a ChangeVisualizationDataModel before.

Functionality

The change visualization aims to help understand changes, especially when there is a complex sequence of original and consequential changes by consistency preservation. In addition to having a tree-based view on changes, the UI allows to trace IDs across changes, such that objects occurring in multiple changes are easier to identify. An interesting use case for the UI is to debug changes to projects by comparing change sequences before and after applying these changes.

Technical Hints

Running Application Tests

Running Vitruv tests, which especially means running tests in classes derived from ViewBasedVitruvApplicationTest and VitruvAplicationTest, have to be executed as JUnit Plug-in tests (not as ordinary JUnit tests). This is due to the reason that those tests start an instance of the Vitruv framework and thus depend on an Eclipse/EMF instance.

The Run Configuration of such a test should be modified afterwards to improve performance. The goal of this modification is to reuse the workspace configuration of previous test runs instead of always creating a new configuration (as this requires much time). To do so, open the Run Configuration, go to Configuration and disable under Configuration Area the checkbox Clear the configuration area before launching.

Maven Tycho

The tycho-surefire-plugin for Maven executes JUnit Plug-in tests in the tests folder if they are appropriately named (e.g. ending with "Test"). If you have ordinary JUnit tests that cannot be executed as Plug-in tests, you have to add some specification to the pom.xml of the specific project to make Maven use the maven-surefire-plugin instead of the tycho-surefire-plugin:

<build>
  <testSourceDirectory>target/classes</testSourceDirectory>
  <plugins>
    <plugin>
      <groupId>org.eclipse.tycho</groupId>
      <artifactId>tycho-surefire-plugin</artifactId>
      <version>${tycho.version}</version>
      <configuration>
        <failIfNoTests>false</failIfNoTests>
        <excludes>
          <exclude>**/.java</exclude>
        </excludes>
      </configuration>
    </plugin>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>${surefire.version}</version>
      <executions>
        <execution>
          <id>test</id>
          <phase>test</phase>
          <configuration>
            <includes>
              <include>**/*Tests.java</include>
            </includes>
            <testClassesDirectory>${project.build.directory}/classes/</testClassesDirectory>
          </configuration>
          <goals>
            <goal>test</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>