Thank you for providing sample database, I will try it out and see if I can discover the issue.

@biagas - Thanks! Please let me know if you need anything else

So you are correct, VisIt's VTK reader only reads the first file in a multi-file setting to determine the variable list.

This was a design choice due to the fact that the entire file must be read just to get the meta-information like variable names. This can be quite costly if there are hundreds or thousands of blocks, or even if there are few blocks, but each are rather large files.

A case could be made for allowing the user to accept this cost, perhaps a read option to the effect 'read all blocks at open.'

I see that your 'regions' (VisIt's reader set them up as 'groups' in VisIt's SIL parlance), each have distinct variable lists. I am not sure VisIt would play nice with this setup, even if the full variable list had been read in. Would have to muck with the reader to try it out and see what happens. Can report back on this later.

I performed a quick mod of the VTK reader so that it would read in multiple blocks and do a better job of populating the variable list. As I suspected, VisIt isn't reacting well to not having all the variables defined in each of the individual files. I think the variables would need to be grouped by their respective regions. For vars that aren't present in both regions, I needed to do a SIL selection to choose the correct Region in order for VisIt to plot it correctly.

Do you consider each region to be a different mesh, or computational space? If VisIt separated them and you wanted to plot 'pressure' you would have to choose which mesh/region, and create 2 plots if you wanted to see both, but vars contained in a single region would plot successfully without needing to use the SIL selection.

GEOSX considers regions to be areas with a set of constitutive models. In practice this could look like:

1. A solid material with a linear-elastic model, darcy flow model
2. A solid material with a linear-elastic model, impermeable to flow
3. A surface region where contact is evaluated, parallel-plate flow model (representing a propagating fracture)

Here, (1) and (2) could exist in the same solid mesh, and (3) would correspond to a separate surface mesh. (1), (2), and (3) could also have distinct variables. I should also mention that we have an active PR to enable multi-mesh models and vtk outputs. Hopefully this makes sense... We could also bring someone like Randy Settgast into the conversation in case we need to address our vtk format.

In practice, a typical non-super-user would likely only run a model with a single solid and single surface region. With our legacy code + silo output, we typically required the user to generate two plots to track properties of the solid/surface regions.

I should also mention that my current work-around is a python script that breaks the vtm file up so that the regions are in different files, and then use the database correlation feature to keep everything in sync.

I think your use case really needs multi-mesh functionality, not currently possible with VTK files (that I know of).
That said, the portion of the VTK reader in VisIt that parses '.vtm' files could be modified to allow different conventions than currently supported:

1. Add a read-option that would allow the highest level blocks to be treated as separate meshes. The highest level blocks are currently treated as 'groups' in VisIt.
2. Update the .vtm parser that VisIt uses to support nested vtkMultiBlockDataSet at the root level, and treat each vtkMultiBlockDataSet as a separate mesh, the .vtm file would look like this:

<?xml version="1.0"?>
<VTKFile type="vtkMultiBlockDataSet" version="1.0">
    <vtkMultiBlockDataSet>
        <Block name="CellElementRegion">
            <Block name="Domain">
                <DataSet name="rank_0" file="000000/0_Domain.vtu" />
            </Block>
        </Block>
        <Block name="WellElementRegion" />
    </vtkMultiBlockDataSet>
    <vtkMultiBlockDataSet>
        <Block name="SurfaceElementRegion">
            <Block name="Fracture">
                <DataSet name="rank_0" file="000000/0_Fracture.vtu" />
            </Block>
        </Block>
    </vtkMultiBlockDataSet>
</VTKFile>

Of course, if changing the format of your output files isn't feasible, then option 1 would be the only choice.

@brugger1, @markcmiller86 Would you have other suggestions?

Thanks for your help on this - I'm planning to check in with other GEOSX devs this week to get their thoughts on this issue.

It may be worth noting that we're planning on adopting the following hierarchy for multibody problems (PR on the GEOSX side: GEOS-DEV/GEOS#1742):

<?xml version="1.0"?>
<VTKFile type="vtkMultiBlockDataSet" version="1.0">
	<vtkMultiBlockDataSet>
		<Block name="backgroundGrid">
			<Block name="CellElementRegion">
				<Block name="Region1">
					<DataSet name="rank_0" file="0.000000/0_Region1.vtu" />
					<DataSet name="rank_1" file="0.000000/1_Region1.vtu" />
				</Block>
			</Block>
			<Block name="WellElementRegion" />
			<Block name="SurfaceElementRegion" />
		</Block>
		<Block name="particles">
			<Block name="CellElementRegion">
				<Block name="Region2">
					<DataSet name="rank_0" file="0.000000/0_Region2.vtu" />
					<DataSet name="rank_1" file="0.000000/1_Region2.vtu" />
				</Block>
			</Block>
			<Block name="WellElementRegion" />
			<Block name="SurfaceElementRegion" />
		</Block>
	</vtkMultiBlockDataSet>
</VTKFile>

I am working of modifying our VTK reader to accommodate your use-case.

Can it be assumed that each dataset within a given region will have the same variables?
If yes, the up-front cost of finding all the variable names can be reduced.

@povolny1 in your example, should the 'backgroundGrid' and 'particles' blocks be considered separate meshes?

@biagas Apologies for the late reply - yes, ideally we'd like to be able to simultaneously plot independent fields from 'backgroundGrid' and 'particles.'

Also, all the elements in a region use the same constitutive model, so they'd all have the same fields/variables.

@povolny1, any progress on this front in terms of solidifying the hierarchical structure GEOSX will be using for .vtm files?

I've pulled something together which I believe addresses most of the issues raised. I hacked the previously provided sample data to match the proposed hierarchy for testing, but 'real' data would be better.

Trying to get this in for our 3.3 release. It may be disabled by default (I view it as experimental at this point without good data for testing).

If your .vtm file structure is still a moving target, I can put this on hold and add it into VisIt when your team is ready.

I'm hoping to get the .vtm hierarchy finalized soon. @biagas What would be the best way to go about testing your changes? Should I just send you a dataset?

To start with, sending me a dataset would be best. I had something sort of working as a separate plugin. But users would have to then select 'GEOSX' from the 'open as' pulldown, so I am currently trying to integrate it with the current VTK plugin, to save user's a step. It also reduces code duplicaton.

Once I think I have things working, we can talk about sharing the plugin with you for your testing before I make a final commit in VisIt.

@cssherman , @pvolny1 what is the status of this on your end? I have been waiting to hear back on the final specs for your .vtm files.

@biagas - sorry for dropping the ball on this! I will put together some example outputs that are produced by GEOS, so that we can evaluate where we are on this problem.

Hi @biagas - again, sorry for the delay in getting you these examples. I've attached a couple of examples here that show some common vtk patterns in GEOS which don't always play well with the current version of VisIt.

The first of these includes a single-phase THM test case which has a fracture contained in its own domain. In this case, we can see most of the information when we load the top-level .pvd file. However, we can't see values that are only present on the fracture domain. Here, I'm loading the fracture separately to produce this visualization:

Top-level available parameters:

Fracture-only available parameters:

Here is a zip file with the relevant vtk files:
thm_single_phase_with_fractures.zip

@biagas - Here's a second case, which has a particle passing through a background grid. This case shows some of the same parameter availability issues as the previous case. However, here the number of ranks in the particle domain increases at it moves across the partition in the center of the model. In the following visualization, I've manually created some vtk/pvd files that only contain the particle and tracked it through time. When the particle passes through the partition, only 2/4 of the domains plot here:

Particle starting position:

Particle crossing partition:

Particle final position:

Top-level parameter availability:

Particle-only parameter availability:

Finally, here is a zip file with the relevant vtk files:

mpm_moving_through_partition.zip

I'm not up-to-date on this conversation, but the current version of VisIt is 3.4.1.

I'm not up-to-date on this conversation, but the current version of VisIt is 3.4.1.

@JustinPrivitera - good point. I've updated to this version, and see the same observations detailed above.

@cssherman thank you for the files, I will resurrect my previous attempts to accommodate the GEOS data.

@cssherman, couple of questions. I know the examples are probably very small samples.
Just wondering if there are ever multiple blocks with 'Level' in the name, eg 'Level0', 'Level1' etc.
Also, will the root file always be a .pvd file with timestate files always in .vtm?
Trying to work out how to distinguish your flavor of VTK from generic so VisIt can present your data in the best possible manner.

@biagas - thanks for your questions. I'm going to ask some of the other GEOS devs to chime in here.

@cssherman, couple of questions. I know the examples are probably very small samples. Just wondering if there are ever multiple blocks with 'Level' in the name, eg 'Level0', 'Level1' etc. Also, will the root file always be a .pvd file with timestate files always in .vtm? Trying to work out how to distinguish your flavor of VTK from generic so VisIt can present your data in the best possible manner.

Yes, currently there is always a pvd file pointing to multiple vtms with different timestamp which then point to the various blocks (vtu files). As you have noticed not all blocks carry the same field since we use them to represent different constitutive regions. One thing to keep in mind is that the multiblockdataset will be deprecated (if it hasn't been already) and we will likely have to move to the new paritionDataset format (https://discourse.paraview.org/t/transition-from-multiblock-to-partitioned-datasets-in-vtk-and-paraview-a-complete-guide/12061).

@cssherman I have made progress incorporating a GEOS-flavor to our VTK database plugin.
I tried to make it seamless (eg just open your .pvd file and the reader will do the right thing). It is currently keyed off the presence of block names CellElementRegion or SurfaceElementRegion or ParticleRegion or WellElementRegion in the first .vtm file. If any of these are present it considers the data to be GEOS flavor, otherwise reads the .pvd and .vtm files as currently handled.

Each .vtm file is brought into VisIt as a single mesh, with multiple groups (could be renamed regions if you prefer). Each group contains however many blocks were specified as the individual .vtu files.

Here is the Pseudocolor menu for the THM test case:

And here is a Mesh plot of the particle region example, with a Pseudocolor of particleDamage at the partition, also showing the Subset selection window with group and block designations. The groups option could be used to turn on/off regions of intereset.

The group names are currently very verbose, I was just trying to capture as much information from your hierarchy as possible while still fitting withing VisIt's infrastructure.

Please let me know your thoughts on the current interface.

If you would like to take it for a spin, I can give you the plugin files to build as a 'private plugin', with instructions on how to build. If you are on Windows, I can simply provide the dlls.

Work is currently off our develop branch, but should work with 3.4.x VisIt.

@biagas - that's great news! I would be happy to test it out on a windows machine if that would be helpful on your end.

Also, I forgot about it until just now, but I believe that there may be another small bug related to the GEOS flavor .pvd file. I've found that when I load the .vtm files directly as a smart-group, time curve picks behave correctly. However, when I do the same for the data loaded as a .pvd, VisIt throws the following error for the "Time curve use picked element" option (but not for the coordinate pick):

Curve:  ()
viewer.exe: The pipeline object is being used improperly: Invalid timestep range requested.

In the following screenshot, I have one of the examples loaded (using v3.4.1) and pick one of the fields to inspect over time (here, pressure):

I will test out the time-pick, then upload the dll's in a tarball.

Time pick works with the updates to the reader. I did need to change the QueryOverTime options for the x-axis to Time as shown below. Timestep would work too. This is due to my not populating 'Cycle' for the reader.

I've attached the dlls that you can install. Unzip them to your VISIT user-home directory. You can double check its location by running visit.exe -env, and the first entry for VISITPLUGINDIR is where VisIt will look for private plugins. Put the dll's in a databases subdirectory.

Let me know what you think.

Oops, here are the dlls.
GEOS-vtk.zip

@biagas - I almost always use the time option, so that works for me. I'll give these a test run!

I need to add more logic to support you opening just a single .vtm file, because that currently isn't handled (for your flavor of .vtm). Will start on that.

@cssherman how did the test-run go? Anything adjustments need to be made? I'd like to get this merged in so it can go in the next version of VisIt.

@cssherman how did the test-run go? Anything adjustments need to be made? I'd like to get this merged in so it can go in the next version of VisIt.

I attempted to test out the updated dll's using Visit 4.3.1 on Windows, but received some error messages that looked like linking issues when I launched the code. I'm guessing that I might need to try out the current version of develop for this to work?

It should have worked with 3.4.1. If you don't mind updating to 3.4.2 and testing again, that would be great. If there are error messages again, please make note and post the errors here.

@biagas - that did the trick. The new capabilities work great on my end! Thank you for your help with this!

@cssherman Thanks for checking, I will move forward with getting this merged.

@cssherman another question: may I add your sample data from this ticket to our repo for use in nightly testing?

@biagas - Sure, there is no issue there