HDF5 driver doesn't support extensions to standard objects

[aowen87]

This works on PDB driver but not HDF5 driver DBPutMaterial(db, "material", "mesh", nmat, matnos, matlist, dims, 3, mix_next, mix_mat, mix_zone, mix_vf, mixlen, DB_FLOAT, NULL); { long count[3]; DBobject udef_matobj = DBMakeObject("userdef_material", DB_MATERIAL, 20); /* Standard material stuf (in order of args to DBPutMaterial but that doesn't matter) / DBAddStrComponent(udef_matobj, "mesh", "mesh"); DBAddIntComponent(udef_matobj, "nmat", nmat); count[0] = nmat; DBWriteComponent(db, udef_matobj, "matnos", "userdef_material", "integer", matnos, 1, count); count[0] = dims[0]; count[1] = dims[1]; count[2] = dims[2]; DBWriteComponent(db, udef_matobj, "matlist", "userdef_material", "integer", matlist, 3, count); count[0] = 3; DBWriteComponent(db, udef_matobj, "dims", "userdef_material", "integer", dims, 1, count); DBAddIntComponent(udef_matobj, "ndims", 3); count[0] = mixlen; DBWriteComponent(db, udef_matobj, "mix_next", "userdef_material", "integer", mix_next, 1, count); DBWriteComponent(db, udef_matobj, "mix_mat", "userdef_material", "integer", mix_mat, 1, count); DBWriteComponent(db, udef_matobj, "mix_zone", "userdef_material", "integer", mix_zone, 1, count); DBWriteComponent(db, udef_matobj, "mix_vf", "userdef_material", "float", mix_vf, 1, count); DBAddIntComponent(udef_matobj, "mixlen", mixlen); DBAddIntComponent(udef_matobj, "datatype", DB_FLOAT); / Ok, lets write some extra arrays with interesting stuff **/ { char strArray[] = {"mark","sandy","fred","steve","sue","JayLo"}; char tmpList = 0; int len; / Add a simple integer valued scalar member / DBAddIntComponent(udef_matobj, "foo", 42); / Add a simple double valued scalar member / DBAddDblComponent(udef_matobj, "M_PI", 3.1415926); / Add a string valued component / DBAddStrComponent(udef_matobj, "make", "Toyota"); / Add an array of strings (Katie's case) / DBStringArrayToStringList(strArray, 6, &tmpList, &len); count[0] = len; DBWriteComponent(db, udef_matobj, "Katies_Names", "userdef_material", "char", tmpList, 1, count); free(tmpList); } / Finally, write the generic object to the file */ DBWriteObject(db, udef_matobj, 0); DBFreeObject(udef_matobj); This code creates a DBmaterial object from scratch by adding all the members that are standard to a DBmaterial object. It also addes a number of custom members. On PDB driver, DBGetMaterial works and retrieves just the standard material object members. A DBGetObject call also works and retreives all members. On HDF5 driver, it does not work. The reason is that HDF5 driver is smart about small arrays (for example, dims, or extents). The PDB driver just handles these as 'var' components inside the DBWriteComponent call writing tiny arrays to PDB. The HDF5 driver puts these in the object 'headers'. Thats fine. But, the implementation of DBWriteComponent in the HDF5 driver does the same thing as it does on the PDB driver, writing whatever it gets passed to HDF5 as a sep. array. Well, the problem happens when HDF5 tries to read the resulting object via a DBGetMaterial call. The object header is read as the 'silo' attribute but he 'dims' member of the memory representation and file representation differ significantly due to the way DBWriteComponent handle it on the customized material object. The result is that H5Aread cannot handle data conversion (between a mem rep. of int3 and a file rep of char) and so fails to read the object header. The solution is for HDF5's implementation of DBWriteComponent to not write these tiny arrays and treat them as 'var' component. But, the data has to get stored somewhere until the object is written with DBWriteObject. The solution is to add a buffer to a DBobject where HDF5 driver can store these tiny arrays and then during DBWriteObject call, construct the memory representation as appropriate before writing to the file. This occurs for all MEMBER_3() arrays in HDF5 driver.

-----------------------REDMINE MIGRATION-----------------------
This ticket was migrated from Redmine. As such, not all
information was able to be captured in the transition. Below is
a complete record of the original redmine ticket.

Ticket number: 1900
Status: Resolved
Project: VisIt
Tracker: Bug
Priority: Normal
Subject: HDF5 driver doesn't support extensions to standard objects
Assigned to: Mark Miller
Category: -
Target version: 4.10
Author: Mark Miller
Start: 07/03/2014
Due date:
% Done: 0%
Estimated time:
Created: 07/03/2014 06:18 pm
Updated: 08/28/2014 07:53 pm
Likelihood: 3 - Occasional
Severity: 2 - Minor Irritation
Found in version: 4.8
Impact:
Expected Use:
OS: All
Support Group: Any
Description:
This works on PDB driver but not HDF5 driver DBPutMaterial(db, "material", "mesh", nmat, matnos, matlist, dims, 3, mix_next, mix_mat, mix_zone, mix_vf, mixlen, DB_FLOAT, NULL); { long count[3]; DBobject udef_matobj = DBMakeObject("userdef_material", DB_MATERIAL, 20); /* Standard material stuf (in order of args to DBPutMaterial but that doesn't matter) / DBAddStrComponent(udef_matobj, "mesh", "mesh"); DBAddIntComponent(udef_matobj, "nmat", nmat); count[0] = nmat; DBWriteComponent(db, udef_matobj, "matnos", "userdef_material", "integer", matnos, 1, count); count[0] = dims[0]; count[1] = dims[1]; count[2] = dims[2]; DBWriteComponent(db, udef_matobj, "matlist", "userdef_material", "integer", matlist, 3, count); count[0] = 3; DBWriteComponent(db, udef_matobj, "dims", "userdef_material", "integer", dims, 1, count); DBAddIntComponent(udef_matobj, "ndims", 3); count[0] = mixlen; DBWriteComponent(db, udef_matobj, "mix_next", "userdef_material", "integer", mix_next, 1, count); DBWriteComponent(db, udef_matobj, "mix_mat", "userdef_material", "integer", mix_mat, 1, count); DBWriteComponent(db, udef_matobj, "mix_zone", "userdef_material", "integer", mix_zone, 1, count); DBWriteComponent(db, udef_matobj, "mix_vf", "userdef_material", "float", mix_vf, 1, count); DBAddIntComponent(udef_matobj, "mixlen", mixlen); DBAddIntComponent(udef_matobj, "datatype", DB_FLOAT); / Ok, lets write some extra arrays with interesting stuff **/ { char strArray[] = {"mark","sandy","fred","steve","sue","JayLo"}; char tmpList = 0; int len; / Add a simple integer valued scalar member / DBAddIntComponent(udef_matobj, "foo", 42); / Add a simple double valued scalar member / DBAddDblComponent(udef_matobj, "M_PI", 3.1415926); / Add a string valued component / DBAddStrComponent(udef_matobj, "make", "Toyota"); / Add an array of strings (Katie's case) / DBStringArrayToStringList(strArray, 6, &tmpList, &len); count[0] = len; DBWriteComponent(db, udef_matobj, "Katies_Names", "userdef_material", "char", tmpList, 1, count); free(tmpList); } / Finally, write the generic object to the file */ DBWriteObject(db, udef_matobj, 0); DBFreeObject(udef_matobj); This code creates a DBmaterial object from scratch by adding all the members that are standard to a DBmaterial object. It also addes a number of custom members. On PDB driver, DBGetMaterial works and retrieves just the standard material object members. A DBGetObject call also works and retreives all members. On HDF5 driver, it does not work. The reason is that HDF5 driver is smart about small arrays (for example, dims, or extents). The PDB driver just handles these as 'var' components inside the DBWriteComponent call writing tiny arrays to PDB. The HDF5 driver puts these in the object 'headers'. Thats fine. But, the implementation of DBWriteComponent in the HDF5 driver does the same thing as it does on the PDB driver, writing whatever it gets passed to HDF5 as a sep. array. Well, the problem happens when HDF5 tries to read the resulting object via a DBGetMaterial call. The object header is read as the 'silo' attribute but he 'dims' member of the memory representation and file representation differ significantly due to the way DBWriteComponent handle it on the customized material object. The result is that H5Aread cannot handle data conversion (between a mem rep. of int3 and a file rep of char) and so fails to read the object header. The solution is for HDF5's implementation of DBWriteComponent to not write these tiny arrays and treat them as 'var' component. But, the data has to get stored somewhere until the object is written with DBWriteObject. The solution is to add a buffer to a DBobject where HDF5 driver can store these tiny arrays and then during DBWriteObject call, construct the memory representation as appropriate before writing to the file. This occurs for all MEMBER_3() arrays in HDF5 driver.

Comments:
I fixed this for 4.10 release