matminer issue: MaximumPackingEfficiency error

[albalu]

this structure featurizer returns the following error that is not forgiven by ignore_errors=True. You can reproduce it with the following example. The reason I put it here is that I reproduce it with our dataset that is not available in matminer, I just put it here not to forget about it...

import matminer.featurizers.structure as sf
from matbench.data.load import load_jdft2d
from matminer.utils.conversions import structure_to_oxidstructure
from pymatgen import Structure
df_init = load_jdft2d()[:2000]
# df_init = df_init[df_init['formula'].isin(['Nb3IrSe8', 'Bi2PbSe4'])]
df_init['structure'] = df_init['structure'].apply(Structure.from_dict)
structure_to_oxidstructure(df_init['structure'], inplace=True)
print(df_init)
featzer = sf.MaximumPackingEfficiency()
df = featzer.fit_featurize_dataframe(df_init, col_id='structure', ignore_errors=True)
print(df)

the error:

Traceback (most recent call last):
  File "/Users/alirezafaghaninia/Documents/py3/py3_codes/matbench/matbench/scratch.py", line 13, in <module>
    df = featzer.fit_featurize_dataframe(df_init, col_id='structure', ignore_errors=True)
  File "/Users/alirezafaghaninia/Documents/py3/py3_codes/matminer/matminer/featurizers/base.py", line 157, in fit_featurize_dataframe
    **kwargs)
  File "/Users/alirezafaghaninia/Documents/py3/py3_codes/matminer/matminer/featurizers/base.py", line 224, in featurize_dataframe
    res = pd.DataFrame(features, index=df.index, columns=labels)
  File "/Users/alirezafaghaninia/Documents/py3/lib/python3.6/site-packages/pandas/core/frame.py", line 387, in __init__
    arrays, columns = _to_arrays(data, columns, dtype=dtype)
  File "/Users/alirezafaghaninia/Documents/py3/lib/python3.6/site-packages/pandas/core/frame.py", line 7475, in _to_arrays
    dtype=dtype)
  File "/Users/alirezafaghaninia/Documents/py3/lib/python3.6/site-packages/pandas/core/frame.py", line 7552, in _list_to_arrays
    content = list(lib.to_object_array(data).T)
  File "pandas/_libs/src/inference.pyx", line 1517, in pandas._libs.lib.to_object_array
TypeError: object of type 'numpy.float64' has no len()

[ardunn]

It looks like the features is just a number, not an array (judging based on this stack trace)

[albalu]

Yes, that's what it looks like. The structures look healthy (e.g. Nb3IrSe8) and more strange is that no error occurs if this is fit_featurize_dataframe to a df that only contains two structures Nb3IrSe8 and Bi2PbSe4. However, when all 2000 entries are present the error occurs when it's Nb3IrSe8's or Bi2PbSe4's turn, etc... I couldn't immediately find out what's going on so put it here to deal with it later.

[ardunn]

@albalu Do you still get this error with updated matminer? I do not, just the MaxPackingEfficiency is Nan for 99% of samples

[albalu]

I haven't tried this recently.

[utf]

This is fixed for me using the latest version of matminer and pymatgen. When I run the snipped posted by @albalu there is no error message and all materials have a max packing efficiency.

I can only imagine it was to do with the process crash detailed in materialsproject/pymatgen#1289.