Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

editing multidimensional array with unlimited dimension by slice #906

Closed
rocheseb opened this issue Apr 7, 2019 · 3 comments
Closed

editing multidimensional array with unlimited dimension by slice #906

rocheseb opened this issue Apr 7, 2019 · 3 comments
Labels

Comments

@rocheseb
Copy link

rocheseb commented Apr 7, 2019

In the code below, I first write a netCDF file with a fixed and unlimited dimension, two 2D variables, and a 1D variable.

Then I open the file in edit mode, and try to update the variable 'v2' which has the first dimension fixed and second unlimited.

It seems I cannot edit the 'v2' variable by slice and instead need to do it element by element ('test 3' does not work while 'test 5' does).

I would expect to be able to edit 'v2' like I do for 'v1' in 'test 6', or 'v3' in test 7

Is this a bug or a usage error?

The code:

import netCDF4
import numpy as np

with netCDF4.Dataset('test.nc','w') as f:
	f.createDimension('d1',3)
	f.createDimension('d2',None)

	f.createVariable('v1',np.float,('d1','d1',))
	f.createVariable('v2',np.float,('d1','d2',))
	f.createVariable('v3',np.float,('d2',))

with netCDF4.Dataset('test.nc','r+') as f:

	print('\ntest 1')
	print(f['v2'])
	print(f['v2'][:])
	
	print('\ntest 2')
	# this is not needed, the next steps behave the same without this one
	f['v2'][:] = np.zeros((3,5))
	print(f['v2'])
	print(f['v2'][:])

	print('\ntest 3')
	# why does this throw an error? f['v2'][0,:] and np.arange(5) both have the same shape: (5,)
	try:
		f['v2'][0,:] = np.arange(5)
	except IndexError:
		print('IndexError')
	else:
		print(f['v2'][:])

	print('\ntest 4')
	# this does not throw an errorm but does nothing?
	f['v2'][:][0,:] = np.arange(5)
	print(f['v2'][:])

	print('\ntest 5')
	# this actually modifies the variable
	for i,elem in enumerate(np.arange(5)):
		f['v2'][0,i] = elem
	print(f['v2'][:])

	print('\ntest 6')
	# with a non unlimited dimension, it works
	f['v1'][0,:] = np.arange(3)
	print(f['v1'][:])

	print('\ntest 7')
	# with a 1D unlimited dimension, it also works
	f['v3'][:] = np.arange(5)
	print(f['v3'][:])

With this ouput:

    test 1
    <class 'netCDF4._netCDF4.Variable'>
    float64 v2(d1, d2)
    unlimited dimensions: d2
    current shape = (3, 0)
    filling on, default _FillValue of 9.969209968386869e+36 used
    
    []
    
    test 2
    <class 'netCDF4._netCDF4.Variable'>
    float64 v2(d1, d2)
    unlimited dimensions: d2
    current shape = (3, 5)
    filling on, default _FillValue of 9.969209968386869e+36 used
    
    [[0. 0. 0. 0. 0.]
     [0. 0. 0. 0. 0.]
     [0. 0. 0. 0. 0.]]
    
    test 3
    IndexError
    
    test 4
    [[0. 0. 0. 0. 0.]
     [0. 0. 0. 0. 0.]
     [0. 0. 0. 0. 0.]]
    
    test 5
    [[0. 1. 2. 3. 4.]
     [0. 0. 0. 0. 0.]
     [0. 0. 0. 0. 0.]]
    
    test 6
    [[0.0 1.0 2.0]
     [-- -- --]
     [-- -- --]]
    
    test 7
    [0. 1. 2. 3. 4.]

And here is the IndexError message:

  ---------------------------------------------------------------------------
   IndexError                                Traceback (most recent call last)
   ~/testdir/testnc.py in <module>()
        25         # why does this throw an error? f['v2'][0,:] and np.arange(5) both have the same shape: (5,)
        26         #try:
   ---> 27         f['v2'][0,:] = np.arange(5)
        28         #except IndexError:
        29         #       print('IndexError')
   
   netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Variable.__setitem__()
   
   ~/anaconda3/lib/python3.7/site-packages/netCDF4/utils.py in _StartCountStride(elem, shape, dimensions, grp, datashape, put, no_get_vars)
       379             elif unlim and e.stop is None and datashape != ():
       380                 if e.start is None:
   --> 381                     length = datashape[i]
       382                 else:
       383                     length = e.start+datashape[i]
   
   IndexError: tuple index out of range


@jswhit
Copy link
Collaborator

jswhit commented Apr 7, 2019

This does look like a bug. The workaround for now is to use f['v2'][0,0:5] = np.arange(5).

@jswhit jswhit added the bug label Apr 7, 2019
jswhit added a commit that referenced this issue Apr 7, 2019
@jswhit
Copy link
Collaborator

jswhit commented Apr 7, 2019

OK, this should be fixed in branch issue906 (pull request #907)

jswhit added a commit that referenced this issue Apr 8, 2019
fix for slicing bug (issue #906)
@jswhit
Copy link
Collaborator

jswhit commented Apr 8, 2019

Merge #907 - if the issue persists for you, feel free to reopen this issue.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants