ImageDataGenerator With 2 channel data?

[varoudis]

Any plans for allowing arbitrary number of channels in the processing utils?
Thanks

[madeofwin]

Have you investigated if and where it fails with 2 channel data? If I remember correctly, the apply_transform method is applying the transformation to each channel separately. In fact, I have used it with data of different channels (2 also), but only by calling the transformation methods (shifting, rotating, etc.) directly, not via ImageDataGenerator.

[varoudis]

To be honest I haven't as I though 99% of training is done with 1 or 3 channels from images. Reading the docs was also stating this.

Ill come back with results on this.

[oeway]

hi @varoudis , you are right, currently ImageDataGenerator only support 1 and 3 channels. But I am working on extending it in many aspects, check #3338 , one goal is to extend it to support arbitrary number of channels, I now have a working version, you are welcome to try and provide feedback.

[varoudis]

that sounds great!

[gledsonmelotti]

@oeway In the keras command, you have to define the image type, that is, color_mode = 'rgb' or 'grayscale' when using the command flow_from_directory with model.fit_generator. So how do I change color_mode in flow_from_directory to use with 2, 4, 5 and 6 channels?

I thank you for your attention,
Gledson

[oeway]

@gledsonmelotti You can try with my extension: #3338 . For images with 2, 4, 5 and 6 channels or more, you need to define your own image_reader.

This is currently we used by default:

def pil_image_reader(filepath, target_mode=None, target_size=None, dim_ordering=K.image_dim_ordering(), **kwargs):
    img = load_img(filepath, target_mode=target_mode, target_size=target_size)
    return img_to_array(img, dim_ordering=dim_ordering)

You can define your own reader, for example, you can load your image in tif format which can contain multiple frames other than 1 or 3.

def customized_image_reader(filepath, target_mode=None, target_size=None, dim_ordering=K.image_dim_ordering(), **kwargs):
    imgTiff = PIL.Image(filepath, mode="r")
    imgs = []
    for i in range(6):
        imgTiff.seek(i)
        img = np.array(imgTiff, dtype="float32")
        imgs.append(img)
    imgArr = np.stack(imgs)
    return imgArr

Then you can use it like this:

datagenX.flow_from_directory(image_folder, image_reader=customized_image_reader, ...)

Please also notice that there maybe compatible issue with latest Keras, you may need to adjust the code yourself.

[thepate94227]

@oeway
the flow.from.directory function doesn't have a parameter called image_reader...
I have the same problem. I have a stack of image labels, in this case to grayscale images. So the stack has the shape (300,200,2). I don't know how to train my model, when i can't use flow_from_directory function...

[oeway]

@thepate94227 That's because my extension is not in the official implementation, perhaps you want to look at this: https://github.com/bernardohenz/ExtendableImageDatagen a further extension made by @bernardohenz.

[thepate94227]

@oeway I tried to do it your way, but i get an error in this line

imgTiff = PIL.Image(filepath, mode="r")
TypeError: 'module' object is not callable

Do you know why? It seems, that PIL.Image is not a function...
Usually i use flow_from_directory and my Filepath is like: /Images/
And there are 2 Folders: Input and Label. In each Folder there is another folder called "0". So basically i have 2 folders /Images/Input/0/ and /Images/Label/0/. This two folders contain my images respectively labels.
Which Filepath do i have to use for my custom_filereader? I tried /Images/ like i use in Keras, and i tried /Images/Label/ and /Images/Label/0/, because i only need the custom_filereader for my labels. But i always get the TypeError.

This is my code so far:

NAME = "DeepLab-{}".format(int(time.time()))
deeplab_model = Deeplabv3(input_shape=(300,200,3), classes=2)
tensorboard = TensorBoard(log_dir="/logs/{}".format(NAME))
deeplab_model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=['accuracy'])

def customized_image_reader(filepath, target_mode=None, target_size=None, dim_ordering=K.image_dim_ordering(), **kwargs):
    imgTiff = PIL.Image.open(filepath, mode="r")
    imgs = []
    for i in range(6):
        imgTiff.seek(i)
        img = np.array(imgTiff, dtype="float32")
        imgs.append(img)
    imgArr = np.stack(imgs)
    return imgArr

image_reader = customized_image_reader(filepath='/Keras3/Label/',
                                                  target_size=(300,200,2),
                                                  target_mode=None)

# we create two instances with the same arguments
data_gen_args = dict(featurewise_center=True,
                     featurewise_std_normalization=True,
                     rotation_range=90,
                     width_shift_range=0.1,
                     height_shift_range=0.1,
                     zoom_range=0.2)

image_datagen = ImageDataGenerator(**data_gen_args)
mask_datagen = ImageDataGenerator(**data_gen_args)

# Provide the same seed and keyword arguments to the fit and flow methods
seed = 1
#image_datagen.fit(images, augment=True, seed=seed)
#mask_datagen.fit(masks, augment=True, seed=seed)

image_generator = image_datagen.flow_from_directory(
    'Keras3/Input/',
    target_size=(300,200),
    class_mode=None,
    seed=seed)

mask_generator = mask_datagen.flow_from_directory(
    class_mode=None,
    seed=seed,
    image_reader=image_reader)

# combine generators into one which yields image and masks
train_generator = zip(image_generator, mask_generator)

deeplab_model.fit_generator(train_generator, steps_per_epoch= np.uint32(2935 / 32), epochs=10, callbacks=[tensorboard])

deeplab_model.save_weights('deeplab_6.h5')
deeplab_model.save('deeplab-6')

session.close()

[oeway]

You should do from PIL import Image and img = Image.open(filepath).

[thepate94227]

You should do from PIL import Image and img = Image.open(filepath).

Dann i get an error: IsADirectoryError: [Errno 21] Is a directory: '/Keras3/Label/0/'

[gledsonmelotti]

@gledsonmelotti You can try with my extension: #3338 . For images with 2, 4, 5 and 6 channels or more, you need to define your own image_reader.

This is currently we used by default:

def pil_image_reader(filepath, target_mode=None, target_size=None, dim_ordering=K.image_dim_ordering(), **kwargs):
    img = load_img(filepath, target_mode=target_mode, target_size=target_size)
    return img_to_array(img, dim_ordering=dim_ordering)

You can define your own reader, for example, you can load your image in tif format which can contain multiple frames other than 1 or 3.

def customized_image_reader(filepath, target_mode=None, target_size=None, dim_ordering=K.image_dim_ordering(), **kwargs):
    imgTiff = PIL.Image(filepath, mode="r")
    imgs = []
    for i in range(6):
        imgTiff.seek(i)
        img = np.array(imgTiff, dtype="float32")
        imgs.append(img)
    imgArr = np.stack(imgs)
    return imgArr

Then you can use it like this:

datagenX.flow_from_directory(image_folder, image_reader=customized_image_reader, ...)

Please also notice that there maybe compatible issue with latest Keras, you may need to adjust the code yourself.

@oeway thanky very much. It is perfect.

[gledsonmelotti]

You can find a good idea in #10499 and https://stackoverflow.com/questions/49404993/keras-how-to-use-fit-generator-with-multiple-inputs. They use two ImageDataGenerator.