Porting KerasAug into KerasCV

[james77777778]

Hi KerasCV team,

I have worked on my own project KerasAug for a while to provide the powerful, performant and bug-free preprocessing/augmentation layers.

As I have noticed that keras-core is planning to refactor the preprocessing layers by incorporating TFDataLayer , it presents a opportunity to port some of the layers from KerasAug into KerasCV.

You can visit benchmarks to see the improvement by KerasAug

Moreover, KerasAug addresses a lot of existing issues within KerasCV:

• demo site for keras-cv layers  #1801
• issue: keras-cv MixedPrecision in KPL #1784
• issue: keras-cv MaybeApply in KPL #1783
• issue: keras-cv RandomChoice in KPL #1782
• JitterResize updates #1768
• Vectorize (+ randomize) Posterization. #1557
• Vectorize CropAndResize #1415

Will this project benefit the community? I'm willing to contribute!

[jbischof]

@james77777778 thanks for the offer. Can you explain the source of the speedups?

[james77777778]

@james77777778 thanks for the offer. Can you explain the source of the speedups?

@jbischof Sure! I pick some layers for clarification:

Layer	Why	Speedup
RandomCropAndResize	utilizes tf.image.crop_and_resize with vectorized design	+1150%
Resize	utilizes tf.image.resize with vectorized design	+1017%
RandomColorJitter	instead of dircetly using 4 preprocessing layers, implements the logic in one place to reduce the overhead	+237%
RandomGaussianBlur	utilizes tf.vectorized_map instead of tf.map_fn	+1055%
GridMask	computes random_transform in vectorized manner	+368%

Additionally:

• adds the bounding_boxes support for Resizing layer with all 3 modes (crop, pad and distort)
• gauratees that the tensor shape is static if it should be. I believe KerasCV has some bugs in Resizing as reported in Ragged Tensor does not work with BoxCOCOMetrics #1894
• fully supports mixed_precision policy (mixed_float16 and mixed_bfloat16). Some casting needs to be added for this functionality.

If these improvements are in the plan of KerasCV, I can send the PR (maybe one fix/improvement one small PR?).

[jbischof]

Sounds great @james77777778! I would suggest starting with one or two layers with big improvements so that we can debug any issues early in the process.

[sachinprasadhs]

Thanks for reporting the issue! We have consolidated the development of KerasCV into the new KerasHub package, which supports image, text, and multi-modal models. Please read keras-team/keras-hub#1831. KerasHub will support all the core functionality of KerasCV.

KerasHub can be installed with !pip install -U keras-hub. Documentation and guides are available at keras.io/keras_hub.

With our focus shifted to KerasHub, we are not planning any further development or releases in KerasCV. If you encounter a KerasCV feature that is missing from KerasHub, or would like to propose an addition to the library, please file an issue with KerasHub.