Addons image transforms crashing with CPU only tensorflow

[charlielito]

System information

• OS Platform and Distribution: Linux Ubuntu 16.04
• TensorFlow version and how it was installed: CPU version 2.2.0, installed inside a conda env with pip install tensorflow-cpu
• TensorFlow-Addons version and how it was installed source or binary: version 0.10.0 , installed inside conda env with pip install tensorflow_addons
• Python version: 3.6.10
• Is GPU used? (yes/no): no
• My Machine does have a GPU

Describe the bug

Simple program crashes after trying to use tfa.image.rotate because some CUDA related stuff.

Code to reproduce the issue
Create a fresh conda env and install all:

conda create -n tfa_test python=3.6
conda activate tfa_test
pip install tensorflow-cpu tensorflow_addons

And the code:

import tensorflow_addons as tfa
import tensorflow as tf

image = tf.random.uniform((32, 32, 3))
image = tfa.image.rotate(image, 45)

Other info / logs

Traceback (most recent call last):
  File "test_addons.py", line 5, in <module>
    image = tfa.image.rotate(image, 45)
  File "/home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow_addons/image/transform_ops.py", line 351, in rotate
    interpolation=interpolation,
  File "/home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 580, in __call__
    result = self._call(*args, **kwds)
  File "/home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 627, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)
  File "/home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 506, in _initialize
    *args, **kwds))
  File "/home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 2446, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 2777, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 2667, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 981, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 441, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 968, in wrapper
    raise e.ag_error_metadata.to_exception(e)
tensorflow.python.framework.errors_impl.NotFoundError: in user code:

    /home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow_addons/image/transform_ops.py:111 transform  *
        output = _image_so.ops.addons_image_projective_transform_v2(
    /home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow_addons/utils/resource_loader.py:64 ops
        self._ops = tf.load_op_library(get_path_to_datafile(self.relative_path))
    /home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py:58 load_op_library
        lib_handle = py_tf.TF_LoadLibrary(library_filename)

    NotFoundError: /home/charlie/miniconda3/envs/tfa_test/lib/python3.6/site-packages/tensorflow_addons/custom_ops/image/_image_ops.so: undefined symbol: __cudaPushCallConfiguration

[bhack]

Dup of #990