Depth Colorization Algorithm Wrong

[AlexanderKhazatsky]

Intel has posted the following algorithm for depth compression: https://dev.intelrealsense.com/docs/depth-image-compression-by-colorization-for-intel-realsense-depth-cameras

After a quick scan, this seems to have the best recovered PSNR to algorithm complexity ratio, so it seems extremely important to make sure this post is usable.

I've found two critical issues with the formula presented (attached below). First, there are clear counterexamples that do not result in an invertible transformation. In fact, I'm actually not sure when it DOES work. So I assume there is a critical issue in the posted formula. Consider the case where:
depth = 5
d_min = 1
d_max = 100

This gets mapped to the RGB value (255, 61, 61), which then gets mapped back to the depth value of 0 instead of 5.

The second issue is that the algorithm is only partially defined for a normalized depth value of 0. The r value has a valid value, but not the green and blue values, since the latter two have strict inequalities in their formulas.

The paper claims that the encoder is implemented in the librealsense SDK, but I am not able to find it. Could somebody please try and find the discrepancy between the blog post and the presumably correct SDK implementation?

[MartyG-RealSense]

Hi @AlexanderKhazatsky The depth image compression by colorization white-paper document was a proof-of-concept experiment published some years ago and whilst it is possible to implement the compression, it is now recommended that it not be used and alternative depth compression methods be used instead, such as LZ4, as discussed at #8117

Whilst it is recommended that the paper's compression method not be used, RealSense users are of course able to try it if they wish to do so,

If your preference is to continue with depth compression by colorization then there are limited references available for debugging problems with it unfortunately due to the experimental nature of the paper and the small number of RealSense users who attempted to use it. The best available reference is at the link below.

https://support.intelrealsense.com/hc/en-us/community/posts/4419463222931-Depth-Image-Colorization-Formula

[AlexanderKhazatsky]

Thanks for the references!! It seems that LZ4 is a lossless algorithm that operates on single frames. I am reading from 6 depth cameras and will have thousands of hours of data, so I will likely need more aggressive compression. Do you have suggestions for video based lossy algorithms?

[MartyG-RealSense]

Instead of using compression, setting the frames per second (FPS) speed to 6 FPS (or 5 FPS on the D455 camera model) instead of the typical 30 FPS should provide a subtantial reduction in data size.

Using a lower stream resolution will also reduce data size.

[MartyG-RealSense]

Hi @AlexanderKhazatsky Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.

[cheind]

Although this comes late in the game, for future reference, here is a vectorized Python implementation with the equations corrected

https://github.com/cheind/hue-depth-encoding

[MartyG-RealSense]

Thanks so much @cheind for sharing your solution!