Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

CreateFromDepthImage uint16 confusion #6962

Open
3 tasks done
rurban opened this issue Sep 11, 2024 · 1 comment
Open
3 tasks done

CreateFromDepthImage uint16 confusion #6962

rurban opened this issue Sep 11, 2024 · 1 comment
Labels
bug Not a build issue, this is likely a bug.

Comments

@rurban
Copy link
Contributor

rurban commented Sep 11, 2024

Checklist

Describe the issue

I get strange result from my realsense Z16 depth images, which has the depths stored as uint16_t (little-endian).
open3d PointCloud::CreateFromDepthImage only accepts num_channels == 1 and bytes_per_channel == 2 and converts the values then via Image::ConvertDepthToFloatImage. The other accepted format is float32 as bytes_per_channel_ == 4. This is fine.

But ConvertDepthToFloatImage does float *p = output->PointerAt<float>(x, y); which accesses the data as float32 array for an uint16 image. Which looks wrong.
The final image looks good though. There must be some confusion somewhere, which I cannot find.

It should really convert the uint16 to float per value first.

Steps to reproduce the bug

    vector<uint16_t> depths;
    // ... read depths from a tiff
    auto image = make_shared<geometry::Image>();
    image->Prepare(width, height, /* num_channels */ 1, /* bytes_per_channel */ 2);
    memcpy(image->data_.data(), depths.data(), depths.size() * 2);
    open3d::camera::PinholeCameraIntrinsic intrinsics(
        width, height, intr->f[0], intr->f[1], intr->pp[0], intr->pp[1]);
    pcd = geometry::PointCloud::CreateFromDepthImage(*image, intrinsics);

Error message

No response

Expected behavior

No response

Open3D, Python and System information

- Operating system: Ubuntu 24.04
- Open3D version: latest main 553ca86cc11d57324b84eb96ae7cda5c90011091
- System architecture: x86
- Compiler version (if built from source): gcc 13

Additional information

No response

@rurban rurban added the bug Not a build issue, this is likely a bug. label Sep 11, 2024
@rurban
Copy link
Contributor Author

rurban commented Sep 12, 2024

As workaround I have to create a FloatImage and create the pixels by myself.

    geometry::Image gimage;
    auto image = gimage.CreateFloatImage();
    //o3d can only handle float images, not others. despite documented otherwise
    image->Prepare(width, height, /* num_channels */ 1, /* bytes_per_channel */ 4);
    for (int k = 0; k < depths.size(); k++) {
      const int x = k % width;
      const int y = k / width;
      float *p = image->PointerAt<float>(x, y);
      *p = depths[k] * 0.001; // mm to m
    }

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Not a build issue, this is likely a bug.
Projects
None yet
Development

No branches or pull requests

1 participant