Segmentation Fault for Colored Point Cloud Registration

[aenoca]

Checklist

• I have searched for similar issues.
• For Python issues, I have tested with the latest development wheel.
• I have checked the release documentation and the latest documentation (for main branch).

Describe the issue

I have been trying to fit two colored point clouds together using colored icp and it returns a segfault every single time. I know some other issue has brought up a similar issue but for the general icp_registration.

Steps to reproduce the bug

import open3d as o3d
import numpy as np

cylinder = o3d.geometry.TriangleMesh.create_cylinder(radius=0.05, height=0.3, resolution=20, split=20)
cylinder.paint_uniform_color([0.5, 0.5, 0.5])
points = cylinder.sample_points_uniformly(3000)

# assign color to the points
points.colors = o3d.utility.Vector3dVector([np.random.rand(3) for i in range(len(points.points))])

# Visualize the pointcloud
# o3d.visualization.draw_geometries([points])

# color the cylinder with the same color as the pointcloud
kdtree = o3d.geometry.KDTreeFlann(cylinder)

# For each point in the point cloud, find the nearest vertex on the mesh
print(len(points.points), len(cylinder.vertices))
for i, point in enumerate(points.points):
    _, idx, _ = kdtree.search_knn_vector_3d(point, 1)  # KNN search for 1 nearest neighbor
    nearest_vertex_index = idx[0]

    # print(nearest_vertex_index, i, len(self.pc_colors), len(self.mesh.vertex_colors))
    
    # Transfer the color from the point cloud to the corresponding mesh vertex
    # print(idx, i)
    cylinder.vertex_colors[nearest_vertex_index] = points.colors[i]

# Visualize the colored mesh
# o3d.visualization.draw_geometries([cylinder])

# sample new points from the cylinder
new_points = cylinder.sample_points_uniformly(2500)
# rotate the cylinder
R = cylinder.get_rotation_matrix_from_xyz((0, 0, np.pi/12))
new_points.points = o3d.utility.Vector3dVector(np.asarray(new_points.points) @ R.T)

# display the new points

voxel_size = 0.001
# downsample the pointclouds
points = points.voxel_down_sample(voxel_size)
new_points = new_points.voxel_down_sample(voxel_size)
new_points.estimate_normals()
o3d.visualization.draw_geometries([points, new_points])

result = o3d.pipelines.registration.registration_icp(
    points, new_points, max_correspondence_distance=1.5*voxel_size, init=np.eye(4),
    estimation_method=o3d.pipelines.registration.TransformationEstimationForColoredICP(),
criteria=o3d.pipelines.registration.ICPConvergenceCriteria(relative_fitness=1e-6,
                                                          relative_rmse=1e-6,
                                                          max_iteration=300))

print(result.transformation)

Error message

Segmentation fault (core dumped)

Expected behavior

A print of the transformation from one pointcloud to the other (if it has converged)

Open3D, Python and System information

- Operating system: Ubuntu 22.04
- Python version: Python 3.10.12
- Open3D version: 0.18.0
- System architecture: x86 
- How did you install Open3D?: pip

Additional information

No response

[nicolaloi]

Theoretically, you could use either registration_colored_icp or registration_icp. But indeed there is a bug when using registration_icp with TransformationEstimationForColoredICP(), it seems the colored point cloud is not initialized correctly. I will open a PR, but you can already use registration_colored_icp.

[aenoca]

Okay great! I'll try that one out.

[kanao-contineu]

@aenoca what's your numpy version?