add curated Wuhan – 8989?

[jGaboardi]

We are curating and testing against all FUAs from uscuni/simplification except Wuhan – 8989.

This was probably because we started setting up over here before Wuhan was finished over there. So should we include Wuhan here in the neatnet repo and test against it?

cc @anastassiavybornova @martinfleis

xref #135

[jGaboardi]

I think I now remember that we actually don't have Wuhan here due to how much longer it takes than other FUAs? Maybe, maybe not... because I actually can't get Wuhan to run to completion now.

It seems simplified Wuhan was last run for sgeop/neatnet on October 23rd and a lot has happened since then.

Below are the details with simplify.DEBUGGING toggled.

DEBUGGING=False (default)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[32], line 1
----> 1 wuhan_simp = neatnet.simplify_network(wuhan_orig)

File [~/github_repos/uscuni/neatnet/neatnet/simplify.py:839](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/simplify.py#line=838), in simplify_network(roads, max_segment_length, min_dangle_length, clip_limit, simplification_factor, consolidation_tolerance, artifact_threshold, artifact_threshold_fallback, area_threshold_blocks, isoareal_threshold_blocks, area_threshold_circles, isoareal_threshold_circles_enclosed, isoperimetric_threshold_circles_touching, eps, exclusion_mask, predicate)
    825 artifacts, _ = get_artifacts(
    826     new_roads,
    827     threshold=threshold,
   (...)
    835     predicate=predicate,
    836 )
    838 # Loop 2
--> 839 final_roads = simplify_loop(
    840     new_roads,
    841     artifacts,
    842     max_segment_length=max_segment_length,
    843     min_dangle_length=min_dangle_length,
    844     clip_limit=clip_limit,
    845     simplification_factor=simplification_factor,
    846     consolidation_tolerance=consolidation_tolerance,
    847     eps=eps,
    848 )
    850 # This is potentially fixing some minor erroneous edges coming from Voronoi
    851 final_roads = induce_nodes(final_roads, eps=eps)

File [~/github_repos/uscuni/neatnet/neatnet/simplify.py:949](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/simplify.py#line=948), in simplify_loop(roads, artifacts, max_segment_length, min_dangle_length, clip_limit, simplification_factor, consolidation_tolerance, eps)
    939     roads = simplify_pairs(
    940         doubles,
    941         roads,
   (...)
    946         consolidation_tolerance=consolidation_tolerance,
    947     )
    948 if not clusters.empty:
--> 949     roads = simplify_clusters(
    950         clusters,
    951         roads,
    952         max_segment_length=max_segment_length,
    953         simplification_factor=simplification_factor,
    954         eps=eps,
    955         min_dangle_length=min_dangle_length,
    956         consolidation_tolerance=consolidation_tolerance,
    957     )
    959 if "coins_group" in roads.columns:
    960     roads = roads.drop(columns=[c for c in roads.columns if c.startswith("coins_")])

File [~/github_repos/uscuni/neatnet/neatnet/simplify.py:531](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/simplify.py#line=530), in simplify_clusters(artifacts, roads, max_segment_length, eps, simplification_factor, min_dangle_length, consolidation_tolerance)
    526     edges = roads.iloc[
    527         roads.sindex.query(cluster_geom, predicate="intersects")
    528     ].copy()
    530     # Clusters of 2 or more nodes and 2 or more continuity groups
--> 531     nx_gx_cluster(
    532         edges=edges,
    533         cluster_geom=cluster_geom,
    534         nodes=nodes,
    535         to_drop=to_drop,
    536         to_add=to_add,
    537         eps=eps,
    538         max_segment_length=max_segment_length,
    539         min_dangle_length=min_dangle_length,
    540         consolidation_tolerance=consolidation_tolerance,
    541     )
    543 cleaned_roads = roads.drop(to_drop)
    545 # Create new roads with fixed geometry.
    546 # Note: ``to_add`` and ``to_drop`` lists shall be global and
    547 # this step should happen only once, not for every artifact

File [~/github_repos/uscuni/neatnet/neatnet/artifacts.py:1479](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/artifacts.py#line=1478), in nx_gx_cluster(edges, cluster_geom, nodes, to_drop, to_add, max_segment_length, min_dangle_length, consolidation_tolerance, eps)
   1476     skeletonization_input = line_segments(merged_edges)
   1478 # skeletonize
-> 1479 skel, _ = voronoi_skeleton(
   1480     skeletonization_input,
   1481     cluster_geom,
   1482     snap_to=False,
   1483     max_segment_length=max_segment_length,
   1484     clip_limit=1e-4,
   1485     consolidation_tolerance=consolidation_tolerance,
   1486 )
   1488 # if we used only segments, we need to remove dangles
   1489 if len(connection) == 1:

File [~/github_repos/uscuni/neatnet/neatnet/geometry.py:273](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/geometry.py#line=272), in voronoi_skeleton(lines, poly, snap_to, max_segment_length, buffer, secondary_snap_to, clip_limit, consolidation_tolerance)
    271 edgelines = shapely.line_merge(edgelines[shapely.length(edgelines) > 0])
    272 edgelines = _as_parts(edgelines)
--> 273 edgelines = _consolidate(edgelines, consolidation_tolerance)
    275 return edgelines, np.array(splitters)

File [~/github_repos/uscuni/neatnet/neatnet/geometry.py:300](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/geometry.py#line=299), in _consolidate(edgelines, consolidation_tolerance)
    298 """Return ``edgelines`` from consolidated nodes, if criteria met."""
    299 if consolidation_tolerance and edgelines.shape[0] > 0:
--> 300     edgelines = consolidate_nodes(
    301         edgelines, tolerance=consolidation_tolerance, preserve_ends=True
    302     ).geometry.to_numpy()
    303 return edgelines

File [~/github_repos/uscuni/neatnet/neatnet/nodes.py:530](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/nodes.py#line=529), in consolidate_nodes(gdf, tolerance, preserve_ends)
    528     if c != "_status":
    529         agg[c] = "first"
--> 530 return remove_false_nodes(
    531     gdf[~gdf.geometry.is_empty].explode(),
    532     # NOTE: this aggfunc needs to be able to process all the columns
    533     aggfunc=agg,
    534 )

File [~/github_repos/uscuni/neatnet/neatnet/nodes.py:352](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/nodes.py#line=351), in remove_false_nodes(gdf, aggfunc, **kwargs)
    350 target_nodes = nodes.geometry.iloc[node_ix[loop_ix == ix]]
    351 if len(target_nodes) == 2:
--> 352     new_sequence = _rotate_loop_coords(loop_geom, not_loops)
    353     fixed_loops.append(shapely.LineString(new_sequence))
    354     fixed_index.append(ix)

File [~/github_repos/uscuni/neatnet/neatnet/nodes.py:373](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/nodes.py#line=372), in _rotate_loop_coords(loop_geom, not_loops)
    371 _shared_node = loop_points.loc[loop_points_ix].geometry.get_coordinates().values
    372 new_start = np.unique(_shared_node, axis=0)
--> 373 _coords_match = (loop_coords == new_start).all(axis=1)
    374 new_start_idx = np.where(_coords_match)[0].squeeze()
    376 rolled_coords = np.roll(loop_coords[:-1], -new_start_idx, axis=0)

ValueError: operands could not be broadcast together with shapes (5,2) (2,2)

• ValueError: operands could not be broadcast together with shapes (5,2) (2,2)

DEBUGGING=True

File [~/github_repos/uscuni/neatnet/neatnet/simplify.py:809](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/simplify.py#line=808), in simplify_network(roads, max_segment_length, min_dangle_length, clip_limit, simplification_factor, consolidation_tolerance, artifact_threshold, artifact_threshold_fallback, area_threshold_blocks, isoareal_threshold_blocks, area_threshold_circles, isoareal_threshold_circles_enclosed, isoperimetric_threshold_circles_touching, eps, exclusion_mask, predicate)
    795 artifacts, threshold = get_artifacts(
    796     roads,
    797     threshold=artifact_threshold,
   (...)
    805     predicate=predicate,
    806 )
    808 # Loop 1
--> 809 new_roads = simplify_loop(
    810     roads,
    811     artifacts,
    812     max_segment_length=max_segment_length,
    813     min_dangle_length=min_dangle_length,
    814     clip_limit=clip_limit,
    815     simplification_factor=simplification_factor,
    816     consolidation_tolerance=consolidation_tolerance,
    817     eps=eps,
    818 )
    820 # This is potentially fixing some minor erroneous edges coming from Voronoi
    821 new_roads = induce_nodes(new_roads, eps=eps)

File [~/github_repos/uscuni/neatnet/neatnet/simplify.py:931](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/simplify.py#line=930), in simplify_loop(roads, artifacts, max_segment_length, min_dangle_length, clip_limit, simplification_factor, consolidation_tolerance, eps)
    927 clusters = artifacts.loc[artifacts["comp"].isin(counts[counts > 2].index)].copy()
    929 if not singles.empty:
    930     # NOTE: this drops attributes
--> 931     roads = simplify_singletons(
    932         singles,
    933         roads,
    934         max_segment_length=max_segment_length,
    935         simplification_factor=simplification_factor,
    936         consolidation_tolerance=consolidation_tolerance,
    937     )
    938 if not doubles.empty:
    939     roads = simplify_pairs(
    940         doubles,
    941         roads,
   (...)
    946         consolidation_tolerance=consolidation_tolerance,
    947     )

File [~/github_repos/uscuni/neatnet/neatnet/simplify.py:249](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/simplify.py#line=248), in simplify_singletons(artifacts, roads, max_segment_length, compute_coins, min_dangle_length, eps, clip_limit, simplification_factor, consolidation_tolerance)
    247     except Exception as e:
    248         if DEBUGGING:
--> 249             raise e
    250         warnings.warn(
    251             f"An error occured at location {artifact.geometry.centroid}. "
    252             f"The artifact has not been simplified. The original message:\n{e}",
    253             UserWarning,
    254             stacklevel=2,
    255         )
    257 cleaned_roads = roads.drop(to_drop)

File [~/github_repos/uscuni/neatnet/neatnet/simplify.py:233](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/simplify.py#line=232), in simplify_singletons(artifacts, roads, max_segment_length, compute_coins, min_dangle_length, eps, clip_limit, simplification_factor, consolidation_tolerance)
    231 elif (n_nodes > 1) and (len(cestype) > 2):
    232     logger.debug("FUNCTION nx_gx")
--> 233     nx_gx(
    234         edges,
    235         artifact=artifact,
    236         to_drop=to_drop,
    237         to_add=to_add,
    238         split_points=split_points,
    239         nodes=nodes,
    240         max_segment_length=max_segment_length,
    241         clip_limit=clip_limit,
    242         min_dangle_length=min_dangle_length,
    243         consolidation_tolerance=consolidation_tolerance,
    244     )
    245 else:
    246     logger.debug("NON PLANAR")

File [~/github_repos/uscuni/neatnet/neatnet/artifacts.py:1218](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/artifacts.py#line=1217), in nx_gx(edges, artifact, to_drop, to_add, split_points, nodes, max_segment_length, clip_limit, min_dangle_length, consolidation_tolerance, eps)
   1210 # mutliple Cs that are not intersecting. Avoid forks on the ends of
   1211 # Voronoi. If one goes to relevant node, keep it. If not, remove both
   1212 # and replace with a new shortest connection
   1213 if (
   1214     len(connections_intersecting_c) > 1
   1215     and len(connections_intersecting_primes) == 0
   1216 ):
   1217     # this also appends to split_points
-> 1218     new_connections = avoid_forks(
   1219         highest_hierarchy,
   1220         new_connections,
   1221         relevant_targets,
   1222         artifact,
   1223         split_points,
   1224     )
   1226 # check for disconnected Cs and reconnect
   1227 new_connections = reconnect(
   1228     conts_groups, new_connections, artifact, split_points, eps
   1229 )

File [~/github_repos/uscuni/neatnet/neatnet/artifacts.py:402](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/artifacts.py#line=401), in avoid_forks(highest_hierarchy, new_connections, relevant_targets, artifact, split_points)
    400 new_connections = new_connections[(int_mask * targets_mask) | np.invert(int_mask)]
    401 cont_diss = highest_hierarchy.dissolve(highest_hierarchy.coins_group).geometry
--> 402 addition, splitters = snap_to_targets(
    403     new_connections,
    404     artifact.geometry,
    405     cont_diss[cont_diss.disjoint(shapely.union_all(new_connections))],
    406 )
    407 split_points.extend(splitters)
    408 new_connections = np.concatenate([new_connections, addition])

File [~/github_repos/uscuni/neatnet/neatnet/geometry.py:369](http://localhost:8888/~/github_repos/uscuni/neatnet/neatnet/geometry.py#line=368), in snap_to_targets(edgelines, poly, snap_to, secondary_snap_to)
    365 else:
    366     # if there is a single component, ensure it gets a shortest line to an
    367     # endpoint from each snapping target
    368     for target in snap_to:
--> 369         sl = shapely.shortest_line(components.boundary.item(), target)
    370         if _is_within(sl, poly):
    371             to_split, to_add = _split_add(sl, to_split, to_add)

File [~/github_repos/uscuni/neatnet/.pixi/envs/all/lib/python3.13/site-packages/pandas/core/base.py:418](http://localhost:8888/~/github_repos/uscuni/neatnet/.pixi/envs/all/lib/python3.13/site-packages/pandas/core/base.py#line=417), in IndexOpsMixin.item(self)
    416 if len(self) == 1:
    417     return next(iter(self))
--> 418 raise ValueError("can only convert an array of size 1 to a Python scalar")

ValueError: can only convert an array of size 1 to a Python scalar

• ValueError: can only convert an array of size 1 to a Python scalar

😓

[jGaboardi]

• consider separate nightly CI action that runs only Wuhan