feat(nodes): faster heuristic resize, support resizing on upload, use it in canvas #8057
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add `heuristic_resize_fast`, which does the same thing as `heuristic_resize`, except it's about 20x faster. This is achieved by using opencv for the binary edge handling isntead of python, and checking only 100k pixels to determine what kind of image we are working with. Besides being much faster, it results in cleaner lines for resized binary canny edge maps, and has results in fewer misidentified segmentation maps. Tested against normal images, binary canny edge maps, grayscale HED edge maps, segmentation maps, and normal images. Tested resizing up and down for each. Besides the new utility function, I needed to swap the `opencv-python` dep for `opencv-contrib-python`, which includes `cv2.ximgproc.thinning`. This function accounts for a good chunk of the perf improvement.
better handling for smaller image sizes
psychedelicious
requested review from
blessedcoolant,
maryhipp,
hipsterusername,
lstein and
jazzhaiku
as code owners
github-actions
bot
added
api
python
|
You had me at spurious dingleberries. |
hipsterusername
approved these changes
May 28, 2025
psychedelicious
changed the title
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Make
heuristic_resize~20x faster. This function is a high-quality image resizing function that preserves Canny edge maps and segmentation maps much better than traditional interpolation. This function was inherited fromcontrolnet_aux.resize_toto upload endpoint. When dimensions are provided, the image is resized usingheuristic_resizeon upload.opencv-pythondep foropencv-contrib-python, which includescv2.ximgproc.thinning. This function accounts for a good chunk of the perf improvement. The previous edge thinning code was mostly implemented in python and a lot slower.A future enhancement will use this resize method for all canvas layer transformations, which will resolve #8004.
Related Issues / Discussions
n/a
QA Instructions
Before and after for
new canvas from raster layer w/ resize. Note how the layer doesn't display at a smaller size for a moment before resizing, and how the resized layer is less pixelated.Screen.Recording.2025-05-28.at.9.32.10.pm.mov
Screen.Recording.2025-05-28.at.9.34.56.pm.mov
Use the
Heuristic Resizenode to test the updated algorithm.Tested against normal images, binary canny edge maps, grayscale HED edge maps, segmentation maps, and normal images.
Tested resizing up and down for each.
Comparisons
Some comparisons on Canny edge maps. Slightly different pixels, but in practically speaking the old and new methods give the same quality results.
Input image:
Old, new at various sizes:
Merge Plan
Checklist
What's Newcopy (if doing a release after this PR)