ControlNet 1.1+ Compatibility

[dunkeroni]

The newer ControlNet options available in 1.1 and onwards offer a level of complexity that the Horde is not currently able to utilize. In addition to having more model types in general, there are also preprocessors that do not use a model (e.g. reference_only mode that allows recreating subjects based on an input image) as well as models that do not rely on preprocessors (e.g. experimental Tile model that allows tile-based upscaling without introducing repetition artifacts). New models are architecturally the same as before, so ComfyUI already supports them. Just the Horde is not conducive to adding new CN features in its current state.

At minimum, the inputs for ControlNet operations should be separated configurable parts of the request, and recommended setups can be handled on the frontends. Ideally it should align with whatever plans are in place for the Horde server side to handle future Comfy pipelines.

GenerationInputStable {
  ...,
  params {
    ...,
    ComfyProcess {
      process_type: "ControlNet",
      controlnet_preprocessor: <string>,
      controlnet_model: <string>,
      controlnet_strength: <number>,
      controlnet_img: <encoded webp>
      controlnet_return_control_map: <boolean>

[tazlin]

• Requires Update controlnet annotator nodes #180