this post was submitted on 18 Jul 2023
23 points (100.0% liked)
Stable Diffusion
4249 readers
38 users here now
Discuss matters related to our favourite AI Art generation technology
Also see
Other communities
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I am no expert but have been experimenting with ComfyUI: https://lemmy.zip/post/510712
ComfyUI seems incredibly powerful and efficient, much faster than Automatic 1111. But I have yet to figure out how to get good results using ControlNet, I can make it work, but quality seems to get lost with ComfyUI and I am yet to figure out why, but expect it to be 'operator error'.
If the node based interface of ComfyUI is intimidating, then it is easy to install ComfyBox, which also lets you easily toggle between a GUI and node interface for ComfyUI: https://github.com/space-nuko/ComfyBox/blob/master/static/screenshot.png
Once I figure out the kinks I expect to transition to ComfyUI as my Stable Diffusion daily driver interface, as it is so much faster, resource efficient and configurable.
I'm no stranger to node based workflows, but I have struggled to see how nodes are beneficial for stable diffusion. It just seems like a ton of extra steps to have to lay down like 10 nodes just to make a simple image, where other interfaces let me do the same thing a lot easier.
When you say it's faster, are you referring to the workflow, or the actual generation? Do you see any other benefits from comfy?
The usefulness of ComfyUI is not just making one simple image. It is the ability to completely customize how that image is created.
For example, I have a workflow that generates a half-resolution preview image, then upscales the latent and puts it through two more sampling nodes. All three of the nodes have a different prompt input, with the focus slowly shifting to style instead of content.
I have also created a custom upscaling workflow, where the image is upscaled with normal upscaling, then re-encoded and put through just a few sampling steps, the re-encoded with a tiled VAE decoder (to save my VRAM). It creates much better results (more detail and control) than a direct ERSGAN upscale, and can even be put through ERSGAN afterward to get a super large image.
Do happen to have a tutorial for ComfyUI at hand, that you can link and that goes into some details? These custom workflows sound intriguing, but I'm not really sure where to start.
The repository links to a list of examples, but the best way to learn is just to mess around with it. It is fairly intuitive to work with (especially if you have used another node-based UI before, like Blender).
The UI also has the ability to import/export your current setup (what I call a workflow) as a json file. If I get some time, I might share some of mine with a pastebin link or something.
I just figured out that I could drag any of my images, made with A1111, into the UI and it would set up the corresponding workflow automatically. I was under the impression that this would only work for images already created with ComfyUI first. However, this gives great starting points to work with. I will play around with it tonight and see if I can extract upscaling and control-net workflows with it as a starting point from existing images.
I didn't even know that, that's pretty cool. I'll have to try it later