infer_hf_stable_diffusion

Run stable diffusion models from Hugging Face.

🚀 Use with Ikomia API

1. Install Ikomia API

We strongly recommend using a virtual environment. If you're not sure where to start, we offer a tutorial here.

pip install ikomia

2. Create your workflow

from ikomia.dataprocess.workflow import Workflow
from ikomia.utils.displayIO import display

# Init your workflow
wf = Workflow()

# Add algorithm
algo = wf.add_task(name="infer_hf_stable_diffusion", auto_connect=False)

# Run  
wf.run()

# Display the image
display(algo.get_output(0).get_image())

☀️ Use with Ikomia Studio

Ikomia Studio offers a friendly UI with the same features as the API.

• If you haven't started using Ikomia Studio yet, download and install it from this page.

• For additional guidance on getting started with Ikomia Studio, check out this blog post.

📝 Set algorithm parameters

• model_name (str) - default 'stabilityai/stable-diffusion-2-base': Name of the stable diffusion model. Other model available:
  • CompVis/stable-diffusion-v1-4
  • runwayml/stable-diffusion-v1-5
  • stabilityai/stable-diffusion-2-base
  • stabilityai/stable-diffusion-2
  • stabilityai/stable-diffusion-2-1-base
  • stabilityai/stable-diffusion-2-1
  • stabilityai/stable-diffusion-xl-base-1.0
  • stabilityai/sdxl-turbo: requires Torch >= 1.13, by default it will not work on Python < 3.10
• prompt (str): Text prompt to guide the image generation.
• negative_prompt (str, optional): The prompt not to guide the image generation. Ignored when not using guidance (i.e., ignored if guidance_scale is less than 1).
• num_inference_steps (int) - default '50': Number of denoising steps (minimum: 1; maximum: 500). For 'sdxl-turbo' we recommend using between 1 and 4 steps.
• guidance_scale (float) - default '7.5': Scale for classifier-free guidance (minimum: 1; maximum: 20). For 'sdxl-turbo' guidance scale will be updated to 0.
• seed (int) - default '-1': Seed value. '-1' generates a random number between 0 and 191965535.
• width (int) - default '512': Output width. If not divisible by 8 it will be automatically modified to a multiple of 8.
• height (int) - default '512': Output height. If not divisible by 8 it will be automatically modified to a multiple of 8.
• use_refiner (bool) - default 'False': Further process the output of the base model (SDXL) with a refinement model specialized for the final denoising steps.

from ikomia.dataprocess.workflow import Workflow
from ikomia.utils.displayIO import display

# Init your workflow
wf = Workflow()

# Add algorithm
algo = wf.add_task(name = "infer_hf_stable_diffusion", auto_connect=False)

algo.set_parameters({
    'model_name': 'stabilityai/stable-diffusion-xl-base-1.0',
    'prompt': 'Astronaut on Mars during sunset',
    'guidance_scale': '7.5',
    'negative_prompt': 'low resolution',
    'num_inference_steps': '50',
    'width': '1024',
    'height': '1024',
    'seed': '1981651',
    'use_refiner': 'False'
})

# Run
wf.run()

# Display the image
display(algo.get_output(0).get_image())

🔍 Explore algorithm outputs

Every algorithm produces specific outputs, yet they can be explored them the same way using the Ikomia API. For a more in-depth understanding of managing algorithm outputs, please refer to the documentation.

from ikomia.dataprocess.workflow import Workflow

# Init your workflow
wf = Workflow()

# Add algorithm
algo = wf.add_task(name="infer_hf_stable_diffusion", auto_connect=False)

# Run 
wf.run()

# Iterate over outputs
for output in algo.get_outputs():
    # Print information
    print(output)
    # Export it to JSON
    output.to_json()