Flux is a state-of-the-art text-to-image model developed by Black Forest Labs, featuring advanced image generation capabilities. ComfyUI, on the other hand, is a powerful and modular diffusion model interface with a node-based workflow system. When combined, they create a robust solution for high-quality image generation that can be accessed programmatically.
Combining the power of Flux models with ComfyUI's versatility can significantly enhance your AI Image generation workflow. This tutorial will guide you through the process of running Flux on ComfyUI as an API, opening up possibilities for automation and integration with other systems.
Apidog offers a comprehensive API development ecosystem with features like automated testing, mock servers, and detailed documentation generation all in one unified platform.
Unlike Postman, Apidog provides seamless API lifecycle management with built-in debugging tools specifically optimized for complex workflows like those in ComfyUI. Its intuitive interface makes testing your Flux ComfyUI endpoints significantly more efficient, with collaborative features that enable team members to share and refine API configurations together.
If you're serious about API development and testing, Apidog's all-in-one solution will streamline your workflow considerably.
Setting Up Your Flux ComfyUI Environment
Before diving into the API implementation, we need to set up the necessary environment. We'll use Modal, a cloud platform that makes it easy to run serverless applications with GPU access.
Prerequisites for Flux ComfyUI
- A Modal account (create one at modal.com)
- Python 3.11 or later
- Git installed on your local machine
- Basic understanding of Python and API concepts
Building the Flux ComfyUI Container Image
Our first step is to create a container image that includes all necessary components for running Flux on ComfyUI. Let's break down the process:
import json
import subprocess
import uuid
from pathlib import Path
from typing import Dict
import modal
# Create a base image with necessary dependencies
image = (
modal.Image.debian_slim(python_version="3.11")
.apt_install("git") # Install git for cloning ComfyUI
.pip_install("fastapi[standard]==0.115.4") # Web dependencies
.pip_install("comfy-cli==1.3.5") # ComfyUI CLI tool
.run_commands(
"comfy --skip-prompt install --nvidia --version 0.3.10" # Install ComfyUI
)
)
This code creates a Modal image based on Debian Slim with Python 3.11, then installs Git, FastAPI, and comfy-cli. The comfy-cli tool is then used to install ComfyUI itself.
Enhancing Your Flux ComfyUI with Custom Nodes
To unlock additional functionality, we'll add custom nodes to our ComfyUI installation:
image = (
image.run_commands(
"comfy node install was-node-suite-comfyui@1.0.2" # Install WAS Node Suite
)
# Add more custom nodes as needed
)
The WAS Node Suite provides additional functionality for image generation and manipulation, which can be particularly useful when working with Flux models.
Setting Up Flux ComfyUI Model Download
Efficient model management is crucial for a smooth experience. We'll create a function to download the Flux model and store it in a persistent volume:
def hf_download():
from huggingface_hub import hf_hub_download
flux_model = hf_hub_download(
repo_id="Comfy-Org/flux1-schnell",
filename="flux1-schnell-fp8.safetensors",
cache_dir="/cache",
)
# Symlink the model to the ComfyUI directory
subprocess.run(
f"ln -s {flux_model} /root/comfy/ComfyUI/models/checkpoints/flux1-schnell-fp8.safetensors",
shell=True,
check=True,
)
# Create a persistent volume for model storage
vol = modal.Volume.from_name("hf-hub-cache", create_if_missing=True)
# Add the model download function to our image
image = (
image.pip_install("huggingface_hub[hf_transfer]==0.26.2")
.env({"HF_HUB_ENABLE_HF_TRANSFER": "1"})
.run_function(
hf_download,
volumes={"/cache": vol},
)
)
# Add our workflow JSON to the container
image = image.add_local_file(
Path(__file__).parent / "workflow_api.json", "/root/workflow_api.json"
)
This code defines a function to download the Flux Schnell model from Hugging Face and create a symlink to the ComfyUI models directory. We also create a Modal Volume to persist the downloaded models between runs.
Interactive Flux ComfyUI Development
Before deploying as an API, it's useful to interactively test your Flux ComfyUI workflow:
app = modal.App(name="example-comfyui", image=image)
@app.function(
allow_concurrent_inputs=10, # Needed for UI startup
max_containers=1, # Limit to one container for interactive use
gpu="L40S", # Powerful GPU for Flux model inference
volumes={"/cache": vol}, # Mount our cached models
)
@modal.web_server(8000, startup_timeout=60)
def ui():
subprocess.Popen(
"comfy launch -- --listen 0.0.0.0 --port 8000",
shell=True
)
You can run this with modal serve your_file.py and access the UI in your browser to design and test your workflows before implementing the API.
Implementing the Flux ComfyUI API
Now for the main event - turning ComfyUI into an API endpoint for Flux model inference:
@app.cls(
allow_concurrent_inputs=10, # Allow multiple concurrent API calls
scaledown_window=300, # Keep containers alive for 5 minutes after use
gpu="L40S",
volumes={"/cache": vol},
)
class ComfyUI:
@modal.enter()
def launch_comfy_background(self):
# Start ComfyUI server in the background
cmd = "comfy launch --background"
subprocess.run(cmd, shell=True, check=True)
@modal.method()
def infer(self, workflow_path: str = "/root/workflow_api.json"):
# Check ComfyUI server health
self.poll_server_health()
# Run workflow with comfy-cli
cmd = f"comfy run --workflow {workflow_path} --wait --timeout 1200 --verbose"
subprocess.run(cmd, shell=True, check=True)
# Find output image
output_dir = "/root/comfy/ComfyUI/output"
workflow = json.loads(Path(workflow_path).read_text())
file_prefix = [
node.get("inputs")
for node in workflow.values()
if node.get("class_type") == "SaveImage"
][0]["filename_prefix"]
# Return image as bytes
for f in Path(output_dir).iterdir():
if f.name.startswith(file_prefix):
return f.read_bytes()
@modal.fastapi_endpoint(method="POST")
def api(self, item: Dict):
from fastapi import Response
# Load workflow template
workflow_data = json.loads(
(Path(__file__).parent / "workflow_api.json").read_text()
)
# Insert the prompt
workflow_data["6"]["inputs"]["text"] = item["prompt"]
# Generate unique ID for this request
client_id = uuid.uuid4().hex
workflow_data["9"]["inputs"]["filename_prefix"] = client_id
# Save modified workflow
new_workflow_file = f"{client_id}.json"
json.dump(workflow_data, Path(new_workflow_file).open("w"))
# Run inference
img_bytes = self.infer.local(new_workflow_file)
return Response(img_bytes, media_type="image/jpeg")
def poll_server_health(self) -> Dict:
import socket
import urllib
try:
# Check if server is healthy
req = urllib.request.Request("<http://127.0.0.1:8188/system_stats>")
urllib.request.urlopen(req, timeout=5)
print("ComfyUI server is healthy")
except (socket.timeout, urllib.error.URLError) as e:
# Stop container if server is unhealthy
print(f"Server health check failed: {str(e)}")
modal.experimental.stop_fetching_inputs()
raise Exception("ComfyUI server is not healthy, stopping container")
This class defines the core API functionality:
launch_comfy_backgroundstarts the ComfyUI server when the container launchesinferruns a workflow and returns the generated imageapiis the FastAPI endpoint that accepts a prompt, modifies the workflow, and returns the imagepoll_server_healthensures the ComfyUI server is responsive
Testing Your Flux ComfyUI API
To test your API, you can create a simple client script:
import requests
import base64
from PIL import Image
import io
import argparse
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--prompt", default="A surreal landscape with floating islands and ethereal light")
parser.add_argument("--endpoint", default="<https://your-modal-endpoint.modal.run>")
args = parser.parse_args()
# Send request to the API
response = requests.post(
f"{args.endpoint}/api",
json={"prompt": args.prompt}
)
# Handle the response
if response.status_code == 200:
# Save and display the image
image = Image.open(io.BytesIO(response.content))
image.save("flux_output.jpg")
print(f"Image generated and saved as flux_output.jpg")
# Optional: display the image if in a suitable environment
try:
image.show()
except:
pass
else:
print(f"Error: {response.status_code}, {response.text}")
if __name__ == "__main__":
main()
Optimizing Flux ComfyUI Performance
To ensure your API remains responsive and cost-effective, consider implementing the following optimizations:
1. Memory Snapshotting for Faster Flux ComfyUI Cold Starts
Cold starts can be a bottleneck for serverless applications. Modal offers memory snapshotting that can significantly reduce startup times:
@app.cls(
allow_concurrent_inputs=10,
gpu="L40S",
volumes={"/cache": vol},
memory_snapshot=modal.MemorySnapshot(
snapshot_path="/root/comfy-snapshot",
boot_command="comfy launch --background",
),
)
class ComfyUI:
# Rest of the class implementation
2. Request Batching for Flux ComfyUI
For high-throughput scenarios, implementing request batching can improve efficiency:
@modal.method()
def batch_inference(self, prompts: list[str]):
results = []
for prompt in prompts:
# Create workflow for each prompt
client_id = uuid.uuid4().hex
workflow_data = json.loads(
(Path(__file__).parent / "workflow_api.json").read_text()
)
workflow_data["6"]["inputs"]["text"] = prompt
workflow_data["9"]["inputs"]["filename_prefix"] = client_id
# Save and process workflow
new_workflow_file = f"{client_id}.json"
json.dump(workflow_data, Path(new_workflow_file).open("w"))
img_bytes = self.infer.local(new_workflow_file)
results.append(img_bytes)
return results
Advanced Flux ComfyUI API Features
Customizing Workflow Parameters
Extend your API to allow additional parameters beyond just the prompt:
@modal.fastapi_endpoint(method="POST")
def advanced_api(self, item: Dict):
from fastapi import Response
workflow_data = json.loads(
(Path(__file__).parent / "workflow_api.json").read_text()
)
# Insert the prompt
workflow_data["6"]["inputs"]["text"] = item["prompt"]
# Optional: Set additional parameters if provided
if "negative_prompt" in item:
workflow_data["7"]["inputs"]["text"] = item["negative_prompt"]
if "cfg_scale" in item:
workflow_data["3"]["inputs"]["cfg"] = item["cfg_scale"]
if "steps" in item:
workflow_data["3"]["inputs"]["steps"] = item["steps"]
# Generate unique ID
client_id = uuid.uuid4().hex
workflow_data["9"]["inputs"]["filename_prefix"] = client_id
# Save and process workflow
new_workflow_file = f"{client_id}.json"
json.dump(workflow_data, Path(new_workflow_file).open("w"))
img_bytes = self.infer.local(new_workflow_file)
return Response(img_bytes, media_type="image/jpeg")
Handling Different Flux ComfyUI Models
Allow switching between different Flux models:
@modal.fastapi_endpoint(method="POST")
def model_selection_api(self, item: Dict):
from fastapi import Response
workflow_data = json.loads(
(Path(__file__).parent / "workflow_api.json").read_text()
)
# Insert the prompt
workflow_data["6"]["inputs"]["text"] = item["prompt"]
# Select model if specified
if "model" in item:
if item["model"] == "flux-schnell":
workflow_data["2"]["inputs"]["ckpt_name"] = "flux1-schnell-fp8.safetensors"
elif item["model"] == "flux-turbo":
workflow_data["2"]["inputs"]["ckpt_name"] = "flux1-turbo-fp8.safetensors"
# Generate unique ID
client_id = uuid.uuid4().hex
workflow_data["9"]["inputs"]["filename_prefix"] = client_id
# Save and process workflow
new_workflow_file = f"{client_id}.json"
json.dump(workflow_data, Path(new_workflow_file).open("w"))
img_bytes = self.infer.local(new_workflow_file)
return Response(img_bytes, media_type="image/jpeg")
Monitoring Your Flux ComfyUI API
Implementing proper monitoring is essential for production deployments:
@app.cls(
# Other parameters
monitor_agent=modal.MonitorAgent(),
)
class ComfyUI:
# Existing implementation
def log_request(self, prompt, model, processing_time):
# Custom logging implementation
print(f"Generated image for prompt: '{prompt}' using {model} model in {processing_time:.2f}s")
Conclusion: Unleashing the Power of Flux ComfyUI as API
By following this tutorial, you've learned how to transform ComfyUI with Flux models into a scalable, production-ready API service. This approach offers several advantages:
- Scalability: Handle multiple requests concurrently with automatic container scaling
- Flexibility: Customize workflows based on user requirements
- Integration: Connect your image generation capabilities with other applications
- Cost-effectiveness: Pay only for the compute you use
The combination of Modal's serverless infrastructure, ComfyUI's powerful workflow system, and Flux's state-of-the-art image generation creates a robust solution for a wide range of creative and business applications.
Whether you're building a creative tool, an e-commerce visualization system, or a content generation platform, running Flux on ComfyUI as an API provides the foundation for innovative AI-powered visual experiences.
