Stability AI released Stable Diffusion XL (SDXL) in July 2023, marking a significant leap in image-generation quality within the open-source family. Unlike Midjourney or DALL-E, SDXL ships with downloadable weights under a permissive license, making it attractive for teams that need control over where and how inference runs.
What Changes vs SD 1.5/2.1
SDXL is a redesigned architecture. Three key differences from earlier models:
- Size: 3.5 billion parameters in U-Net (base) + 6.6 billion in refiner, vs ~900M in SD 1.5. This explains both the quality jump and the increased hardware requirements.
- Native resolution: trained at 1024×1024, vs 512×512 for SD 1.5. Images have better composition and fewer upscaling artefacts.
- Additional conditioning: SDXL takes as input not just the prompt but also the original crop size during training, reducing artefacts like duplicated limbs or illegible text.
A less-obvious practical change: SDXL handles long prompts and specific details better. SD 1.5 saturated quickly with prompts over 30-40 tokens; SDXL flows with 75+ token prompts.
Hardware Requirements
The open-source promise comes with hardware cost. SDXL runs realistically on:
- NVIDIA GPU with 12+ GB VRAM (RTX 3060 12GB minimum; ideal RTX 4090).
- Optional refiner: adds quality but doubles memory usage. Many workflows skip it after confirming base-model output is sufficient.
- CPU alternatives: theoretically possible but take minutes per image instead of seconds.
For those not wanting to manage GPUs, managed APIs — Replicate, Together AI, Stability Cloud — expose SDXL at ~0.01-0.05 USD per image depending on resolution.
Practical Comparison: SDXL vs Midjourney vs DALL-E 3
Each generator has its profile:
- SDXL: maximum technical control. Sampler, CFG scale, seed, ControlNet, custom LoRA adjustments. Ideal when you need reproducibility, style consistency across images, or integration into your own pipeline.
- Midjourney: best average aesthetic with no configuration. If you want “the pretty image by default”, Midjourney wins. Less controllable, closed, via Discord.
- DALL-E 3: best natural-language prompt adherence. If you want “an orange cat with sunglasses sitting on a red leather sofa”, DALL-E 3 interprets spatial relationships better than the other two.
No absolute winner. Product teams often test all three in parallel with the same prompts before deciding which fits their use.
Recommended Workflow
For serious SDXL use, a workflow that scales well:
- Base prompt + style as LoRA. Train a LoRA (small fine-tune, ~50-200 images) with your brand’s visual style. Then generate with base prompt + LoRA, ensuring visual consistency.
- ControlNet for composition. When you need specific layout (say, product in foreground with blurred background), ControlNet lets you condition generation with a sketch, pose skeleton, or depth map.
- Refiner for the final pass. Two phases: generate with base model (faster), then pass the best candidates through refiner (slower but better detail on faces and textures).
- Inpainting for targeted corrections. Instead of regenerating the whole image, replace only the problematic region (hands, text, specific objects).
Tools like Automatic1111 WebUI, ComfyUI, or InvokeAI wrap this flow with a UI; for production integrations, diffusers from Hugging Face gives programmatic control.
License Considerations
SDXL is published under the OpenRAIL++-M License — allows commercial use but with restrictions on generating illegal, deceptive, or harmful content. For product use, review the clauses: they limit specific cases (non-consensual sexual content, deliberate disinformation), not general use.
On the training of these models, the legal situation is active. Several lawsuits question whether training on copyrighted images without licence complies with copyright law. Outcomes will likely affect the whole ecosystem.
Relatedly, see how generative AI is changing creative disciplines and how it fits with established workflows.
Conclusion
SDXL consolidates open image generation as a competitive alternative to proprietary models. For teams wanting control, reproducibility, or predictable cost, it justifies its hardware cost. For sporadic use or without specific technical requirements, managed models remain the lowest-friction path.
Follow us on jacar.es for more on generative AI applied to image, video, and audio.
