Replicate· Platform

FLUX is fast and it's open source

FLUX is now much faster on Replicate, and we’ve made our optimizations open-source so you can see exactly how they work and build upon them. Here are the end-to-end speeds: FLUX.1 [schnell] at 512x512 and 4 steps: 0.29 seconds (P90: 0.49 seconds) FLUX.1 [schnell] at 1024x1024 and 4 steps: 0.72 seconds (P90: 0.95 seconds) FLUX.1 [dev] at 1024x1024 and 28 steps: 3.03 seconds (P90: 3.90 seconds) This is from the west coast of the US using the Python client. Here’s a demo of FLUX.1 [schnell]. (It’s live, just start typing!) Here’s the full app, and source code , if you’d like to check it out. How did we do it? Most of the models on Replicate are contributed by our community, but we maintain the FLUX models in collaboration with Black Forest Labs . We’ve done two main things to make FLUX faster: We optimized the model. We used Alex Redden’s flux-fp8-api as a starting point, then optimized it with torch.compile and used fast CuDNN attention kernels in the nightly Torch builds. We added a new synchronous HTTP API that makes all image models much faster on Replicate. …

Announced

10 Oct 2024 00:00 UTC