一致性解码器
[DALL·E 3] [通过更好的描述提升图像生成] [一致性模型]
改进稳定扩散VAE的解码。
安装
$ pip install git+https://github.com/openai/consistencydecoder.git
使用方法
import torch
from diffusers import StableDiffusionPipeline
from consistencydecoder import ConsistencyDecoder, save_image, load_image
# 使用稳定扩散VAE进行编码
pipe = StableDiffusionPipeline.from_pretrained(
"runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16, device="cuda:0"
)
pipe.vae.cuda()
decoder_consistency = ConsistencyDecoder(device="cuda:0") # 模型大小:2.49 GB
image = load_image("assets/gt1.png", size=(256, 256), center_crop=True)
latent = pipe.vae.encode(image.half().cuda()).latent_dist.mean
# 使用GAN解码
sample_gan = pipe.vae.decode(latent).sample.detach()
save_image(sample_gan, "gan.png")
# 使用一致性解码器解码
sample_consistency = decoder_consistency(latent)
save_image(sample_consistency, "con.png")
示例
原始图像 | GAN解码器 | 一致性解码器 |
---|---|---|