关于训练时候的一些问题

[hongsheng-Z]

作者您好，感谢开源您的工作。但是我在复现您代码的时候有个疑惑，请问每次执行扩散过程只在一个随机patch上扩散吗，我好像没有找到如何对patch移动的代码？也就是下面这段代码:
https://github.com/mlpc-ucsd/Patch-DM/blob/f4a9e0ad0fe83115e50f531d7bd2fbe3e326880c/diffusion/base.py#L142
按我的理解，如果original_image=[256*256]，patch_size=64，那么应该分成至少16个patch，然后在这16个patch上依次执行forward diffusion, 还是我理解有误呢？ 谢谢您

[Mq-Zhang1]

Yes, the denoise is operated on every patch. Based on your setting, there are 44 patches for one single 256256 image

• For training: learn to denoise one random patch. So in the code, we only choose one target patch. We don't need to train the model on all patches simultaneously for the whole image, which may require much more computation
• For inference: learn to denoise on all (16) patches

[hongsheng-Z]

感谢您的回复，不过我始终没有找到您是如何将多个patch合并输出为一张完整图像的代码。谢谢您的帮助！

[Mq-Zhang1]

Could refer to this function

Patch-DM/diffusion/base.py

Line 1104 in f4a9e0a

def ddim_sample_loop_progressive(

which includes how to split one image into patches

Patch-DM/diffusion/base.py

Line 1160 in f4a9e0a

img_new = rearrange(img_new, 'b c (p1 h) (p2 w) -> (b p1 p2) c h w', h = patch_size, w = patch_size)

and after the network, concatenate patches back into one image

Patch-DM/diffusion/base.py

Line 1183 in f4a9e0a

img_new = rearrange(out['sample'], '(b p1 p2) c h w -> b c (p1 h) (p2 w)', p1 = patch_num_x+1, p2 = patch_num_y+1)

Hope this helps!