stuck at 0b_tileLoop_deepzoom6.py when tiles a very large tiff image.

[WeiLiuY]

Hi,

I have some problems when I run 0b_tileLoop_deepzoom6.py to tile a very large tiff file (around 20GB).
In the output directory, I can see the folder contains tiles but the 0b_tileLoop_deepzoom6.py is stuck on these output and never stops. Is it stuck at generating dzi file?

The code I used:

python ~/00_preprocessing/0b_tileLoop_deepzoom6.py -s 224 -e 0 -j 10 -B 25 -N 57,22,-8,20,10,5 -o ./level_1  "level_1.tiff"

Output in the terminal:

Tiling slide: wrote 130257/130257 tiles 
end tiler
/home/weiliu/DeepPATH/DeepPATH_code/00_preprocessing/0b_tileLoop_deepzoom6.py:104: RuntimeWarning: divide by zero encountered in double_scalars
  tmp = ((Lab[:,:,i] - TileMean[i]) * (newStd[i] / TileStd[i])) + newMean[i]
/home/weiliu/DeepPATH/DeepPATH_code/00_preprocessing/0b_tileLoop_deepzoom6.py:104: RuntimeWarning: invalid value encountered in multiply
  tmp = ((Lab[:,:,i] - TileMean[i]) * (newStd[i] / TileStd[i])) + newMean[i]

I have no problem with the small tiff (less than 10GB).
My machine configure is CPU: 32GB and Memory is 256GiB.
Thanks in advance by looking at this issue.

Best,
Wei

[ncoudray]

Hi Wei,

I am sorry but I don't have a fix for this right now. Not sure what's causing it. Are all the tiles saved? For the following steps (if you use our pipeline at least), you don't need the dzi file, so if it fails at this last stage, you should still be able to move on and use the tiles

Best,
Nicolas

[WeiLiuY]

Hi Nicolas,

Thank you so much for the reply. I think it is due the memory issue.
I have tilled the small tiffs (< 10GB) by 0b_tileLoop_deepzoom6.py and running PRL for clustering at the moment.

Many thanks!

Best,
Wei