When generating: RuntimeError: CUDA out of memory

[GitHubGeniusOverlord]

Hi, I am running on a GeForce RTX3070 with 8GB.
I can train the model if I adjust the sample_size, which I interpret as batch_size.

However when generating, this parameter obviously does not help.
duration = generator.generate()
4885/14662 samples are generated. Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 39, in generate wavenet\model.py", line 71, in generate outputs = self.net(inputs) RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 8.00 GiB total capacity; 6.95 GiB already allocated; 0 bytes free; 7.31 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
Any Tips on how to avoid this? Thanks!

[GitHubGeniusOverlord]

The model is unable to repeat generation of output more than once. I reduced the generate - function to a repetition of the same output, like this:
def generate(self): inputs, audio_length = self._get_seed_from_audio(self.args.seed) i = 0 while True: print(i) print(inputs) new = self.wavenet.generate(inputs) print(new) i+=1 if i == 4: break
This should run four times and generate always the same output. However, it only runs once and then goes into the CUDA out of memory error.