Unstable training

[Vishal-S-P]

Hi,

I am training SEED-small model on OpenWebText dataset. After few iterations of training the loss value (eval and train) explodes and training becomes unstable. Has anyone encountered this issue before?

[ZackAXue]

Hi Vishal, I faced the same problem. Not much of a clue so far. Please let me know if you solve the problem. Thanks!