Training DDPM with WPUQ Dataset

[fl-llm]

I am trying to reproduce the results reported for the WPuQ dataset in your paper. My current configuration is as follows:

model_class=gpt-2
batch_size=256
num_sample=4000
num_sampling_step=100
decoder_layer=12
dim_base=512
dim_feedforward=2048
learning_rate=1e-4
train_season=whole_year
num_train_steps=100000
Data preprocessing is performed using preprocess_hp.py, and training is conducted with train_ddpm_pl.py.

The data settings are:
record_year=[2018, 2019, 2020]
resolution=1min
However, the evaluation metrics I obtained differ significantly from those reported in Table II of the paper.

Could you please advise whether this discrepancy is likely due to incorrect hyperparameter settings or issues in the data preprocessing pipeline?

[sentient-codebot]

Judging by the numbers, I think the model probably did not converge or manage to learn anything useful at all or the sampling went wrong. How did the training go? It could be multiple reasons.

• Data processing
• Training
• Sampling

To rule out the actual issue, try plotting out the processed data first. They should be within -1 to 1 range (as I recall?), and check the shapes of the data so see if they make sense.

Then you should check the training dynamics. See if the loss descends. It should converge to a very small number, the exact value I forgot. At least 10^{-2} order, or two/three orders lower.

The sampling process can also lead to errors. If the DPM solver didn't work, you can try ancestral sampling, which is easier to diagnose. Usually when sampling goes wrong, you can observe the generated data looks like pure noise. It's quite distinguishable.