Improve performance on large datasets

[parser-algebra-0m]

Hello,

Do you have suggestions on how to improve performance on large datasets (100K+ cells)?
I have been stuck running step2 for several days.

[huBioinfo]

Hi,

There are several choices to accelerate the learning process for samples containing more than 100,000 cells .
(1) Increase Learning_Rate=1E-3, but you must check the resulting loss file to make sure the training loss decreases normally when you increase the learning rate.
(2) Increase Loss_Cutoff=-0.2. This modification would impact the accuracy of TCNs.
(3) Decrease Num_Epoch=200 or a value which can guarantee the final loss <= -0.2. This modification could also impact the accuracy.
(4) Decrease Num_Run=10, or parallelize these runs and then execute Step3 after obtaining all results of independent runs.

[huBioinfo]

Hi, if you are still interested, we just released an extended version CytoCommunity+, which can handle large datasets with millions of cells while using limited (graphics) memory. Thanks! https://github.com/huBioinfo/CytoCommunity-plus