How to guarantee that kv cache transmission finished

[HarryWu99]

I find that cudaMemcpy2DAsync is used for kv cache transmission. However, in Paraworker:migrate_blocks, there is no torch.cuda.synchronize()

DistServe/distserve/worker.py

Line 250 in 3a5c539

def migrate_blocks(

I wonder how it guarantee that kv cache transmission finished?

looking forward to your reply, thanks! @interestingLSY @PKUFlyingPig