Question on batch size #3
Description
Thank you for making this open source!
Why is the batch size one for the prompt phase ? -
TokenSim/TokenSim/llm/llm_engine.py
Line 337 in 3a0da97
{{ message }}
Thank you for making this open source!
Why is the batch size one for the prompt phase ? -
TokenSim/TokenSim/llm/llm_engine.py
Line 337 in 3a0da97