You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Huge performace drop when using DeepSpeed-ZeRO2 on 4 A40 GPUs. #44
I tried to use 4 A40 GPUs (48GB each) to reproduce your results. Due to the memory limit, I applied DeepSpeed-ZeRO2. However, the performance on APPS and livecodebench dropped significantly. Do you have any suggestions?
I tried to use 4 A40 GPUs (48GB each) to reproduce your results. Due to the memory limit, I applied DeepSpeed-ZeRO2. However, the performance on APPS and livecodebench dropped significantly. Do you have any suggestions?
I'm encountering almost the same issue. Have you found a solution yet? Could it be caused by DeepSpeed?
Activity
wanglingxiang0717 commentedon Apr 10, 2025
I'm encountering almost the same issue. Have you found a solution yet? Could it be caused by DeepSpeed?