Skip to content

[DON'T MERGE] Revert hip cooperative groups workaround [REDUX] #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
jithunnair-amd wants to merge 79 commits into
base: master
Choose a base branch
from
Draft

[DON'T MERGE] Revert hip cooperative groups workaround [REDUX] #22

jithunnair-amd wants to merge 79 commits into from

Conversation

@jithunnair-amd
Copy link
Collaborator

@jithunnair-amd jithunnair-amd commented Jun 29, 2021

This is a re-Merge of pull request #21 from rraminen/PR_Revert_HIP_Cooperative_Groups_Workaround, it reverts commit 5b0fac7.

Depends on new Cooperative Groups HIP headers available in ROCm4.3+

H/T: @rraminen for the work in this PR.

@jithunnair-amd
1. Disable third_party_install so flow doesn't automatically build AP…
513f861 
…EX. 2. Comment out pip uninstall command as it errors out if deepspeed isn't already installed
@jithunnair-amd
Update setup.py to hipify before building extension
ed421e9
@jithunnair-amd
Cooperative groups is not supported by HIP yet, so replace with worka…
e82fa34 
…round
@jithunnair-amd
Use ROCm APEX
bdb8421
@jithunnair-amd
Update ROCm APEX commit to get FusedLayerNorm and hipification fixes
7c0e6ac
@jithunnair-amd
Update requirements to use tensorflow-rocm package instead of tensorf…
23200d4 
…low-gpu
@jithunnair-amd
Use DeepSpeedExamples fork
c10bdcb
@jithunnair-amd
Typo
43212b3
@jithunnair-amd
Use changes_for_rocm_build branch for jithunnair-amd fork of DeepSpee…
54ad8a5 
…dExamples
@jithunnair-amd
Update ROCm APEX commit
453d501
@jithunnair-amd
Update DeepSpeedExamples commit
4454bc2
@jithunnair-amd
Update ROCm Apex commit
db28f75
@jithunnair-amd
Merge branch upstream into master
30f661e
@jithunnair-amd
Enable cooperative groups for ROCm
077638d
@jithunnair-amd
Update setup.py to build lamb extension for ROCm
66c135e
@jithunnair-amd
Do not install torch and torchvision for ROCm using pip
9379918
@jithunnair-amd
Use ROCm fork of DeepSpeedExamples
b5866a6
@jithunnair-amd
Update DeepSpeedExamples commit to use ROCm fork master branch
9c624c2
@jithunnair-amd
Update DeepSpeedExamples commit
ab6aca1
@jithunnair-amd
ROCm PyTorch can be installed in the user local area in some cases
884f08e
@jithunnair-amd
Remove requirements.txt since upstream moved it to requirements folder
17febe5
@jithunnair-amd
Add Dockerfile for ROCm
46d64e2
@jithunnair-amd
Add skips for unit tests that fail on ROCm. Current status: 72 passed…
c2d4cc0 
…, 149 skipped
@jithunnair-amd
Enable CPU adam extension for ROCm
9f0c80d
@jithunnair-amd
Install requirements as appropriate for ROCm
cb3f83a
@jithunnair-amd
Skip additional unit tests that fail on CI (but not locally)
617027f
@jithunnair-amd
Do not skip unit tests which pass with latest PyTorch
a508e62
@jithunnair-amd
Modify include files to build CPU Adam extension
3dd5e2d
@jithunnair-amd
Update setup.py for latest hipify
77cd5c3
@jithunnair-amd
Update CPU Adam header files to remove ifdefing unnecessary with late…
7f9bbeb 
…st hipify
jithunnair-amd and others added 23 commits May 12, 2021 17:29
@jithunnair-amd
Merge pull request #13 from ROCmSoftwarePlatform/IFU-master-2021-05-12
b1563d6 
IFU-master-2021-05-12
@jithunnair-amd
Update DeepSpeedExamples commit
2066405
Merge remote-tracking branch 'upstream/master'
0a87051
@jithunnair-amd
Use branch name in PR title/branch name
e827515
@jithunnair-amd
Merge pull request deepspeedai#14 from ROCmSoftwarePlatform/IFU-maste…
ae10359 
…r-2021-05-17

IFU-master-2021-05-17
@jithunnair-amd
Add email functionality
4c7a252
@rraminen
IFU-master-2021-05-27
7b900de
@rraminen
Pointed DeepSpeedExamples to latest commit after IFU
5de081e
@jithunnair-amd
Merge pull request deepspeedai#17 from rraminen/IFU_5_27
1850f88 
IFU-master-2021-05-27
@rraminen
Revert "Add patched CG headers to rocm install path (#9)"
d296665 
This reverts commit 1c69737.
@rraminen
Revert "Update headers and include_dirs to enable transformer extensi…
f50fa7b 
…on (#8)"

This reverts commit 7be71d3.
@rraminen
Added back the required code from the commits, 1c69737 and 7be71d3
2585f29
@rraminen
Revert "Cooperative Groups workaround for transformer kernels extension"
0be9645 
This reverts commit fbddd93.
@rraminen
Added defined(__HIP_PLATFORM_HCC__) to kernels code
f428da5
@rraminen
Revert "Enable cooperative groups for ROCm"
ed2ee34 
This reverts commit 077638d.
@rraminen
Enable cooperative groups for ROCm
742fd64
@rraminen
Added CuPy installation from source
1d20b14
@rraminen
Added h5py installation
f6c79ae
@jithunnair-amd
Merge pull request deepspeedai#20 from rraminen/PR_Update_Dockerfile
0cf3306 
Update dockerfile.rocm for cupy and h5py
@rraminen
hip cooperative groups functionality for coalesced_group in fused_lam…
81b744e 
…b_cuda_kernel.cu is not implemented yet
@jithunnair-amd
Merge pull request deepspeedai#21 from rraminen/PR_Revert_HIP_Coopera…
bf2979b 
…tive_Groups_Workaround

Revert hip cooperative groups workaround
@jithunnair-amd
Revert "Merge pull request deepspeedai#21 from rraminen/PR_Revert_HIP…
5b0fac7 
…_Cooperative_Groups_Workaround"

This reverts commit bf2979b, reversing
changes made to 0cf3306.
@jithunnair-amd
Re-Merge pull request deepspeedai#21 from rraminen/PR_Revert_HIP_Coop…
909f9bf 
…erative_Groups_Workaround

This reverts commit 5b0fac7.
@jithunnair-amd
Copy link
Collaborator Author

jithunnair-amd commented Feb 8, 2022

retest this please

@jithunnair-amd jithunnair-amd force-pushed the master branch 5 times, most recently from 4c6f1a5 to 8cded57 Compare October 4, 2024 21:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@rraminen rraminen

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@jithunnair-amd @rraminen @jeffdaily