Adds IR level PGO Instrumentation options #1992
Conversation
chiragramani
changed the title
ellishg
left a comment
ellishg
left a comment
There was a problem hiding this comment.
Do you plan to add support for -ir-profile-use and -cs-profile-use?
|
Just a friendly heads-up that new changes need to be added with this process: |
Yes, I’ve added |
Thank you for the heads-up! I’ll take a look and make sure to follow that process. |
This PR introduces three new instrumentation flags and plumbs them through to IRGen: 1. `-ir-profile-generate` - enable IR-level instrumentation. 2. `-cs-profile-generate` - enable context-sensitive IR-level instrumentation. 3. `-ir-profile-use` - IR-level PGO input profdata file to enable profile-guided optimization (both IRPGO and CSIRPGO) **Context:** https://forums.swift.org/t/ir-level-pgo-instrumentation-in-swift/82123 **Swift-driver PR:** swiftlang/swift-driver#1992 **Tests and validation:** This PR includes ir level verification tests, also checks few edge-cases when `-ir-profile-use` supplied profile is either missing or is an invalid IR profile. However, for argument validation, linking, and generating IR profiles that can later be consumed by -cs-profile-generate, I’ll need corresponding swift-driver changes. Those changes are being tracked in swiftlang/swift-driver#1992
This PR introduces three new instrumentation flags and plumbs them through to IRGen: 1. `-ir-profile-generate` - enable IR-level instrumentation. 2. `-cs-profile-generate` - enable context-sensitive IR-level instrumentation. 3. `-ir-profile-use` - IR-level PGO input profdata file to enable profile-guided optimization (both IRPGO and CSIRPGO) **Context:** https://forums.swift.org/t/ir-level-pgo-instrumentation-in-swift/82123 **Swift-driver PR:** swiftlang/swift-driver#1992 **Tests and validation:** This PR includes ir level verification tests, also checks few edge-cases when `-ir-profile-use` supplied profile is either missing or is an invalid IR profile. However, for argument validation, linking, and generating IR profiles that can later be consumed by -cs-profile-generate, I’ll need corresponding swift-driver changes. Those changes are being tracked in swiftlang/swift-driver#1992
chiragramani
force-pushed
the
instr_options
branch
2 times, most recently
from
df3add1 to
4edf9dd
Compare
Hey @artemcm , I have updated the PR post following https://github.com/swiftlang/swift-driver?tab=readme-ov-file#rebuilding-optionsswift. Please let me know if you have any other suggestions. |
This PR introduces three new instrumentation flags and plumbs them through to IRGen: 1. `-ir-profile-generate` - enable IR-level instrumentation. 2. `-cs-profile-generate` - enable context-sensitive IR-level instrumentation. 3. `-ir-profile-use` - IR-level PGO input profdata file to enable profile-guided optimization (both IRPGO and CSIRPGO) **Context:** https://forums.swift.org/t/ir-level-pgo-instrumentation-in-swift/82123 **Swift-driver PR:** swiftlang/swift-driver#1992 **Tests and validation:** This PR includes ir level verification tests, also checks few edge-cases when `-ir-profile-use` supplied profile is either missing or is an invalid IR profile. However, for argument validation, linking, and generating IR profiles that can later be consumed by -cs-profile-generate, I’ll need corresponding swift-driver changes. Those changes are being tracked in swiftlang/swift-driver#1992
|
Hey @artemcm / @cachemeifyoucan , when you get a moment, could you take a look at this PR please? Thanks! |
|
Hi @artemcm / @cachemeifyoucan, would you be able to review this or loop in the appropriate POC? Appreciate it! |
| try commandLine.appendLast(.profileGenerate, from: &parsedOptions) | ||
| try commandLine.appendLast(.irProfileGenerate, from: &parsedOptions) | ||
| try commandLine.appendLast(.irProfileGenerateEQ, from: &parsedOptions) | ||
| try commandLine.appendLast(.irProfileUse, from: &parsedOptions) |
There was a problem hiding this comment.
This is a flag that points to the file. It needs path remapping if needed. Use addPathOption.
Also means this is not handled correctly in swift-frontend/scanner. See the comment I added to the other review.
There was a problem hiding this comment.
Thanks @cachemeifyoucan for looking into it. Are you suggesting something as follows
if let irProfileUsePath = parsedOptions.getLastArgument(.irProfileUse)?.asMultiple.last {
try addPathOption(option: .irProfileUse,
path: VirtualPath(path: irProfileUsePath),
to: &commandLine,
remap: jobNeedPathRemap)
}
On trying the above, I get a fatalError in CommandLineArguments.appendFlag function
fatalError("Option cannot be appended as a flag: \(option)")
I think the option .irProfileUse is of .commaJoined similar to existing flag: profileUse and would also have the same issue, do you any suggestions on how to go about it?
There was a problem hiding this comment.
Hey @cachemeifyoucan / @drodriguez ,
Can you guide me on how to go about addressing this crash please?
There was a problem hiding this comment.
I don't understand the why of the request from @cachemeifyoucan . I cannot help with that. .profileUse is also a .commaJoined flag, and it is following the same pattern a couple of lines below. Seems that it has not been a problem until now, so not sure why it should be a problem to follow this pattern now. When the addPathOption was introduced a couple of years ago it seemed to be fine to not change .profileUse.
I think addPathOption is probably not directly usable here because if I understand correctly .commaJoined means that you can add several paths into a single argument separated by commas, and addPathOption is not prepared for that (or for adding several paths). It sounds like one has to write special code to do the same thing that's happening in addPathOption when the flags are .commaJoined, which does whatever is needed with VirtualPath for each element of the input argument (not only the .last).
There was a problem hiding this comment.
Thanks @drodriguez , I share the same understanding.
@cachemeifyoucan, @artemcm please let me know what you all think and what I can do to unblock this PR?
There was a problem hiding this comment.
@cachemeifyoucan should it be done as part of this PR or as part of a follow up PR, since .profileUse is also affected, and it is part of the current code without issues? If this is the only feedback left for this PR, it will allow merging this, and handling the specifics of the new functionality in a different PR.
There was a problem hiding this comment.
Please address it here if there is no good reason. You can do it in separate commit, but it is not something you can drop on the ground and let it break for days.
There was a problem hiding this comment.
@cachemeifyoucan it has been "broken" for a couple of years for .profileUse (still "broken" on main in fact). It will not be left more "broken" than it has already been.
There was a problem hiding this comment.
Sorry, I thought you always meant .irProfileUse and didn't realize you are talking about an old existing feature.
Please file an issue about missing caching support for both profile usage. I just checked that both profile usages don't work with swift caching even from swift-frontend side so there are more works to do in the compiler before swift-driver can make it possible to work.
There was a problem hiding this comment.
That said. I no longer block this change once we have the GitHub issue to tracking the missing feature.
chiragramani
force-pushed
the
instr_options
branch
from
4edf9dd to
5f5e3dc
Compare
Update Options.swift Update as per suggestions
chiragramani
force-pushed
the
instr_options
branch
from
5f5e3dc to
5839e6c
Compare
|
Hey @cachemeifyoucan, @artemcm please let me know what you all think and what we can do to unblock this PR? |
5 participants
This PR is adding the driver changes for functionality introduced in swiftlang/swift#84335
Context: https://forums.swift.org/t/ir-level-pgo-instrumentation-in-swift/82123