Tracking issue for RFC 2137: Support defining C-compatible variadic functions in Rust (c_variadic)

[aturon]

This is a tracking issue for the RFC "Support defining C-compatible variadic functions in Rust" (rust-lang/rfcs#2137).

Steps:

• Implement the RFC (PR Support defining C compatible variadic functions #57760)
• Adjust documentation (see instructions on forge)
• Stabilization PR (see instructions on forge)

Unresolved questions

• It would be useful to have a miri implementation to nail down the semantics

We have a rough idea of how to support c-variadic arguments in miri in https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/c_variadic.20API.20and.20ABI/near/527283677:

beetrees: From my preliminary look at it, I think there'd need to be some hooks in InterpCx::init_stack_frame to allow Miri to handle the varargs.

RalfJ: ah, that part

RalfJ: okay I think that part might be best handled inside the interpreter

RalfJ: the intrinsics should be in miri

RalfJ: as in, add a new varargs field to struct Frame and initialize it in init_stack_frame as part of the general argument handling

RalfJ: that should probably be a Vec<MPlaceTy> that hold the values of these arguments (OpTy is not suited for long-term storage)

RalfJ: and then when popping a frame, they need to be freed alongside the other local variables

Resolved questions

Continuing bikeshed on the ... syntax

There has been no discussion of a different syntax in recent years.

When implementing this feature, we will need to determine whether the compiler can provide an appropriate lifetime that prevents a VaList from outliving its corresponding variadic function.

The compiler internally determines that lifetime, to make sure that a VaList<'_> does not outlive the function that created it.

Ensure that even when this gets stabilized for regular functions, it is still rejected on const fn.

This is tested in https://github.com/rust-lang/rust/blob/main/tests/ui/parser/variadic-ffi-semantic-restrictions.rs and gives the following error:

error: functions cannot be both `const` and C-variadic
  --> $DIR/variadic-ffi-semantic-restrictions.rs:33:1
   |
LL | const unsafe extern "C" fn f4_1(x: isize, _: ...) {}
   | ^^^^^ `const` because of this             ^^^^^^ C-variadic because of this

We reject c-variadic async functions in a similar way.

Issue #125431

• Poor interaction between NLL-borrowck, async, and c_variadic's ... desugaring (VaListImpl<'_>) #125431

The VaList definition introduced in #141980 circumvents this issue.

Issue #61275

• Varargs are completely unchecked if passed as generics #61275

This is an issue with c-variadic calls, not with c-variadic definitons.

Issue #141618

• VaList<'_> does not carry its ABI in its type #141618

We have some forwards-compatible ideas for how to potentially add the ABI to the type in the future. That is only required if we want to allow calling c-variadic functions with a non-system ABI. Because C only allows using the system ABI for c-variadic calls, that is unlikely to come up in practice.

What exactly are the ABI requirements around variadic calls? What are the ABI compatibility rules? Do we have to be concerned about LLVM target features affecting variadic ABI?

Variadic calls using the system ABI do not impose any additional constraints. We already handle some ABI compatibility issues using target modifiers.

Implementation history

va_arg codegen

• codegen: Fix va_list - aarch64 iOS/Windows #56599
• Properly define va_arg and va_list for aarch64-apple-darwin #78126
• implement va_arg for x86_64 systemv #141538
• implement va_arg for powerpc #141622
• use custom types to clarify arguments to emit_ptr_va_arg #141623
• match clang's va_arg assembly on arm targets #144549

VaList and its layout

• libcore: Add VaList and variadic arg handling intrinsics #49878
• tests: Simplify VaList run-make test #56396
• VaList::copy should not require a mutable ref #57311
• Support defining C compatible variadic functions #57760
• core: ensure VaList passes improper_ctypes lint #58938
• Fix segfaults in release build C-variadic fns #58985
• ffi: rename VaList::copy to VaList::with_copy #59371
• Refactor C FFI variadics to more closely match their C counterparts, and add Clone implementation #59625
• Make VaListImpl<'f> invariant over the 'f lifetime #62639
• Fix intra-docs link in core::ffi::VaList #77784
• Give VaList its own home #126921
• c_variadic: make VaList abi-compatible with C #141980

other

• Allow "C-unwind" fn to have C variadics #126843
• core: VaArgSafe is an unsafe trait #126927
• use cfg_select! to select the right VaListImpl definition #141361
• c_variadic: Add future-incompatibility warning for ... arguments without a pattern outside of extern blocks #143619
• Add #[rustc_pass_indirectly_in_non_rustic_abis] #144529
• improve c-variadic error reporting #146165
• Improve C-variadic error messages: part 2 #146342
• c-variadic: allow c-variadic inherent and trait methods #146434
• document core::ffi::VaArgSafe #146521
• implement feature(c_variadic_naked_functions) #148770
• c-variadic: bpf and spirv do not support c-variadic definitions #149557

c-variadic adjacent

• fix issue with cmse-nonsecure-entry ABI being both async and c-variadic #146551
• kcfi: only reify trait methods when dyn-compatible #146865

[plietar]

I'd like to work on this, I already have some prototype

[aturon]

Awesome @plietar! It'd probably be good to bring this up in the "middle-end" compiler working group channel, which would also be a good place to get any help you might need.

[joshtriplett]

@plietar How goes the implementation? I remember you showing a mostly complete prototype on IRC.

[thedataking]

@plietar any news to share? This is a blocker for teams working on C to Rust transpilers, so this addition would be very welcome. (I'm part of one such team).

[plietar]

Hey,
Sorry I've been busy and then forgot about this. I'll get my prototype back in shape, hopefully by this weekend.

[harpocrates]

@plietar Any update on this? Do you have a WIP branch I can check out to try / fiddle with this?

[dlrobertson]

Looks like I'm a little late to the party 😄 ... sorry about that

A few questions:

1. How would functions that use a va_list multiple times work without the ability to explicitly use va_start and va_end? Are they expected to use copy? E.g. execl typically loops through the arguments to to get argc, creats a array argv of size argc, loops through the list again populating argv, and finally call execv.

2. The structure of a va_list varies greatly between the architectures. The intrinsic functions work with the architecture specific structure bitcast to an i8*, but AFAIK we'll still need to define the structure. Which architectures will be expected to be supported in the first iteration? Or am I mistaken that we'll have to define the structure?

@plietar if you don't have the time to work on this any more or if there is any way I could help out, I'd be more than happy to do so. I haven't worked on rustc much, but I'd be happy to help however I can with the implementation of this.

[nikomatsakis]

@dlrobertson

Seems likely that @plietar doesn't have much time, though they can speak for themselves.

I've not really looked closely at what would be needed to implement this, but if you need any help, please ping me, or reach out on gitter/IRC.