Const stability checks v2 #131349

RalfJung · 2024-10-07T07:01:06Z

The const stability system has served us well ever since const fn were first stabilized. It's main feature is that it enforces recursive validity -- a stable const fn cannot internally make use of unstable const features without an explicit marker in the form of #[rustc_allow_const_fn_unstable]. This is done to make sure that we don't accidentally expose unstable const features on stable in a way that would be hard to take back. As part of this, it is enforced that a #[rustc_const_stable] can only call #[rustc_const_stable] functions. However, some problems have been coming up with increased usage:

It is baffling that we have to mark private or even unstable functions as #[rustc_const_stable] when they are used as helpers in regular stable const fn, and often people will rather add #[rustc_allow_const_fn_unstable] instead which was not our intention.
The system has several gaping holes: a private const fn without stability attributes whose inherited stability (walking up parent modules) is #[stable] is allowed to call arbitrary unstable const operations, but can itself be called from stable const fn. Similarly, #[allow_internal_unstable] on a macro completely bypasses the recursive nature of the check.

Fundamentally, the problem is that we have three disjoint categories of functions, and not enough attributes to distinguish them:

const-stable functions
private/unstable functions that are meant to be callable from const-stable functions
functions that can make use of unstable const features

Functions in the first two categories cannot use unstable const features and they can only call functions from the first two categories.

This PR implements the following system:

#[rustc_const_stable] puts functions in the first category. It may only be applied to #[stable] functions.
#[rustc_const_unstable] by default puts functions in the third category. The new attribute #[rustc_const_stable_indirect] can be added to such a function to move it into the second category.
const fn without a const stability marker are in the second category if they are still unstable. They automatically inherit the feature gate for regular calls, it can now also be used for const-calls.

Also, all the holes mentioned above have been closed. There's still one potential hole that is hard to avoid, which is when MIR building automatically inserts calls to a particular function in stable functions -- which happens in the panic machinery. Those need to be manually marked #[rustc_const_stable_indirect] to be sure they follow recursive const stability. But that's a fairly rare and special case so IMO it's fine.

The net effect of this is that a #[unstable] or unmarked function can be constified simply by marking it as const fn, and it will then be const-callable from stable const fn and subject to recursive const stability requirements. If it is publicly reachable (which implies it cannot be unmarked), it will be const-unstable under the same feature gate. Only if the function ever becomes #[stable] does it need a #[rustc_const_unstable] or #[rustc_const_stable] marker to decide if this should also imply const-stability.

Adding #[rustc_const_unstable] is only needed for (a) functions that need to use unstable const lang features (including intrinsics), or (b) #[stable] functions that are not yet intended to be const-stable. Adding #[rustc_const_stable] is only needed for functions that are actually meant to be directly callable from stable const code. #[rustc_const_stable_indirect] is used to mark intrinsics as const-callable and for #[rustc_const_unstable] functions that are actually called from other, exposed-on-stable const fn. No other attributes are required.

Also see the updated dev-guide at rust-lang/rustc-dev-guide#2098.

I think in the future we may want to tweak this further, so that in the hopefully common case where a public function's const-stability just exactly mirrors its regular stability, we never have to add any attribute. But right now, once the function is stable this requires #[rustc_const_stable].

Open question

There is one point I could see we might want to do differently, and that is putting #[rustc_const_unstable] functions (but not intrinsics) in category 2 by default, and requiring an extra attribute for #[rustc_const_not_exposed_on_stable] or so. This would require a bunch of extra annotations, but would have the advantage that turning a #[rustc_const_unstable] into #[rustc_const_stable] will never change the way the function is const-checked. Currently, we often discover in the const stabilization PR that a function needs some other unstable const things, and then we rush to quickly deal with that. In this alternative universe, we'd work towards getting rid of the rustc_const_not_exposed_on_stable before stabilization, and once that is done stabilization becomes a trivial matter. #[rustc_const_stable_indirect] would then only be used for intrinsics.

I think I like this idea, but might want to do it in a follow-up PR, as it will need a whole bunch of annotations in the standard library. Also, we probably want to convert all const intrinsics to the "new" form (#[rustc_intrinsic] instead of an extern block) before doing this to avoid having to deal with two different ways of declaring intrinsics.

Cc @rust-lang/wg-const-eval @rust-lang/libs-api
Part of #129815 (but not finished since this is not yet sufficient to safely let us expose const fn from hashbrown)
Fixes #131073 by making it so that const-stable functions are always stable

try-job: test-various

rustbot · 2024-10-07T07:01:15Z

r? @cjgillot

rustbot has assigned @cjgillot.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

RalfJung · 2024-10-07T13:34:57Z

library/core/src/iter/traits/collect.rs

    fn into_iter(self) -> Self::IntoIter;
 }

-#[rustc_const_unstable(feature = "const_intoiterator_identity", issue = "90603")]


I found two remaining rustc_const_unstable attributes on impl blocks, I think those should all have been removed with the const trait purge and were probably just forgotten.

RalfJung · 2024-10-07T13:55:59Z

I have re-done the way we do const stability inheritance, by simply fetching the required data from the stab_map/const_stab_map after all other inheritance is done. This is a bit silly, but I'd rather not completely refactor the stability code here -- one would hope the code first computes the overall stability and then puts it in the stab_map, but no, it has various different paths that each put a different thing into stab_map. This at least ensures we are using the same stability information as the rest of the compiler when computing the const stability info.

I have updated the PR description to match the new version.

RalfJung · 2024-10-07T14:10:44Z

We have a slightly bigger problem with the proc_macro_harness. Took me a bit to figure out what happens...

The harness synthesizes calls to const fn from the proc_macro library:

rust/compiler/rustc_builtin_macros/src/proc_macro_harness.rs

Lines 256 to 272 in c682aa1

    
           // Creates a new module which looks like: 
        
           // 
        
           //      const _: () = { 
        
           //          extern crate proc_macro; 
        
           // 
        
           //          use proc_macro::bridge::client::ProcMacro; 
        
           // 
        
           //          #[rustc_proc_macro_decls] 
        
           //          #[used] 
        
           //          #[allow(deprecated)] 
        
           //          static DECLS: &[ProcMacro] = &[ 
        
           //              ProcMacro::custom_derive($name_trait1, &[], ::$name1); 
        
           //              ProcMacro::custom_derive($name_trait2, &["attribute_name"], ::$name2); 
        
           //              // ... 
        
           //          ]; 
        
           //      } 
        
           fn mk_decls(cx: &mut ExtCtxt<'_>, macros: &[ProcMacro]) -> P<ast::Item> {

It carefully computes a span for the identifier that has allow_internal_unstable(proc_macro_internals):

rust/compiler/rustc_builtin_macros/src/proc_macro_harness.rs

Lines 273 to 279 in c682aa1

    
           let expn_id = cx.resolver.expansion_for_ast_pass( 
        
               DUMMY_SP, 
        
               AstPass::ProcMacroHarness, 
        
               &[sym::rustc_attrs, sym::proc_macro_internals], 
        
               None, 
        
           ); 
        
           let span = DUMMY_SP.with_def_site_ctxt(expn_id.to_expn_id());

However, the span it sets for the call expression is different, it is this one:

rust/compiler/rustc_builtin_macros/src/proc_macro_harness.rs

Lines 299 to 302 in c682aa1

    
           let span = match m { 
        
               ProcMacro::Derive(m) => m.span, 
        
               ProcMacro::Attr(m) | ProcMacro::Bang(m) => m.span, 
        
           };

The const stability check does not have access to the span of the identifer, it only has the span of the call expression. So when we check whether that span allows the desired call, we error. Previously the const stability check had a bug where it didn't consider this call at all, but now that bug is fixed.

@petrochenkov maybe you can help us with this... is there a deep reason that the synthesized call expression in the proc macro static DECLS uses a different span than the harness_span that is used for the identifiers? The way things are set up right now, I don't think it is possible for this code to be accepted with a non-buggy const stability checker, because MIR building does not preserve the identifier spans.

RalfJung · 2024-10-07T15:29:08Z

@bors try
@rust-timer queue

bors · 2024-10-07T15:30:21Z

⌛ Trying commit eb8d6c1 with merge eddc24c...

bors · 2024-10-07T17:16:16Z

☀️ Try build successful - checks-actions
Build commit: eddc24c (eddc24c69611bc710799441c01201e42db2ed8cc)

compiler-errors

r=me after this try build is done of course

RalfJung · 2024-10-25T20:27:56Z

The try build is done. :)

@bors r=compiler-errors rollup=never

bors · 2024-10-25T20:27:59Z

📌 Commit 8849ac6 has been approved by compiler-errors

It is now in the queue for this repository.

bors · 2024-10-25T23:29:45Z

⌛ Testing commit 8849ac6 with merge 54761cb...

bors · 2024-10-26T01:59:09Z

☀️ Test successful - checks-actions
Approved by: compiler-errors
Pushing 54761cb to master...

rust-timer · 2024-10-26T03:17:45Z

Finished benchmarking commit (54761cb): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.1%	[-0.1%, -0.1%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.1%	[-0.1%, -0.1%]	1

Max RSS (memory usage)

Results (primary 0.1%, secondary -2.1%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.5%	[1.5%, 1.5%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.2%	[-1.2%, -1.2%]	1
Improvements ✅ (secondary)	-2.1%	[-3.2%, -1.2%]	5
All ❌✅ (primary)	0.1%	[-1.2%, 1.5%]	2

Cycles

Results (primary -1.1%, secondary 2.1%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.1%	[2.1%, 2.1%]	1
Improvements ✅ (primary)	-1.1%	[-1.1%, -1.1%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-1.1%	[-1.1%, -1.1%]	1