Should finalizers be able to hint how much memory may be freed when they run?

[wingo]

WebAssembly is a key use case for finalizers. In a system with WebAssembly and JavaScript, the total memory consumption can be approximately said to be the JS heap plus the WebAssembly heap.

Some objects in the JS heap will have finalizers which will release memory in the WebAssembly heap. One goal for the combined system is to keep total memory consumption under control, which means periodically "running GC". Generally speaking, the more JS allocates, the more the JS garbage collector runs. If JS isn't allocating much it's not worth it to run GC, so engines will put it off for a while.

However, the issue is that the GC only sees the JS allocation rate when it's making decisions. It would be nice if a library author could give a hint to the GC that e.g. a small JS allocation rate goes along with a large off-heap allocation rate, and thus that a sooner GC might be useful.

Examples of APIs of this kind:

• https://searchfox.org/mozilla-central/source/js/src/gc/GCRuntime.h#351-370
• https://chromium.googlesource.com/v8/v8.git/+/refs/heads/master/src/api.cc#7926 (not quite the same as V8 uses malloc less than SM and embedder has to keep limit)
• https://www.gnu.org/software/guile/docs/master/guile.html/Memory-Blocks.html#index-scm_005fgc_005fregister_005fallocation

Example user problems when such an API is not present:

• Memory leak with sequences emcconville/wand#300 (users having problems with python not finalizing often enough, small GC objects protecting large off-GC data)

For me, the right API here is not something that attempts to precisely measure the WebAssembly heap size, but rather something that attempts to measure off-heap allocation rate. So, like another argument to FinalizationGroup.prototype.register or something that takes a number of bytes as a parameter, as in Guile's API. I would not have a corresponding registerFree thing as it's too complicated and error-prone. Anyway, this issue is to determine if such an API is desired in the weakrefs spec, and if it is desired by engine maintainers.

NB I am not an expert GC implementor, so corrections (e.g. "lol this is not needed") are very welcome :)

[littledan]

This seems like an important consideration to me. Even if we don't make the change in this proposal, it'd be good to know that we have the capacity to add it in a follow-on.

For example, we could say that the fourth argument of register is reserved for an options bag, which, in the future, could contain this hint. (Or should this really be the third argument, and the unregisterToken is passed into the options bag?)

Is there potential for abuse of this API, though? In particular, if someone wants their finalizer to be called "deterministically", they could make it higher priority by giving it a very large size hint. Implementers may want to take care to not humor such abuse, and use this number only to prioritize among finalizers in a program, not in an absolute way.

[mathiasbynens]

cc @ulan

[gsathya]

Seems like a candidate for a follow on proposal to me

[mhofman]

For example, we could say that the fourth argument of register is reserved for an options bag, which, in the future, could contain this hint. (Or should this really be the third argument, and the unregisterToken is passed into the options bag?)

If we want to change the 3rd argument to an options bag, you'd need to do that now as unregisterToken is already an object, so a follow on proposal wouldn't be able to differentiate based on the argument's type.

[gsathya]

If we want to change the 3rd argument to an options bag, you'd need to do that now as unregisterToken is already an object, so a follow on proposal wouldn't be able to differentiate based on the argument's type.

It does seem future proof to add an options bag as the third option.

[ulan]

Since the value is controlled by JS code, I don't think we can trust it to drive the GC scheduling heuristics.
If we do, then I can imagine that it is abused to implement a "collectGarbage()" function in JS.

For the WebAssembly case I think we would need some other trusted mechanism to invoke e.g. V8::AdjustAmountOfExternalAllocatedMemory.

[littledan]

@ulan Any ideas for how we could do this for WebAssembly? My understanding was that Wasm code is generally just as untrusted as JS.

[ulan]

@littledan I don't know. You're right that Wasm code is untrusted. So it looks like we have to either expose "collectGarbage()" or do not take Wasm heap into account for GC scheduling.

[wingo]

I think it's correct that a registerMalloc() facility makes implementing a collectGarbage() in JS easier but, it's not impossible to implement currently:

function registerMalloc(n) {
  var x = [];
  const sizeofSmiArrayElement = 4;
  for (var i=0; i*sizeofSmiArrayElement<n; i++)
    x.push(42);
  return x.length;
}
function collectGarbage() { return registerMalloc(1e7); }

Of course this is subject to optimizations, etc, but the point stands in general. It's not like what we're doing here is granting capabilities that do not exist at all. It's still up to the engine when GC is scheduled, or indeed whether to run GC.

I think it's a coherent position to not take the Wasm heap into account for GC scheduling but I also think the likely consequence would be overall higher memory usage, and also more frequent out-of-memory conditions for users that could have been avoided (see python bug above). I don't know all of the tradeoffs on the implementation side to know if this is a good conclusion or not.

[wingo]

Current status is, I think, waiting on implementors to know if this kind of API would be helpful for their systems. I found it useful in Guile but YMMV, I think this needs buy-in from people that would bear the costs of maintaining this interface. OK to punt or drop also from my perspective.

[littledan]

For context, @ulan is one of the maintainers of the V8 GC.

If we are worried about trust, I wonder if we could somehow indicate, with an ArrayBufferView that indicates the subsequence "held" by the holdings. But to prevent double use, it might be more trouble in bookkeeping than the feature is worth.

Let's conclude for now that we could use the fourth argument for an options bag for these kinds of things.

[mhofman]

Interesting idea @littledan. But why restrict it to ArrayBufferView? why not have an Array or Set that lists the objects that would probably be released by the cleanup callback?

Expanding on that idea, wouldn't an implementation be able to guess/infer the amount of memory held by a user's finalizer by looking at what is reachable from the registered holding? Sure it adds complexity to the the collector, but at least it doesn't expose weird knobs to the userland.
The problem is that the holding is often reachable itself from user code, so it may not be sufficient information.
Maybe the FinalizationGroup user can provide it's memory "roots" to the constructor?