Add the ability to get a list of memory pages

[slxdy]

My proposal is to add functions to Memory and ExternalMemory that allow for iterating through all the available memory pages to get their details such as start address, length and flags.

If I get some free time, I might start working on a PR, but for now I'm creating this issue for discussion.

[slxdy]

This would also serve as a base for the solution of an issue from SigScan:
Reloaded-Project/Reloaded.Memory.SigScan#3

[Sewer56]

Reloaded.Memory.Buffers may be more suitable, as it already has code for doing page walking on multiple OSes:

• Example https://github.com/Reloaded-Project/Reloaded.Memory.Buffers/blob/master/src/Reloaded.Memory.Buffers/Internal/BufferAllocator.Windows.cs

Perhaps lifting out the code from buffers that does the walk, and doing the page walk in a separate module/library altogether.

[slxdy]

Since Buffers relies on Memory, I do believe that this functionality should be a part of Memory, considering that this library was made to get information about and of process memory. This feature is also important because it allows the user to check which part of the memory can be read from or written to in the first place.

[slxdy]

Also, since Memory already provides the functionality to change the protection level for memory pages, it would make sense for it to also get which memory pages are available

[Sewer56]

I'd be willing to shuffle some things around.
Doing that would mean a breaking change though, but I don't mind that.

Essentially I want to lay out the library in a way such that it can be efficiently shared across AssemblyLoadContext(s) (so all load contexts don't re-JIT it and share same instance); which means breaking changes wouldn't be allowed. Right now that's not a use case, so I can still break things, at least once.

9.X was technically supposed to be that, but I have concerns that I put too much functionality into the main Reloaded.Memory library in 9.X. The binary's almost 100KB. Chucking out the streams stuff into a separate binary, as that has a lot of methods would probably be ideal there as a start.

Page walking is not a frequent operation either, which is why I'd ideally prefer to throw that into a separate binary. At least that's my take.

[slxdy]

Lots of modding tools require this functionality to perform pattern scans (so that'd be the SigScan library). It is true that you mostly wanna do those for finding functions, but what about other variables in the memory? If you can't iterate through the non-module pages, how are you going to find anything? This is my main motivation for this issue and the issue from SigScan. The libraries lack the functionality to easily find memory outside of modules.

[Sewer56]

It really depends, there are 3 different classes of modding tools here.

There's:

• Mods which live in-process
• General purpose mod tools (e.g. file format exporters/importers)
• External mod tools (e.g. input visualizers).

The main library was built for the first of the three.
The streams stuff was built for the second of the three; but it should ideally have been put in a separate sub-library like Buffers is.

When you're writing regular game mods, you shouldn't really be scanning the heap. If you're after memory which can be anywhere in the heap you must have a very, very specific special use case; for example, finding emulator memory across a huge version range of emulator; or you're building a memory scanner like cheat engine.

When writing mods for games, you should instead scan for the code which allocates some specific memory you want. Then either extract the field address that stores the pointer from the code (if it's a static and it's possible), or hook and extract it from the register. Relying on scanning the heap there is very unreliable. The game may update and have different data on the heap, or the user may inject external code (e.g. Shader injectors like ReShade on Windows); so relying on that is not recommended.

I'm not against putting this functionality in the core library for 9.X, but it's just additional context.
Some use cases are also tricky

[slxdy]

Let me give you some context on what exactly I'm trying to achieve.

I'm working on an external tool for Team Fortress 2 for advanced Discord Rich Presence on Linux.
The tool is supposed to read the current map name, match info, etc.

Let's just focus on the map name for now. The map name field is stored in one of the game modules. Now, here is the thing. On Windows, getting the name would be as simple as pattern scanning in the module itself, considering that the fields before the name field almost always have the same value (and I already found a common pattern). On Linux however, this data mapped just besides the module, but not IN the module range. So, in order to find that specific page, I at the very least need to get the pages list to find the first rw page after the module. Now, say I knew the base and length of that page, SigScan does not provide the functionality to scan custom ranges, which brings us to the other issue.

either extract the field address that stores the pointer from the code (if it's a static and it's possible), or hook and extract it from the register

Wouldn't that only complicate things? If I already have a common pattern, why get it through the code, which will most likely change in the future either way?

[Sewer56]

Sorry for delay, was eating.

through the code, which will most likely change in the future either way?

A sigscan on the code would fail either if:

• The code being scanned for directly changes
• Compiler settings are changed

Both of these are highly unlikely.
On the other hand; searching for a specific page based on 'this page usually appears after module page' could potentially fail on:

• An update of libc (glibc etc.), which provides malloc and the likes
• A kernel update (responsible for actual allocation by malloc)
• The code allocating the memory changes

That said, I think that without running code inside the game process; which I would not recommend for an online game, sigscanning is probably the best option you have; if you can't pull an address without grabbing it from the game's assembly. So you're probably doing the right thing here.

Do remember though that the page the name may be in could change due to changes in external unrelated code. Developers usually don't manually allocate pages after all, so these pages really are what the implementation of malloc in libc has most likely assigned after getting a mmap'd region from the kernel. Allocating more or less memory in some other part of the program prior to the name being allocated could potentially move the name to another page. With that in mind, I wouldn't recommend hardcoding a page size during the search.

In any case, I wouldn't mind accepting a PR for doing a page walk (via enumerator preferably, as pages change frequently). The code in Buffers library could be adjusted for this fairly easily 👍