Scalability beyond thousands of nodes

[CubBossa]

Maybe this should be more a discussion than an actual issue, but once the amount of nodes reaches the limit of what the server can handle to be cached all at once, the plugin might be supposed to load nodes in "chunks".

Like in editmode, it should only load nodes near the player and cache them and evict least used nodes for example.
No need to load nodes that are thousands of blocks away if only editing the ones close by.

The most difficult part would be to run a path finding. Chunks would then for example be associated with all search terms and permissions and with all edges that go into the chunk and leave the chunk, a new graph of chunks would occur. Pathfinding would only solve chunks if an approximation would tell that the shortest path could possibly go through this chunk.

[CubBossa]

Contraction Hirachies is a way to reduce the graph and only load parts. This applies for pathfinding.

For editmode a chunk system would suffice where each node is sorted in a 32x32 region or something and while the player works only according 3x3 chunks must be loaded and might already be cached.

[CubBossa]

https://jlazarsfeld.github.io/ch.150.project/sections/8-contraction/
^ Explanation for the algorithm

This processing should start whenever all editmodes are closed. When editmode is entered, it should be cancelled. Players could not use optimized pathfinding in this time.

[CubBossa]

Current performance:
1691006932666 - start graph build
1691006932908 - end graph build -> ~230ms
1691006932995 - start solving
1691006934352 - stop solving -> ~1.4s
1691006934353 - setup particles
1691006934504 - cmd output -> ~150ms

Dijkstra takes longest, because currently whole graph is being traversed, can be improved by only processing until target reached

Particles can hardly be improved, building graph can be optimized by caching, no need to always recreate when no changes were done