Add hierarchical DocumentSymbol support (LSP 3.10)

[zen010101]

Summary

Implement dual-mode DocumentSymbol provider supporting both LSP 2.x flat mode and LSP 3.x hierarchical mode, enabling better IDE integration and symbol navigation.

Changes

• Add TSymbolMode enum and TSymbolBuilder class for dual-mode symbol building
• Detect client hierarchicalDocumentSymbolSupport capability during initialization
• Build hierarchical structure with parent-child relationships when supported
• Return DocumentSymbol (with range/children) or SymbolInformation (with location/containerName) based on client capability

Benefits

• Better IDE integration: Outline view shows proper nesting in VS Code
• Hierarchical navigation: Enables symbol search with Class/Method paths (e.g., TUser/GetFullName)
• Backward compatible: Clients not supporting hierarchical mode still work with flat SymbolInformation
• LSP 3.10+ compliant: Follows the official LSP specification for DocumentSymbol

Testing

Tested and verified with:

• ✅ VS Code extension - Hierarchical mode working, Outline view shows proper nesting
• ✅ Serena MCP - Hierarchical mode with depth parameter working correctly

Screenshots

VS Code Outline view showing hierarchical structure:

• Classes can be expanded/collapsed
• Methods are nested under their parent classes with proper indentation
• Clear tree structure instead of flat list

References

• LSP 3.10 DocumentSymbol specification
• Backward compatible with LSP 2.x SymbolInformation

---

This is the first PR in a series. Future work will add Property and Field symbol extraction.

[zen010101]

[zen010101]

[zen010101]

Merry Christmas to everyone! 🎄🎁

[genericptr]

I see two problems here:

1. The symbols in the toolbar below the tabs in VSCode are in some methods showing as ... where they used to show the full path. Can you test that again? I think there was a regression.
2. Nested functions are not appearing in the outline now which is another regression.

btw, @zen010101 do you know how much this interacts with #98? I didn't get a chance to review this and I see multiple problems with it which I think you're building on with DocumentSymbol changes.

Here's at least two places where the structure is wrong and are missing their parents. I seem to be remember SublimeText showed the hierarchy as Interface > TClass > Method > NestedFunc etc... if it was setup properly. My old code before that PR used to use dot notation like Interface.TClass.Method.NestedFunc which I preferred for SublimeText and I see you have a "flat" option now. Could we bring back the dot notation for that setting?

Hmmm I just tested this in VSCode and it looks much better here so maybe that PR is working better than I thought but I still see regression in your code. It's pretty clear then we need a flat version for editors that don't support the hierarchy as well. Remember the LSP was made by Microsoft so they custom built VSCode around it and not all editors are willing to have the same UI.

[genericptr]

one last comment, the outline already looks correct to me in VSCode so what really changed here? Why do we need these changes at all?

[zen010101]

About PR #98

PR #98 attempted to add hierarchical display in flat mode by hardcoding interface/implementation string constants. This approach is problematic because:

• Flat mode (SymbolInformation) shouldn't have hierarchy - it's designed for clients that don't support nesting
• The implementation mixed concepts between flat and hierarchical modes
• It didn't properly solve the breadcrumb navigation issue

My PR takes a different approach: keeping flat mode unchanged while implementing proper hierarchy only in hierarchical mode (DocumentSymbol with children).

About SublimeText

Regarding SublimeText compatibility: I don't currently have SublimeText installed. I'll download and test it in the coming days to ensure flat mode works correctly with the dot notation format you mentioned.

Why hierarchical mode is needed

1. Modern IDE expectations: LSP 3.10 introduced hierarchical DocumentSymbol specifically because flat SymbolInformation couldn't meet modern IDE navigation needs. Most LSP clients now prefer hierarchical mode when available.

2. MCP/AI agent integration: Tools like Serena MCP rely on hierarchical symbol trees for operations such as find_symbol with name paths like TClass/Method, and depth=1,2 to retrieve symbol descendants. These capabilities are impossible with flat mode's simple string-based containerName.

Flat mode still works for editors like SublimeText that only support flat lists. This PR keeps flat mode unchanged while adding proper hierarchical support for modern IDEs and AI-powered tools.

[genericptr]

PR #98 attempted to add hierarchical display in flat mode by hardcoding interface/implementation string constants. This approach is problematic because:

• Flat mode (SymbolInformation) shouldn't have hierarchy - it's designed for clients that don't support nesting
• The implementation mixed concepts between flat and hierarchical modes
• It didn't properly solve the breadcrumb navigation issue

My PR takes a different approach: keeping flat mode unchanged while implementing proper hierarchy only in hierarchical mode (DocumentSymbol with children).

Here's how it used to look using SymbolInformation. So it's a flat list but looks like Pascal code. It's a good compromise for editors that don't use trees structures. At the very least I want to bring that back and then we can put that behind an option if people really hate it for some reason.

I tried the flat mode and it uses containerName which produces two rows of text in SublimeText even though it's mostly just empty lines so I think that should be removed.

[zen010101]

That's looking pretty good. #98 should have the old code so you can compare but it's basically just the hierarchy as normal Pascal namespace rules, with the exception of interface/implementation which is why I made that strange === symbol in the list.

Btw, I had Claude make me a Sublime Text package which is something I was dreading doing before but it seems to work now. I haven't made any binaries yet so automatic downloading doesn't work. https://github.com/genericptr/pasls-sublime-text.

@genericptr Do you mean this kind of effect?

[genericptr]

There is another bug which came from #98. Do you see how TDocumentSymbol produces 3 items in the list? They are:

1. the forward class.
2. the class definition.
3. a meta symbol which encompasses all the methods in the class.

The third one makes no sense to me and I'd like to remove that since it clusters the list when searching for classes. Is that in your new PR? if it is could you please remove it?

The forward class doesn't seem useful to me but it's a real symbol. It's annoying it's the same symbol kind as the real class but I don't see anything better in TSymbolKind.

[zen010101]

There is another bug which came from #98. Do you see how TDocumentSymbol produces 3 items in the list? They are:

1. the forward class.
2. the class definition.
3. a meta symbol which encompasses all the methods in the class.

The third one makes no sense to me and I'd like to remove that since it clusters the list when searching for classes. Is that in your new PR? if it is could you please remove it?

The forward class doesn't seem useful to me but it's a real symbol. It's annoying it's the same symbol kind as the real class but I don't see anything better in TSymbolKind.

This is the show of my PR, it can be noticed that I added a namespace before the symbol like interface or implementaion, but the forward declare is there yet. What exactly do you mean by the third point (meta symbol)?

[mvancanneyt]

Please, don't commit python code. Python is really a pain to use.
Since you're using claude, have it translate the python code to pascal. It's a simple tool, it can easily do that.

[mvancanneyt]

But other than that: the changes work great in VS Code, thank you!

[zen010101]

Please, don't commit python code. Python is really a pain to use. Since you're using claude, have it translate the python code to pascal. It's a simple tool, it can easily do that.

Haha, I only used Python because I didn't succeed in implementing it with Pascal :-)
Anyway, I'll remove them in the next commit.

[genericptr]

Please, don't commit python code. Python is really a pain to use. Since you're using claude, have it translate the python code to pascal. It's a simple tool, it can easily do that.

Oh yes I forgot about those tests. So far we have zero automated tests in the project so I think we just remove those from this PR entirely and make a new dedicated issue for tests. Having just one random test for the bread crumbs in VSCode is not very useful and there's no documentation on how to run them anyways.

I'm totally opposed to python or some other scripting language if it's just a test harness to talk to the server but of course doing it in pascal is preferable.

[genericptr]

Please, don't commit python code. Python is really a pain to use. Since you're using claude, have it translate the python code to pascal. It's a simple tool, it can easily do that.

Haha, I only used Python because I didn't succeed in implementing it with Pascal :-) Anyway, I'll remove them in the next commit.

yeah I think we need to make a dedicated commit to tests and do it properly from the bottom up.

[zen010101]

But other than that: the changes work great in VS Code, thank you!

Of course, there are still some bugs that I have found ;-)
Actually, I think it works fine in Sublime Text too ;-) It just depends on whether it fits @genericptr's usual habits. 😀

[genericptr]

Here is the third "meta symbol". Notice how it selects all the methods in the class? Maybe that's useful for some reason but that's not a real symbol so we need to make sure it's removed.

[genericptr]

@genericptr Do you mean this kind of effect?

yes, thank you I think you got that right. Or at least it's nearly back to usual.

So the idea is that smFlat is selected if hierarchicalDocumentSymbolSupport is not in the capabilities? In SublimeText I see just documentSymbolProvider so in theory on SymbolInformation should be used. Is that working?

I just pulled your branch, did you include the dot notation stuff? I don't see it in there. This is what I see now in ST. I think this is the smFlat setting but could you put the dot notation behind an initialization option? The big reason for this as because ST is really just a flat list but if you do TDocumentSymbol it will also show all the methods within the class which is useful for navigation. VSCode is the only editor I've seen that really has good support for trees like the LSP allows.

[zen010101]

Here is the third "meta symbol". Notice how it selects all the methods in the class? Maybe that's useful for some reason but that's not a real symbol so we need to make sure it's removed.

The range of the previous code was too large. If he didn't do this, the breadcrumb function would behave abnormally in VS Code. The current code will only select the first method of the symbol in the implementation space in the ST but the same in the VS Code. As shown in the figure below:

However, when you press Enter on the selected symbol, it will still stay on the first line both in VS Code or Sublime Text.

[genericptr]

Actually, I think it works fine in Sublime Text too ;-) It just depends on whether it fits @genericptr's usual habits. 😀

Thanks for being patient. It's not your fault my code got deleted but now that you've done this little overhaul of the document symbols I've lost track of how to fix it. 🙏

[genericptr]

The range of the previous code was too large. If he didn't do this, the breadcrumb function would behave abnormally in VS Code. The current code will only select the first method of the symbol in the implementation space in the ST but the same in the VS Code. As shown in the figure below:

I ran your branch and yes it's fixed now. Thank you.

[mvancanneyt]

Here is the third "meta symbol". Notice how it selects all the methods in the class? Maybe that's useful for some reason but that's not a real symbol so we need to make sure it's removed.

The range of the previous code was too large. If he didn't do this, the breadcrumb function would behave abnormally in VS Code. The current code will only select the first method of the symbol in the implementation space in the ST but the same in the VS Code. As shown in the figure below:

However, when you press Enter on the selected symbol, it will still stay on the first line both in VS Code or Sublime Text.

Looking at the code: Free Pascal has a case of for strings. A little easier to read.

[genericptr]

I see another bug. See how it selected the forward (and the selection is wrong) then the second TDocumentSymbol goes to the first method?

EDIT: sorry I didn't understand your previous comment about this being needed for breadcrumbs. That's ok I guess although not ideal (probably an LSP bug or we're doing it wrong). The only problem then is the selection is wrong and if there's a forward that's the one which is used. Maybe I was wrong initially and we do need all three symbols then (forward, class, first method). I almost want to make an option to hide forwards since that's never where you want to go when searching for classes.

[genericptr]

@zen010101 here is an example of why I want dot notation back for sublime text. Notice how sublime text doesn't filter in the parents so you can't to TDocumentSymbol.Create and need to cycle through each create method. In a file with lots of classes this is very slow to jump to the method of the class you want.

VSCode has this problem too. So how can you jump to a particular method by typing? I do that all the time and find it faster then using arrows keys to walk through each method.

Here is the built-in global symbol search which uses the syntax definition. Notice how I can jump to any method? Maybe we should just make this an option to include dot notation in the name of all symbols? I would not include the interface/implementation because it's not a real type. It's more text to look at but at least you can filter the full the type name.

[zen010101]

VSCode has this problem too. So how can you jump to a particular method by typing? I do that all the time and find it faster then using arrows keys to walk through each method.

in vscode, you can type "create tdocumentsymbol" as search pattern.

[genericptr]

in vscode, you can type "create tdocumentsymbol" as search pattern.

yes that's the global search like in ST that uses the syntax definition. We have workspace symbols which use the language server. It's still not the same as filtering in the current file.

[zen010101]

in vscode, you can type "create tdocumentsymbol" as search pattern.

yes that's the global search like in ST that uses the syntax definition. We have workspace symbols which use the language server. It's still not the same as filtering in the current file.

in vscode, using “@” to search current file and "#" to search workspace files, both using the same search pattern: "create tdocumentsymbol". But in ST, there are some issues and I should take a deep look.

[genericptr]

in vscode, using “@” to search current file and "#" to search workspace files, both using the same search pattern: "create tdocumentsymbol". But in ST, there are some issues and I should take a deep look.

it doesn't seem to be able to filter methods though like # does. Same in ST, unless the words exist in the symbol name it can't be filtered.

[zen010101]

in vscode, using “@” to search current file and "#" to search workspace files, both using the same search pattern: "create tdocumentsymbol". But in ST, there are some issues and I should take a deep look.

it doesn't seem to be able to filter methods though like # does. Same in ST, unless the words exist in the symbol name it can't be filtered.

The "progressive" search in vscode requires "create tdocumentsymbol" instead of "tdocumentsymbol create". Because the internal data structure first searches for leaf nodes and then matches the path, "tdocumentsymbol" is already the root symbol. Of course, this can also be understood as a bug in the search program :-)

[genericptr]

The "progressive" search in vscode requires "create tdocumentsymbol" instead of "tdocumentsymbol create". Because the internal data structure first searches for leaf nodes and then matches the path, "tdocumentsymbol" is already the root symbol. Of course, this can also be understood as a bug in the search program :-)

yes you're right i see that now. ST doesn't support that though. I think VSCode being a Microsoft product is more highly integrated with the LSP (also Microsoft) while other editors just filter based on the symbol name. I haven't used the emacs plugin yet but I assume it's going to be similar.

[genericptr]

This thread has gotten really hard to follow at this point so let me try to break down the remaining problems that need addressing:

• In SublimeText something is wrong with the selection range for symbols. It looks like the start/end range are the same position so there is no selection, just a cursor location (I think you can see this in some of the videos above).
• As you showed me VSCode does handle these symbol names correctly but in SublimeText it's just not going to work (at least for now). Can you make an initializationOption that shows them like in the screenshot below using dot notation. This matches what the syntax definition does and how global built-in search works too. It's just TSymbolInformation with no containerName so it becomes a single row of text.
• Please remove the tests for now and open another issue if you want to implement comprehensive automatic tests.
• The symbol kind for methods is wrong, it should be method (see the screenshot below).

[zen010101]

• In SublimeText something is wrong with the selection range for symbols. It looks like the start/end range are the same position so there is no selection, just a cursor location (I think you can see this in some of the videos above).

Did you give my latest commit a test? I figure this problem ought to be gone now.

• As you showed me VSCode does handle these symbol names correctly but in SublimeText it's just not going to work (at least for now). Can you make an initializationOption that shows them like in the screenshot below using dot notation. This matches what the syntax definition does and how global built-in search works too. It's just TSymbolInformation with no containerName so it becomes a single row of text.

I need to study this further: whether removing containerName and changing all function names to the Class.Method format can achieve the effect you mentioned. From what I can see now, in the List box provided by ST's LSP: go to symbol function, each item takes up two lines. Even if there is no containerName or children information, the second line is still retained, but it is just a blank line. This cannot achieve the single-line effect of ST's internal Goto Symbol/Goto Symbol in Project.

Here is what Lazarus does when filtering in the Code Explorer window:

YES, it includes hierarchical information for the declaration of types in interface section or impl. section but only FLAT information for the method of class with Class.Method format in impl. section.

I'm considering whether to implement it like this as well we called soLazarus (TSymbolOrganization).

• Please remove the tests for now and open another issue if you want to implement comprehensive automatic tests.

I have removed the Python test code in the latest commit. Now, only the unit tests for the new features under the FPCUnit framework remain. If you don't want this either, I can remove it in the next commit.

[genericptr]

I need to study this further: whether removing containerName and changing all function names to the Class.Method format can achieve the effect you mentioned. From what I can see now, in the List box provided by ST's LSP: go to symbol function, each item takes up two lines. Even if there is no containerName or children information, the second line is still retained, but it is just a blank line. This cannot achieve the single-line effect of ST's internal Goto Symbol/Goto Symbol in Project.

You can go back and look at the old code in #98 where it got removed. It used to do that and I remember it was just SymbolInformation with no container name. If you're seeing two lines then maybe you're still using DocumentSymbol?

[genericptr]

Did you give my latest commit a test? I figure this problem ought to be gone now.

I have the latest commit "Skip forward class declarations in DocumentSymbol extraction"

see how when I select the items there's no selection range? It goes to the symbol but there's no range. Is that not what you're seeing?

[genericptr]

YES, it includes hierarchical information for the declaration of types in interface section or impl. section but only FLAT information for the method of class with Class.Method format in impl. section.

I'm considering whether to implement it like this as well we called soLazarus (TSymbolOrganization).

Just no parameters in the list because it will really pollute the filtering which is supposed to just be types and function names.

I have removed the Python test code in the latest commit. Now, only the unit tests for the new features under the FPCUnit framework remain. If you don't want this either, I can remove it in the next commit.

Yes please remove all of it and we can do that later. If we're going to have tests they should be comprehensive and have a good test framework. Thanks.

[mvancanneyt]

YES, it includes hierarchical information for the declaration of types in interface section or impl. section but only FLAT information for the method of class with Class.Method format in impl. section.
I'm considering whether to implement it like this as well we called soLazarus (TSymbolOrganization).

Just no parameters in the list because it will really pollute the filtering which is supposed to just be types and function names.

I have removed the Python test code in the latest commit. Now, only the unit tests for the new features under the FPCUnit framework remain. If you don't want this either, I can remove it in the next commit.

Yes please remove all of it and we can do that later. If we're going to have tests they should be comprehensive and have a good test framework. Thanks.

Ryan, please: I added the test framework a long time ago.
Please don't ask to undo that. We can always add tests, but removing tests : no.

[genericptr]

Ryan, please: I added the test framework a long time ago. Please don't ask to undo that. We can always add tests, but removing tests : no.

I'm sorry I never looked closely at that. Are some of his tests compatible with your code? I don't see we committed any tests so far. What I was expected is .pas files then we run LSP command and verify the output is correct.

[zen010101]

YES, it includes hierarchical information for the declaration of types in interface section or impl. section but only FLAT information for the method of class with Class.Method format in impl. section.
I'm considering whether to implement it like this as well we called soLazarus (TSymbolOrganization).

Just no parameters in the list because it will really pollute the filtering which is supposed to just be types and function names.

WIP: This is the current progress. Next, I will further check the code deleted in #98.

[mvancanneyt]

Ryan, please: I added the test framework a long time ago. Please don't ask to undo that. We can always add tests, but removing tests : no.

I'm sorry I never looked closely at that. Are some of his tests compatible with your code? I don't see we committed any tests so far. What I was expected is .pas files then we run LSP command and verify the output is correct.

That is exactly the kind of test that @zen010101 added.

But you can also have more low-level unit tests just to verify that the streaming is correct or some other aspect of the code.

No matter the kind of test, it can all be integrated in the test framework. So no need to remove that.

[genericptr]

No matter the kind of test, it can all be integrated in the test framework. So no need to remove that.

Got it, so just the python needs to be removed. Indeed we should have tests for all the various features which are implemented. Claude would go for this. :)

[zen010101]

            "initializationOptions": {
                "workspaceSymbols": true,
                "symbolMode": "flat",
                "symbolDatabase": "/tmp/symbols-sublimetext.db"
            }

symbolMode:

• auto -- Determined based on client capabilities
• flat -- Returns results in the symbolInformation[] format, removes containerName, and uses the class.method.nested approach as the symbol name, It has a good visual effect in Sublime Text.
• hierarchical -- Returns results in the documentSymbol[] format, compatible with modern LSP clients such as VS Code and Serena MCP.

[genericptr]

Great, you brought it back how it used to be.

Sorry one last thing I see now. Remember the "meta" symbol which is the first method the class in the implementation section? That feels to me like it's a hack for VSCode (for the breadcrumbs right?) but it's confusing in SublimeText because now there's always 2 duplicate classes and this comes with zero benefit to the user.

What could we do about this? My first thought was to remove it in flat mode but then we'd bread crumbs in VSCode right? Maybe that's ok though because VSCode doesn't need a flat list anyways. We could also consider just making a list of clients that block it and include Sublime Text in there (see TClients.SublimeTextLSP where I do this already in some places).

[zen010101]

You've raised an important point that actually touches on a broader architectural question I've been thinking about.

The "meta" class container in implementation section serves a specific purpose for hierarchical mode (LSP 3.10+ DocumentSymbol): it provides proper containment so that methods like TTestClassA.MethodA1 appear as children of TTestClassA, enabling correct breadcrumb navigation (e.g., implementation > TTestClassA > MethodA1).

However, as you noted, this creates issues for clients like Sublime Text that work better with flat mode. This brings us to a design decision I'd like to discuss:

Option A: LSP Protocol-First Approach

• Output LSP-compliant responses regardless of client
• Let each client handle how to display symbols
• Pros: Simpler server, consistent behavior
• Cons: Suboptimal UX for some clients

Option B: Client-Adaptive Approach

• Detect client and customize output accordingly
• Pros: Optimized UX per client
• Cons: More complexity, scattered if ClientInfo.name = ... checks

Currently we have a mix (see TClients.SublimeTextLSP usage in ApplyEdit and Completion). If we go with Option B more extensively, we should consider a proper strategy pattern rather than scattered conditionals:

  IClientAdapter = interface
    function GetSymbolMode: TSymbolMode;
    function ShouldIncludeImplementationClassContainer: Boolean;
    function GetContainerNameBehavior: TContainerNameBehavior;
    // ... other client-specific behaviors
  end;

This would allow each client's quirks (symbol filtering, selectionRange behavior, containerName handling, version nullability) to be encapsulated in one place.

[zen010101]

Happy New Year, everyone!

Here comes the first commit of 2026:

Introduce TClientProfile mechanism to manage editor-specific feature toggles,
replacing the previous symbolMode setting with a more flexible feature-based
approach.

Changes:

• Add TClientFeature enum with feature flags:

  • cfFlatSymbolMode: Use flat SymbolInformation instead of hierarchical
  • cfExcludeSectionContainers: Exclude interface/implementation namespace symbols
    (only effective when cfFlatSymbolMode is enabled)
  • cfExcludeInterfaceMethodDecls: Exclude interface method declarations
    (only effective when cfFlatSymbolMode is enabled)
  • cfExcludeImplClassDefs: Exclude implementation class definitions
    (only effective when cfFlatSymbolMode is enabled)
  • cfNullDocumentVersion: Use nil instead of 0 for document version
    (migrated from existing Sublime Text handling in ApplyEdit)
  • cfFilterTextOnly: Only set filterText in completion, not label
    (migrated from existing Sublime Text handling in Completion)

• Add TClientProfile class with profile registry:

  • Auto-detection based on clientInfo.name from initialize request
  • Built-in profiles:
    • Default (vscode): No features enabled, uses LSP defaults
    • Sublime Text LSP: cfFlatSymbolMode, cfExcludeSectionContainers,
      cfExcludeInterfaceMethodDecls, cfExcludeImplClassDefs,
      cfNullDocumentVersion, cfFilterTextOnly

• User feature customization via initializationOptions:

  • clientProfileEnableFeatures: Comma-separated list of features to enable
  • clientProfileDisableFeatures: Comma-separated list of features to disable
  • User settings are applied on top of the detected profile

  VSCode example (settings.json):

      "pascalLanguageServer.initializationOptions.clientProfileEnableFeatures": "FlatSymbolMode",
      "pascalLanguageServer.initializationOptions.clientProfileDisableFeatures": "CompletionAddBrackets"

Sublime Text example (LSP.sublime-settings):

      "initializationOptions": {
          "clientProfileEnableFeatures": "FlatSymbolMode",
          "clientProfileDisableFeatures": "FilterTextOnly"
      }

• Refactor existing code to use profile system:

  • PasLS.Symbols: Use feature flags for symbol filtering
  • PasLS.Completion: Migrate Sublime Text check to cfFilterTextOnly
  • PasLS.ApplyEdit: Migrate Sublime Text check to cfNullDocumentVersion

• Remove deprecated symbolMode configuration option

• Add comprehensive tests:

  • Tests.ClientProfile: Unit tests for profile mechanism
  • Tests.SublimeProfile: Integration tests for Sublime Text profile