RFC: Implement language features (Lexer and Parser)

[mateusabelli]

Summary

Despite not being a programming language, HTTP exhibits certain language-like characteristics such as attribute value pairs like Content-Type: application/json, keywords like POST or GET, and a structural format. These features make it an excellent candidate for incorporating programming language features like a lexer and parser.

Possible Solution

Note Your comments are appreciated!

One possible approach to implementing HTTP as a minimal language, is by using the ply library.
With it, instead of using regex patterns like this, or file based resources to loop through and pick the corresponding item

pattern = r"\b(?!GET|POST)[A-Z]{3,7}\b"

it would be more effective doing something like this:

tokens = [
    'CODE',
    'METHOD',
    '...'
]

And let ply build language-like characteristics to help the language server provide its own features like diagnostics, completion and hover docs.

Illustration

The following image provides an example of the structured format of HTTP messages:

Image Source

[msaad7777]

The proposal suggests implementing HTTP as a minimal language using the ply library, which could help provide programming language features like a lexer and parser to HTTP messages. The suggested approach involves defining tokens that represent different language features and using them to create a structured format for HTTP messages.

Using the ply library to build HTTP as a language-like construct can help improve the parsing and analysis of HTTP messages, allowing for more efficient extraction of information from them. This can, in turn, help to provide advanced language server features like diagnostics, completion, and hover docs.

The proposed solution also includes an example of the structured format of HTTP messages, which helps demonstrate how the different language features of HTTP can be identified and parsed. The approach suggested in the proposal could make it easier to extract these features and improve the overall functionality of HTTP.

An alternative solution could be to use existing HTTP parsing libraries like http-parser or http-parser-python, which can handle the parsing of HTTP messages and provide callbacks for different message parts like headers, body, etc. These libraries can be used to extract information from HTTP messages and provide advanced functionality like message filtering and manipulation.

Overall, the proposal suggests a promising approach for building HTTP as a minimal language using the ply library, and using this approach could enhance the performance and functionality of HTTP as a communication protocol. However, it's important to consider existing solutions and libraries like http-parser when choosing an approach for parsing and analyzing HTTP messages.

[mateusabelli]

Hello @msaad7777

Thank you for sharing your thoughts I appreciate your suggestion to consider existing HTTP parsing libraries like http-parser or http-parser-python as alternative solutions for parsing and analyzing. It's always important to evaluate and weigh the advantages and disadvantages of different approaches before making a decision.

I still need to look a bit more into then to evaluate if it could fit the project, but so far I still think that ply might still be a better option due to its flexibility. I think it would fit better because it could allow us to start with a tiny parser and then evolve it into something bigger according to the project needs. I'm always open for more suggestions if you may have, please let me know what you think.