x/pkgsite: Implement HTTP/MCP API for pkg.go.dev

[ethanalee-work]

New HTTP/MCP API for pkg.go.dev

We are in the process of designing and implementing HTTP and MCP API access for pkg.go.dev.

Introduction

Introducing a new pkgsite API for use via HTTP and MCP protocol has been a feature that has long been requested by the community. It is a vital resource for Go developers, but without an official API, external services and tools must resort to unreliable methods such as web scraping. Convenient access to Go package information would make Go developers more productive.

Motivations

A new API surface addresses the following high-impact user needs:

• Discovering Package Importers: The most requested feature is the ability to discern which packages import a given package. This is vital for maintainers to understand their use base and the potential of breaking changes.
• Programmatic Search: A search endpoint that returns structured JSON data would allow tools to find relevant packages based on keywords, similar to the search functionality already provided on pkgsite.
• Accessing Package Metadata: Users will be able to access a variety of metadata about packages including available versions.
• AI and LLM Integration: Providing MCP access will enable LLMs to generate code that uses the latest and most compatible library versions.
• Enhance Context Engineering: In a future of agentic-powered workflows, making use of limited context windows will be important. Instead of relying on embedding information beforehand, we can access it “just in time”. By providing a way to access pkgsite via MCP tools, we can provide high-quality, fresh data to agents.

API

The pkgsite API provides programmatic access to Go package information through both an HTTP and an MCP interface. The HTTP API is hosted at pkg.go.dev/api, and the MCP API is at pkg.go.dev/mcp.

ModuleInfo

This endpoint retrieves detailed information about a specific module version, including details about the module, its licenses, and the packages it contains. Users will be able to include various optional request structs so that the response will only contain requested data.

• Route: POST /api/ModuleInfo
• Request Body:

  {
    "Path": "golang.org/x/text",
    "Version": "v0.3.7",
    "Details": null,
    "Licenses": null,
    "Packages": null
  }

• MCP Tool Call:

  pkgsite.ModuleInfo(
    Path='golang.org/x/text',
    Version='v0.3.7',
    Details=True,
    Licenses=True,
    Packages=True
  )
  

MajorVersions

This endpoint returns a list of all major versions for a given module. This is particularly useful for discovering non-contiguous major versions that are not easily found through the Go proxy alone.

• Route: POST /api/MajorVersions
• Request Body:

  {
    "ModulePath": "test.org/zap",
    "ExcludePseudo": true,
    "Max": 10
  }

• MCP Tool Call:

  pkgsite.MajorVersions(Path='test.org/zap')
  

Search

This endpoint allows you to search for packages using a query string and returns a list of matching results.

• Route: POST /api/Search
• Request Body:

  {
    "Query": "http client",
    "Max": 100,
  }

• MCP Tool Call:

  pkgsite.Search(Query='http client')