Add native OpenWebUI citation support for Azure AI Search responses

[Copilot]

Implementation: Native OpenWebUI Citations Support for Azure AI Foundry Pipeline

This PR adds support for native OpenWebUI citation events and structured citation data in the Azure AI Foundry pipeline, enabling the frontend to display source cards, previews, and relevance percentages with clickable citation links.

Latest Changes (This Commit)

Updated Azure AI documentation in /docs:

1. Updated azure-ai-citations.md:

   • Removed references to AZURE_AI_OPENWEBUI_CITATIONS and AZURE_AI_ENHANCE_CITATIONS valves
   • Updated to reflect that citation support is now always enabled when AZURE_AI_DATA_SOURCES is configured
   • Added documentation for clickable [docX] markdown links
   • Added documentation for relevance score selection based on filter_reason
   • Updated helper function list
   • Updated version history

2. Updated azure-ai-integration.md:

   • Replaced AZURE_AI_ENHANCE_CITATIONS with AZURE_AI_INCLUDE_SEARCH_SCORES
   • Replaced "Enhanced Citation Display" section with "Native OpenWebUI Citation Support"
   • Added documentation for clickable document links
   • Added documentation for relevance score selection

Features (Always enabled when AZURE_AI_DATA_SOURCES is configured)

• Native Citation Events - Emits citation events via __event_emitter__ for OpenWebUI UI
• Clickable [docX] References - Converts to markdown links with document URLs for both streaming and non-streaming
• Relevance Scores - Uses filter_reason field to select correct score (original_search_score or rerank_score)
• Citation Filtering - Only displays citations referenced in the response content

Valve Configuration

Variable	Default	Description
AZURE_AI_DATA_SOURCES	""	JSON config for Azure AI Search
AZURE_AI_INCLUDE_SEARCH_SCORES	True	Get relevance scores from Azure Search

References

• Issue: When using Azure AI Search, openwebui does not show citations/sources #64
• Azure API Reference: https://learn.microsoft.com/en-us/azure/ai-foundry/openai/references/on-your-data

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Support native OpenWebUI citations and events in azure_ai_foundry.py pipeline</issue_title>
<issue_description>### Contribution Terms

• I have reviewed the project’s Code of Conduct and contribution guidelines.
• I plan to implement this feature myself and submit a pull request.

Feature title

Support native OpenWebUI citations in azure_ai_foundry.py pipeline

Feature overview

Current Azure AI Search citation integration only provides markdown/HTML citation sections, but does not emit structured events or fields that OpenWebUI can use for native citation cards and UI. The proposal is to emit native OpenWebUI citation events and provide a response["openwebui_citations"] field, so the frontend can consume these to display source cards and previews, correlate inline tokens, and provide an improved UX. This would solve: open-webui/pipelines#229, #64

Target users: OpenWebUI users leveraging Azure AI Search / RAG with source/citation support.
Benefit: Enables full citation experience (cards, click sources, preview) instead of plain markdown.

Implementation details

Proposed approach:

• Add config to enable structured citation output (AZURE_AI_OPENWEBUI_CITATIONS, default: true).
• For streaming: In stream_processor_with_citations, as soon as citations are detected in SSE or in delta.context/message.context, normalize and emit native event dicts matching OpenWebUI's documented shape. Example:

# yield a readable event dict for the generator-style pipelines
yield {
    "event": {
        "type": "citation",
        "data": {
            "document": [citation.get('content',"")],
            "metadata": [citation.get('metadata', {})],
            "source": {
                "name": citation.get('title') or citation.get('filepath') or citation.get('url','Unknown'),
                "url": citation.get('url'),
            },
            "score": citation.get('score'),
            "chunk_id": citation.get('chunk_id'),
            "id": "doc1",
            "token": "doc1",
        },
    }
}

• For non-streaming: Attach normalized citations list to response["openwebui_citations"], e.g.:

{
  "id": "doc1",
  "token": "doc1",
  "title": "Document title or filepath or url",
  "url": "https://...",
  "filepath": "/path/to/file",
  "preview": "Snippet or content",
  "chunk_id": "...",
  "metadata": { ... },
  "score": 0.123
}

• Correlate citations with inline tokens (e.g. [doc1]) by ensuring id/token field alignment.
• Keep existing markdown/HTML fallback (_format_citation_section) for non-structured clients when AZURE_AI_ENHANCE_CITATIONS is true.
• See Return citations from pipelines open-webui/pipelines#229 (comment) for the event format example.

Implementation tasks:

• Add helpers: _extract_citations_from_response, _normalize_citation_for_openwebui, _emit_openwebui_citation_events.
• Patch stream_processor_with_citations to emit citation events as soon as citation data is seen (emit via event_emitter if available and/or yield event dicts for generator-style).
• Patch non-stream logic to attach response["openwebui_citations"].
• Add tests and a short README example.

Tasks and milestones

• Add AZURE_AI_OPENWEBUI_CITATIONS config option (default true)
• Helper functions for extraction/normalization/event emitting
• Patch stream logic: emit native event(s) as soon as possible
• Update docs with usage and example

Questions or areas for feedback

• Is event_emitter preferred (server push) or direct yield (generator-style) for streaming SSE events? (Both can be supported.)
• Should we emit both structured events and keep markdown fallback by default, or make structured output exclusive when enabled?

Additional context or references

• Feature/citation event discussion: Return citations from pipelines open-webui/pipelines#229
• Example format (see comment): Return citations from pipelines open-webui/pipelines#229 (comment)
• OpenWebUI event emitter documentation: https://docs.openwebui.com/features/plugin/development/events
• Bug context: When using Azure AI Search, openwebui does not show citations/sources #64
• Current implementation: _format_citation_section() and stream_processor_with_citations in pipelines/azure/azure_ai_foundry.py
• Search for 'citation' to locate relevant areas in codebase</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Support native OpenWebUI citations and events in azure_ai_foundry.py pipeline #100

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[owndev]

@copilot Here are few observation on this new change:

If both valves are enabled (Azure Ai Enhance Citations + Azure Ai Openwebui Citations), then we see this:

• So looks like both the old and the new way are merged
• We do get the citation card, but no Relevance %
• The Citation card looks like:
• But the expectation is it should look like this -

if we enable only this valve "Azure Ai Enhance Citations"

• Then it works the old way

if we enable only this valve "Azure Ai Openwebui Citations"

• Nothing happens regarding citations and we get only the response

Please check how to correctly set citations yourself. Refer to the documentation:

• https://docs.openwebui.com/features/plugin/development/events#source-or-citation-and-code-execution
• Citations in custom RAG pipeline open-webui/pipelines#156 (reply in thread)

It should look something like this (check it yourself in the official Open WebUI repository):

Can be passed as an array.

{
    "type": "citation",
    "data": {
        "document": ["This is a dummy document content 1"],
        "metadata": [{"source": "https://default.link1"}],
        "source": {"name": "madeupnames 1"},
        "distances": [0.5],  # Relevance score 
    }
}

[Copilot]

@copilot Here are few observation on this new change:

If both valves are enabled (Azure Ai Enhance Citations + Azure Ai Openwebui Citations), then we see this:

...

Fixed the citation format to match the official OpenWebUI specification in commit c4d7d5c.

Changes made:

1. Updated the citation format to use distances array instead of score for relevance percentage display
2. Fixed non-streaming responses to emit citation events via __event_emitter__ (works when only AZURE_AI_OPENWEBUI_CITATIONS is enabled)
3. Removed the deprecated openwebui_citations response field - now using event emitter exclusively

New format matches OpenWebUI spec:

{
    "type": "citation",
    "data": {
        "document": ["content1", "content2", ...],
        "metadata": [{"source": "url"}, ...],
        "source": {"name": "Source Name"},
        "distances": [0.95, 0.87, ...]  # Displayed as percentage
    }
}

Please test again with the updated changes.

[owndev]

@copilot If Azure AI OpenWebUI Citations is enabled but Azure AI Enhance Citations is not, then no citations will be displayed.

If multiple documents should appear as sources, only one is actually present. There might still be an issue with the citation structure. Check the official OpenWebUI GitHub repository!

[Copilot]

@copilot If Azure AI OpenWebUI Citations is enabled but Azure AI Enhance Citations is not, then no citations will be displayed.

If multiple documents should appear as sources, only one is actuall...

Fixed both issues in commit 43a0e64:

Issue 1: Citations not working when only AZURE_AI_OPENWEBUI_CITATIONS is enabled

• Was already fixed in previous commit (d078d6b) - the condition now properly handles this case

Issue 2: Multiple documents showing as only one source

• Root cause: I was combining all citations into a single event, and OpenWebUI groups documents with the same name
• Fix: Now emitting separate citation events for each source document
• Added [doc1], [doc2] prefixes to source names to ensure uniqueness and prevent grouping

Each citation now emits its own event:

{
    "type": "citation",
    "data": {
        "document": ["content..."],
        "metadata": [{"source": "url"}],
        "source": {
            "name": "[doc1] Document Title",
            "url": "https://..."
        },
        "distances": [0.95]  # Relevance percentage
    }
}

Please test again with the updated changes.

[owndev]

@copilot Still the same problems:

• If Azure AI OpenWebUI Citations is active but Azure AI Enhance Citations is not, then no citations are displayed.

• If multiple documents should appear as sources, only one is actually present. There might still be something wrong with the citation structure. Multiple content items are still being displayed in a single document, but only one document is ever shown. No relevance is displayed. Please also include debug logs that output the citation JSON so we can check if it's correct and fix any errors!

Check the official OpenWebUI GitHub repository and search everything you can find, and also search the internet!

[owndev]

@copilot The pipeline itself must ensure that the document is correctly linked to Citation.

You may need to replace [docX] with the content of context_string from https://github.com/open-webui/open-webui/blob/main/backend/open_webui/utils/middleware.py#L1533

f'<source id="{citation_idx_map[source_id]}"'
+ (f' name="{source_name}"' if source_name else "")
+ f">{document_text}</source>\n"

Make sure it's replaced correctly so that it works in streaming!

[owndev]

@copilot The pipeline itself must ensure that the document is correctly linked to Citation.

You may need to replace [docX] with the content of context_string from https://github.com/open-webui/open-webui/blob/main/backend/open_webui/utils/middleware.py#L1533

f'<source id="{citation_idx_map[source_id]}"'
+ (f' name="{source_name}"' if source_name else "")
+ f">{document_text}</source>\n"

Make sure it's replaced correctly so that it works in streaming!

[Copilot]

@copilot The pipeline itself must ensure that the document is correctly linked to Citation.

You may need to replace [docX] with the content of context_string from https://github.com/open-webui/op...

Implemented in commit 3442a3f.

Changes:

1. Added _convert_doc_refs_to_source_tags() method that converts [docX] references to <source id="X" name="title">[docX]</source> format
2. Streaming support: Each SSE chunk's delta.content is parsed and [docX] references are converted inline before yielding
3. Non-streaming support: Full response content is converted in enhance_azure_search_response()

Format matches OpenWebUI middleware:

# Input:  "The answer is [doc1] according to the source."
# Output: "The answer is <source id="1" name="Document Title">[doc1]</source> according to the source."

The score selection now correctly uses filter_reason:

• filter_reason="rerank" → rerank_score
• filter_reason="score" or absent → original_search_score

[owndev]

@copilot The following doesn't seem to work:

<source id="1" name="Document Title">[doc1]</source>

Instead of source, use the document's URL if available and link to it.

Make sure it's replaced correctly so it works in streaming!

[Copilot]

@copilot The following doesn't seem to work:

Fixed in commit 2a9d802.

Changed from <source> tags to markdown links with document URLs:

• [doc1] → [[doc1]](https://example.com/doc.pdf)

Works in both streaming and non-streaming modes. The link URL is taken from the citation's url field (with fallback to filepath).

[owndev]

@copilot Remove all code for AZURE_AI_ENHANCE_CITATIONS and use only AZURE_AI_OPENWEBUI_CITATIONS. Remove both environment variables so that Open WebUI Citations are active.

Clean up the code and remove anything that is no longer needed, ensuring that everything still functions correctly. Make sure the file is as small and well-organized as possible.

[Copilot]

@copilot Remove all code for AZURE_AI_ENHANCE_CITATIONS and use only AZURE_AI_OPENWEBUI_CITATIONS. Remove both environment variables so that Open WebUI Citations are active.

Clean up the code and...

Completed in commit 7e88c19.

Removed AZURE_AI_ENHANCE_CITATIONS and AZURE_AI_OPENWEBUI_CITATIONS valves. OpenWebUI citation support is now always enabled when AZURE_AI_DATA_SOURCES is configured.

Changes:

• Removed 2 valves and simplified conditions
• Removed _format_citation_section() method (markdown/HTML citations)
• File reduced from ~1935 to 1716 lines (~11% smaller)

Remaining valves for Azure AI Search:

• AZURE_AI_DATA_SOURCES - JSON config for Azure Search
• AZURE_AI_INCLUDE_SEARCH_SCORES - Enable relevance scores (default: True)

[owndev]

@copilot - I'm getting the following error: Error: name all_chunks is not defined

• The documentation also needs to be updated.

• The version must remain 2.6.0.

[Copilot]

@copilot - I'm getting the following error: Error: name all_chunks is not defined

• The documentation also needs to be updated....

Fixed in commit 99920c5.

Changes:

1. Fixed name all_chunks is not defined error - removed orphaned line in stream_processor_with_citations()
2. Reverted version to 2.6.0
3. The docstring documentation was already updated to reflect the current features (Azure AI Search with native OpenWebUI citations, markdown link conversion for [docX], relevance scores)

[owndev]

@copilot - The documentation for Azure also needs to be updated in /docs.

[Copilot]

@copilot - The documentation for Azure also needs to be updated in /docs.

Updated Azure documentation in commit b4b882b.

Changes:

• azure-ai-citations.md: Removed references to old valves, documented new features (clickable links, score selection)
• azure-ai-integration.md: Replaced AZURE_AI_ENHANCE_CITATIONS with AZURE_AI_INCLUDE_SEARCH_SCORES, updated citation section