Skip to content

Conversation

@twaugh
Copy link
Owner

@twaugh twaugh commented Nov 23, 2025

Enhanced the Phase 3 integration prompt to ensure the LLM always provides at least one suggestion, and preferably multiple (2-3) suggestions when there are valid alternatives.

Changes to System Prompt:

  1. Added explicit requirement to ALWAYS output at least one decision
  2. Set preference for 2-3 decisions when multiple valid placements exist
  3. Removed hard confidence threshold - if all candidates < 0.30, still output the best match with its actual confidence score
  4. Reorganized prompt structure with reasoning-first format (matching classification prompt pattern)
  5. Added detailed reasoning guidelines with 4-step decision framework
  6. Changed system prompt role from 'JSON-only integration planner' to 'knowledge organization expert specializing in Logseq integration'. This leads with expertise/capability rather than constraints, creating a stronger professional identity that may improve LLM performance.

Changes to User Prompt (Examples):

  1. Replaced "no output" examples with positive examples showing:
    • Best match even with weak semantic fit (confidence 0.55)
    • Multiple suggestions for ambiguous content (3 suggestions)
  2. Expanded examples from 5 to 7 with richer reasoning chains
  3. Added "CRITICAL REMINDERS" section reinforcing key requirements
  4. Structured with and tags for better attention

Expected Benefits:

  • User always gets at least one placement suggestion to consider
  • Multiple options shown when knowledge could fit in several places
  • More transparent reasoning helps user understand LLM decisions
  • Consistent with classification prompt's chain-of-thought approach
  • Better handling of edge cases (new topics, weak matches)

Assisted-by: Claude Code

Enhanced the Phase 3 integration prompt to ensure the LLM always
provides at least one suggestion, and preferably multiple (2-3)
suggestions when there are valid alternatives.

Changes to System Prompt:
1. Added explicit requirement to ALWAYS output at least one decision
2. Set preference for 2-3 decisions when multiple valid placements exist
3. Removed hard confidence threshold - if all candidates < 0.30, still
   output the best match with its actual confidence score
4. Reorganized prompt structure with reasoning-first format (matching
   classification prompt pattern)
5. Added detailed reasoning guidelines with 4-step decision framework
6. Changed system prompt role from 'JSON-only integration planner' to
   'knowledge organization expert specializing in Logseq integration'.
   This leads with expertise/capability rather than constraints, creating
   a stronger professional identity that may improve LLM performance.

Changes to User Prompt (Examples):
1. Replaced "no output" examples with positive examples showing:
   - Best match even with weak semantic fit (confidence 0.55)
   - Multiple suggestions for ambiguous content (3 suggestions)
2. Expanded examples from 5 to 7 with richer reasoning chains
3. Added "CRITICAL REMINDERS" section reinforcing key requirements
4. Structured with <examples> and <task> tags for better attention

Expected Benefits:
- User always gets at least one placement suggestion to consider
- Multiple options shown when knowledge could fit in several places
- More transparent reasoning helps user understand LLM decisions
- Consistent with classification prompt's chain-of-thought approach
- Better handling of edge cases (new topics, weak matches)

Assisted-by: Claude Code
@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.59%. Comparing base (d200961) to head (e6192ef).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #35   +/-   ##
=======================================
  Coverage   84.59%   84.59%           
=======================================
  Files          48       48           
  Lines        5084     5084           
=======================================
  Hits         4301     4301           
  Misses        783      783           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@twaugh twaugh merged commit b78f18c into main Nov 23, 2025
1 check passed
@twaugh twaugh deleted the feat/integration-prompt-always-suggest branch November 23, 2025 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants