Skip to content

Conversation

@Hebx
Copy link
Contributor

@Hebx Hebx commented Nov 12, 2025

Problem

Wallet token prices were showing as $0.00 when:

  • Birdeye API was rate-limited (429 Too Many Requests)
  • Jupiter API had connection issues (ECONNREFUSED)
  • API responses had unexpected formats or missing data
  • Network timeouts occurred during price fetching

This resulted in inaccurate wallet balance calculations and poor user experience.

Solution

Implemented a comprehensive multi-tier fallback system for fetching token prices with robust error handling and resource management.

Price Fetching Priority Chain

  1. Primary: Jupiter Price API V3 (real-time Solana prices, batch requests)
  2. Fallback 1: CoinGecko API (free, reliable, no API key needed)
  3. Fallback 2: Birdeye API (existing integration)
  4. Final Fallback: Hardcoded stablecoin prices (USDC/USDT = $1.00)

Key Features

🚀 Jupiter Price API V3 Integration

  • Batch price fetching for multiple tokens in a single request
  • Supports both lite API (no key) and pro API (with key)
  • Efficient ids parameter with comma-separated mint addresses
  • Real-time price data with usdPrice field validation
  • Automatic endpoint selection based on API key presence

🔄 CoinGecko Fallback

  • Free API integration (no API key required)
  • Special handling for SOL (ID-based endpoint)
  • Support for arbitrary Solana tokens via contract address
  • Reliable fallback when Jupiter doesn't have token data

🛡️ Robust Error Handling

  • Timeout protection: 5-second timeouts with proper cleanup
  • Resource leak prevention: try-finally blocks ensure clearTimeout() always executes
  • Graceful handling of rate limits (429), connection errors (ECONNREFUSED)
  • Comprehensive logging for debugging price fetch failures
  • Validation of price data (finite, positive numbers only)

✅ Data Validation

  • isFinite() checks prevent Infinity/NaN values
  • Positive number validation for all prices
  • Type checking before price calculations
  • Prevents downstream calculation errors

🧪 Comprehensive Testing

  • Unit tests for Jupiter Price API V3 integration
  • Tests for fallback chain order and behavior
  • Tests for CoinGecko API integration
  • Error handling and edge case coverage
  • Response format validation tests

🔧 CI/CD Improvements

  • Secret validation before server startup (fails fast with helpful messages)
  • Improved error messages for missing environment variables
  • CI-aware error handling in Supabase initialization
  • Better debugging output for server startup failures
  • Environment variable precedence (CI secrets override .env files)

Changes Made

Core Functionality

  • ✅ Integrated Jupiter Price API V3 as primary price source
  • ✅ Added CoinGecko API integration for SOL and arbitrary tokens
  • ✅ Implemented prioritized fallback chain (Jupiter → CoinGecko → Birdeye)
  • ✅ Added batch price fetching for efficiency
  • ✅ Enhanced price validation with isFinite() checks

Code Quality

  • ✅ Fixed timeout handler resource leaks (try-finally blocks)
  • ✅ Improved error handling and logging
  • ✅ Added comprehensive test coverage
  • ✅ Fixed inverted logic in endpoint selection tests

Infrastructure

  • ✅ Added secret validation step in CI workflows
  • ✅ Improved CI error messages and debugging
  • ✅ Fixed environment variable handling (CI precedence)
  • ✅ Added GRID_ENV configuration for production

Technical Details

API Endpoints Used

Jupiter Price API V3:

  • Lite: https://lite-api.jup.ag/price/v3?ids=<comma-separated-addresses>
  • Pro: https://api.jup.ag/price/v3?ids=<comma-separated-addresses> (with API key)

CoinGecko:

  • SOL: https://api.coingecko.com/api/v3/simple/price?ids=solana&vs_currencies=usd
  • Tokens: https://api.coingecko.com/api/v3/simple/token_price/solana?contract_addresses=<address>&vs_currencies=usd

Performance Optimizations

  • Batch requests reduce API calls (1 request for N tokens vs N requests)
  • Parallel CoinGecko requests for missing tokens
  • Efficient token filtering (only fetch missing tokens from fallbacks)

Resource Management

  • All fetch calls wrapped in try-finally blocks
  • Timeout handlers always cleaned up (prevents memory leaks)
  • Proper AbortController usage for request cancellation

Testing

  • ✅ 66+ unit tests passing
  • ✅ Comprehensive integration test coverage
  • ✅ Error handling and edge case tests
  • ✅ Response format validation tests
  • ✅ Fallback chain behavior verification

Before & After

Before ❌

  • Prices showed $0.00 when APIs failed
  • No fallback mechanism
  • Resource leaks from uncleaned timeouts
  • Poor error messages in CI
  • Missing price validation (Infinity/NaN could pass through)

After ✅

  • Prices always fetched from fallback sources
  • Real-time SOL price from Jupiter or CoinGecko
  • Accurate wallet balances even when primary API fails
  • Proper resource cleanup (no memory leaks)
  • Clear error messages in CI with actionable guidance
  • Robust price validation prevents calculation errors

Files Changed

  • apps/server/src/routes/wallet/holdings.ts - Core price fetching logic
  • apps/server/src/routes/wallet/__tests__/holdings-price-fallback.test.ts - Test suite
  • apps/server/src/routes/wallet/__tests__/jupiter-price-api-v3.test.ts - Jupiter API tests
  • .github/workflows/test.yml - CI improvements
  • apps/server/src/lib/supabase.ts - CI-aware error handling
  • apps/server/src/server.ts - Environment variable handling

Related Issues

Fixes #93 - Real Token Holdings Not Displayed in Wallet & Grid Wallet Balance Not Triggered in Chat

Checklist

  • All tests passing
  • Code follows project style guidelines
  • Error handling implemented
  • Resource leaks fixed
  • CI improvements added
  • Comprehensive test coverage
  • Documentation updated (via changesets)

@vercel
Copy link

vercel bot commented Nov 12, 2025

@Hebx is attempting to deploy a commit to the dark Team on Vercel.

A member of the Team first needs to authorize it.

@vercel
Copy link

vercel bot commented Nov 12, 2025

Deployment failed with the following error:

The provided GitHub repository does not contain the requested branch or commit reference. Please ensure the repository is not empty.

@edgarpavlovsky
Copy link
Member

this is a good enhancement, because birdeye data is definitely expensive. the mallory wallet does support arbitrary spl tokens though, so we shouldn't just rely on fetching SOL and USDC/T prices.

i'm not 100% sure off hand how robust coingecko's API is. can we refactor to do something like this:

  1. prioritize jupiter's API first for arbitrary token holdings
  2. fallback to coingecko's API if jupiter not available
  3. fallback to birdeye's API if jupiter + coingecko not available?

and operate under the assumption that the user will have arbitrary SPL tokens in their wallet?

@Hebx
Copy link
Contributor Author

Hebx commented Nov 14, 2025

Related Feature PR

This fix PR is now followed by a feature PR that builds on top of it:

Feature PR: feat/jupiter-price-api-integrationfix/wallet-price-fallback-sources

The feature PR adds:

  • Jupiter Price API V3 integration for prioritized price fetching
  • Enhanced fallback chain: Jupiter → CoinGecko → Birdeye
  • Batch price fetching for better performance

Once this fix PR is merged, the feature PR can be reviewed and merged.

Hebx added 2 commits November 21, 2025 02:41
- Add CoinGecko API as fallback for SOL price (free, no API key needed)
- Improve Jupiter API error handling with timeouts
- Add multi-tier fallback system: Birdeye -> Jupiter -> CoinGecko
- Handle rate limits (429) gracefully with fallback sources
- Add better logging for price fetching
- Support multiple Birdeye response formats
- Use fallback prices only for stablecoins (USDC/USDT)

Fixes issue where wallet prices showed --.00 when Birdeye API was rate-limited or Jupiter had connection issues.
- Add tests for Jupiter API integration (endpoint selection, timeout handling)
- Add tests for CoinGecko fallback mechanism
- Add tests for fallback chain order and stablecoin prices
- Add tests for error handling (ECONNREFUSED, rate limits)
- Verify NaN validation for Jupiter API responses
@Hebx Hebx force-pushed the fix/wallet-price-fallback-sources branch from 256a355 to 1a0d8a9 Compare November 21, 2025 01:48
Hebx added 2 commits November 21, 2025 02:48
…allback chain

- Integrate Jupiter Price API V3 for efficient batch price fetching
- Implement prioritized fallback chain: Jupiter → CoinGecko → Birdeye
- Add support for arbitrary token price fetching (not just SOL)
- Fix response parsing to handle Jupiter API V3 format (usdPrice field)
- Add comprehensive logging for debugging price fetch failures
- Improve error handling and fallback logic

This ensures we get the most accurate real-time prices from Jupiter first,
with reliable fallbacks for tokens Jupiter doesn't have prices for.
- Add tests for batch price fetching with Jupiter Price API V3
- Add tests for prioritized fallback chain (Jupiter -> CoinGecko -> Birdeye)
- Add tests for arbitrary token price fetching from CoinGecko
- Add tests for response format validation (usdPrice field)
- Add tests for error handling and performance optimizations
- Verify batch request efficiency vs individual requests
Hebx added 10 commits November 21, 2025 02:52
- Fix hasApiKey check from !process.env.JUPITER_API_KEY to !!process.env.JUPITER_API_KEY
- Update test to conditionally check for correct endpoint based on API key presence
- Test now correctly verifies lite-api.jup.ag when no key, api.jup.ag when key is set

Fixes test failure where inverted logic caused wrong endpoint to be tested
- Fix hasApiKey check from !process.env.JUPITER_API_KEY to !!process.env.JUPITER_API_KEY
- Update test to conditionally check for correct endpoint based on API key presence
- Test now correctly verifies lite-api.jup.ag when no key, api.jup.ag when key is set

Fixes test failure where inverted logic caused wrong endpoint to be tested
- Add GRID_ENV: production to both integration and E2E test jobs
- Add debug step to verify environment variables are set before server starts
- Improve error messages with better logging when server fails to start
- This should fix backend server startup failures in GitHub Actions

The debug step will help identify if secrets are missing or empty,
which was causing the server to crash immediately on startup.
- Fix: Call fetchTokenPricesWithFallbacks instead of fetchMarketDataWithFallbacks
  This ensures Jupiter Price API V3 is prioritized as intended (Jupiter -> CoinGecko -> Birdeye)
- Fix: Add isFinite check to price validation to prevent Infinity values
  This matches the test expectations and prevents calculation errors downstream

Fixes the bug where the new Jupiter-first implementation was never called,
and ensures price validation properly rejects Infinity values.
- Add CI-aware error messages in supabase.ts that detect GitHub Actions context
- Configure dotenv to not override existing env vars (respects GitHub Actions secrets)
- Add early validation checks in workflow before starting server
- Improve error messages to help debug missing environment variables
The test was validating Jupiter's old swap/v1 endpoint, but the
implementation now uses the price/v3 endpoint. Updated all endpoint
references and validation logic to match the actual implementation:

- Changed endpoint URLs from swap/v1 to price/v3
- Updated test descriptions to reference Price API V3
- Replaced outAmount validation with usdPrice validation
- Added test for batch request URL construction with ids parameter
- Updated fallback chain description to mention Price API V3

This ensures the test validates the actual production code and will
catch regressions in the Jupiter Price API V3 integration.
Add isFinite() validation to prevent Infinity and -Infinity values
from passing price validation checks. This fixes calculation errors
downstream when computing token values.

Fixed validation points:
- CoinGecko SOL price validation
- CoinGecko token price validation
- Jupiter prices in result map
- CoinGecko prices in result map
- Birdeye prices in fallback chain
- Birdeye prices in market data fetch
- SOL price from fallbacks

All price validations now follow the pattern:
typeof price === 'number' && isFinite(price) && price > 0

This matches the test expectations in holdings-price-fallback.test.ts
which explicitly tests for isFinite validation (line 71, 80).
- Add 'Validate required secrets' step in integration-tests and e2e-tests jobs
- Check if GitHub secrets exist before attempting to start server
- Provide clear error messages with instructions when secrets are missing
- Fail fast with helpful guidance instead of generic environment variable errors

This prevents the server from attempting to start with missing credentials
and provides actionable error messages pointing to where secrets should be
configured in GitHub Actions.
…ck-sources

- Integrated Jupiter Price API V3 as primary price source
- Added comprehensive test coverage for price fallback system
- Updated CI workflow with secret validation
- Improved error handling and validation for price data
- Resolved merge conflicts in holdings-price-fallback.test.ts
@Hebx Hebx force-pushed the fix/wallet-price-fallback-sources branch from fa719d1 to 28772e2 Compare November 21, 2025 03:44
- Wrap fetch calls in try-finally blocks to ensure clearTimeout always executes
- Fixes resource leaks when fetch() throws errors before timeout cleanup
- Matches pattern used in birdeye client service
@Hebx Hebx force-pushed the fix/wallet-price-fallback-sources branch from 414f84e to b81c86d Compare November 21, 2025 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Issue: Real Token Holdings Not Displayed in Wallet & Grid Wallet Balance Not Triggered in Chat

2 participants