Skip to content

Release/v1.8.0 #358

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
PavelInjective merged 20 commits into from
Nov 14, 2024
Merged

Release/v1.8.0 #358

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
01ee3f9
(fix) Change version in pyproject.toml to 1.7.0-pre
aarmoa Aug 7, 2024
206383b
Merge pull request #342 from InjectiveLabs/feat/sync_dec_with_master_…
aarmoa Aug 7, 2024
c1487d0
(feat) Implemented OFAC list address check
shibaeff Sep 6, 2024
a87553e
Merge pull request #346 from InjectiveLabs/ofac-python3-sdk
PavelInjective Sep 11, 2024
f205480
(fix) Change version in pyproject.toml to 1.7.0-pre
aarmoa Aug 7, 2024
aad28f0
Merge branch 'dev' of https://github.com/InjectiveLabs/sdk-python int…
aarmoa Sep 18, 2024
387b1fc
Merge pull request #348 from InjectiveLabs/fix/sync_dev_after_v170
aarmoa Sep 18, 2024
932388c
(fix) Change version in pyproject.toml to 1.7.0-pre
aarmoa Aug 7, 2024
c8ad358
(fix) Fixed the logic to get the full path to the ofac.json file
aarmoa Sep 23, 2024
a86166d
(fix) Updated CHANGELOG.md file and version in pyproject.toml
aarmoa Sep 23, 2024
c54d107
Merge branch 'fix/sync_dev_after_v1_7_1' of https://github.com/Inject…
aarmoa Sep 23, 2024
9f7b8e3
Merge pull request #352 from InjectiveLabs/fix/sync_dev_after_v1_7_1
aarmoa Sep 23, 2024
2b202ee
(feat) Refactored markets and tokens initialization logic in AsyncCli…
aarmoa Nov 12, 2024
be9cc3f
Merge branch 'master' of https://github.com/InjectiveLabs/sdk-python …
aarmoa Nov 13, 2024
74a6433
Merge pull request #357 from InjectiveLabs/feat/sync_dev_with_v1_7_2
aarmoa Nov 13, 2024
845e813
Merge branch 'dev' of https://github.com/InjectiveLabs/sdk-python int…
aarmoa Nov 13, 2024
1764dd9
Merge pull request #355 from InjectiveLabs/feat/refactor_markets_toke…
aarmoa Nov 13, 2024
50bdfbe
(fix) Updated version number in pyproject.toml
PavelInjective Nov 14, 2024
bff2fa9
(fix) Edit version in the CHANGELOG.md
PavelInjective Nov 14, 2024
f206f58
(fix) Edit date in the CHANGELOG.md
PavelInjective Nov 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

All notable changes to this project will be documented in this file.

## [1.8.0] - 2024-11-14
### Changed
- The markets initialization in AsyncClient has been modified to get markets information from the chain endpoints instead of the Indexer endpoints

Comment on lines +5 to +8
Copy link
Contributor

@coderabbitai coderabbitai bot Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Enhance changelog entry with migration guidance

The changelog indicates significant changes in the markets initialization process, but lacks guidance for users who need to migrate their code. Consider adding a migration section that explains:

  1. The differences between chain endpoints and Indexer endpoints
  2. Impact on existing applications
  3. Code examples showing the old vs new approach

Example addition:

 ### Changed
 - The markets initialization in AsyncClient has been modified to get markets information from the chain endpoints instead of the Indexer endpoints
+
+### Migration Guide
+#### Markets Initialization
+```python
+# Before (using Indexer endpoints)
+markets = await client.get_markets_from_indexer()
+
+# After (using chain endpoints)
+markets = await client.get_markets_from_chain()
+```
+
+#### Impact
+- Performance: Chain endpoints provide real-time market data
+- Consistency: Direct access to on-chain market state
+- Breaking Change: Applications using Indexer endpoints need to be updated

### Removed
- Removed the legacy deprecated markets and tokens initialization using the denoms INI files in the SDK. Removed also the INI files from the SDK

## [1.7.2] - 2024-11-13
### Fixed
- Fixed link to official `ofac.json` file
Expand Down
146 changes: 72 additions & 74 deletions pyinjective/async_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,8 +157,8 @@ def __init__(
self._initialize_timeout_height_sync_task()

self._tokens_and_markets_initialization_lock = asyncio.Lock()
self._tokens_by_denom: Optional[Dict[str, Token]] = None
self._tokens_by_symbol: Optional[Dict[str, Token]] = None
self._tokens_by_denom = dict()
self._tokens_by_symbol = dict()
Comment on lines +160 to +161
Copy link
Contributor

@coderabbitai coderabbitai bot Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Variable Initialization May Prevent Tokens and Markets from Initializing Properly

By initializing _tokens_by_denom and _tokens_by_symbol to empty dictionaries at lines 160-161, the conditional checks for None in methods like all_tokens, all_spot_markets, all_derivative_markets, and all_binary_option_markets will never trigger. This means _initialize_tokens_and_markets may not be called, leading to uninitialized tokens and markets.

To fix this issue, update the conditions to check if the dictionaries are empty instead of None.

Apply this diff:

@@ -195,7 +195,7 @@ class AsyncClient:
     async def all_tokens(self) -> Dict[str, Token]:
-        if self._tokens_by_symbol is None:
+        if not self._tokens_by_symbol:
             async with self._tokens_and_markets_initialization_lock:
-                if self._tokens_by_symbol is None:
+                if not self._tokens_by_symbol:
                     await self._initialize_tokens_and_markets()
         return deepcopy(self._tokens_by_symbol)

Repeat similar changes for the other methods:

@@ -202,7 +202,7 @@ class AsyncClient:
     async def all_spot_markets(self) -> Dict[str, SpotMarket]:
-        if self._spot_markets is None:
+        if not self._spot_markets:
             async with self._tokens_and_markets_initialization_lock:
-                if self._spot_markets is None:
+                if not self._spot_markets:
                     await self._initialize_tokens_and_markets()
         return deepcopy(self._spot_markets)

Committable suggestion skipped: line range outside the PR's diff.

self._spot_markets: Optional[Dict[str, SpotMarket]] = None
self._derivative_markets: Optional[Dict[str, DerivativeMarket]] = None
self._binary_option_markets: Optional[Dict[str, BinaryOptionMarket]] = None
Expand Down Expand Up @@ -3291,87 +3291,81 @@ async def _initialize_tokens_and_markets(self):
derivative_markets = dict()
binary_option_markets = dict()
tokens_by_symbol, tokens_by_denom = await self._tokens_from_official_lists(network=self.network)
markets_info = (await self.fetch_spot_markets(market_statuses=["active"]))["markets"]
valid_markets = (
market_info
for market_info in markets_info
if len(market_info.get("baseTokenMeta", {}).get("symbol", "")) > 0
and len(market_info.get("quoteTokenMeta", {}).get("symbol", "")) > 0
)

for market_info in valid_markets:
base_token = self._token_representation(
token_meta=market_info["baseTokenMeta"],
denom=market_info["baseDenom"],
tokens_by_denom=tokens_by_denom,
tokens_by_symbol=tokens_by_symbol,
)
quote_token = self._token_representation(
token_meta=market_info["quoteTokenMeta"],
denom=market_info["quoteDenom"],
tokens_by_denom=tokens_by_denom,
tokens_by_symbol=tokens_by_symbol,
)
self._tokens_by_denom.update(tokens_by_denom)
self._tokens_by_symbol.update(tokens_by_symbol)

markets_info = (await self.fetch_chain_spot_markets(status="Active"))["markets"]
for market_info in markets_info:
base_token = self._tokens_by_denom.get(market_info["baseDenom"])
quote_token = self._tokens_by_denom.get(market_info["quoteDenom"])
Comment on lines +3299 to +3300
Copy link
Contributor

@coderabbitai coderabbitai bot Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Handle Potential None Values for Tokens in Market Initialization

At lines 3299-3300, base_token and quote_token may be None if the tokens are not found in self._tokens_by_denom. Passing None to the SpotMarket constructor could lead to errors or unexpected behavior.

Add checks to ensure tokens are found before proceeding:

base_token = self._tokens_by_denom.get(market_info["baseDenom"])
quote_token = self._tokens_by_denom.get(market_info["quoteDenom"])

if base_token is None or quote_token is None:
    # Handle missing tokens, e.g., log a warning or skip this market
    continue  # or raise an exception

market = SpotMarket(
    id=market_info["marketId"],
    status=market_info["status"],
    ticker=market_info["ticker"],
    base_token=base_token,
    quote_token=quote_token,
    # ... rest of the initialization
)


market = SpotMarket(
id=market_info["marketId"],
status=market_info["marketStatus"],
status=market_info["status"],
ticker=market_info["ticker"],
base_token=base_token,
quote_token=quote_token,
maker_fee_rate=Decimal(market_info["makerFeeRate"]),
taker_fee_rate=Decimal(market_info["takerFeeRate"]),
service_provider_fee=Decimal(market_info["serviceProviderFee"]),
min_price_tick_size=Decimal(market_info["minPriceTickSize"]),
min_quantity_tick_size=Decimal(market_info["minQuantityTickSize"]),
min_notional=Decimal(market_info["minNotional"]),
maker_fee_rate=Token.convert_value_from_extended_decimal_format(Decimal(market_info["makerFeeRate"])),
taker_fee_rate=Token.convert_value_from_extended_decimal_format(Decimal(market_info["takerFeeRate"])),
service_provider_fee=Token.convert_value_from_extended_decimal_format(
Decimal(market_info["relayerFeeShareRate"])
),
min_price_tick_size=Token.convert_value_from_extended_decimal_format(
Decimal(market_info["minPriceTickSize"])
),
min_quantity_tick_size=Token.convert_value_from_extended_decimal_format(
Decimal(market_info["minQuantityTickSize"])
),
min_notional=Token.convert_value_from_extended_decimal_format(Decimal(market_info["minNotional"])),
)

spot_markets[market.id] = market

markets_info = (await self.fetch_derivative_markets(market_statuses=["active"]))["markets"]
valid_markets = (
market_info
for market_info in markets_info
if len(market_info.get("quoteTokenMeta", {}).get("symbol", "")) > 0
)

for market_info in valid_markets:
quote_token = self._token_representation(
token_meta=market_info["quoteTokenMeta"],
denom=market_info["quoteDenom"],
tokens_by_denom=tokens_by_denom,
tokens_by_symbol=tokens_by_symbol,
)

market = DerivativeMarket(
id=market_info["marketId"],
status=market_info["marketStatus"],
ticker=market_info["ticker"],
oracle_base=market_info["oracleBase"],
oracle_quote=market_info["oracleQuote"],
oracle_type=market_info["oracleType"],
oracle_scale_factor=market_info["oracleScaleFactor"],
initial_margin_ratio=Decimal(market_info["initialMarginRatio"]),
maintenance_margin_ratio=Decimal(market_info["maintenanceMarginRatio"]),
markets_info = (await self.fetch_chain_derivative_markets(status="Active", with_mid_price_and_tob=False))[
"markets"
]
for market_info in markets_info:
market = market_info["market"]
quote_token = self._tokens_by_denom.get(market["quoteDenom"])

Comment on lines +3329 to +3330
Copy link
Contributor

@coderabbitai coderabbitai bot Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Ensure quote_token is Valid in Derivative Market Initialization

In lines 3329-3330, quote_token may be None if not present in self._tokens_by_denom. This could cause issues when initializing DerivativeMarket.

Include a check for quote_token:

quote_token = self._tokens_by_denom.get(market["quoteDenom"])

if quote_token is None:
    # Handle missing token
    continue  # or raise an exception

derivative_market = DerivativeMarket(
    id=market["marketId"],
    status=market["status"],
    ticker=market["ticker"],
    # ... rest of the initialization
    quote_token=quote_token,
    # ... rest of the initialization
)

derivative_market = DerivativeMarket(
id=market["marketId"],
status=market["status"],
ticker=market["ticker"],
oracle_base=market["oracleBase"],
oracle_quote=market["oracleQuote"],
oracle_type=market["oracleType"],
oracle_scale_factor=market["oracleScaleFactor"],
initial_margin_ratio=Token.convert_value_from_extended_decimal_format(
Decimal(market["initialMarginRatio"])
),
maintenance_margin_ratio=Token.convert_value_from_extended_decimal_format(
Decimal(market["maintenanceMarginRatio"])
),
quote_token=quote_token,
maker_fee_rate=Decimal(market_info["makerFeeRate"]),
taker_fee_rate=Decimal(market_info["takerFeeRate"]),
service_provider_fee=Decimal(market_info["serviceProviderFee"]),
min_price_tick_size=Decimal(market_info["minPriceTickSize"]),
min_quantity_tick_size=Decimal(market_info["minQuantityTickSize"]),
min_notional=Decimal(market_info["minNotional"]),
maker_fee_rate=Token.convert_value_from_extended_decimal_format(Decimal(market["makerFeeRate"])),
taker_fee_rate=Token.convert_value_from_extended_decimal_format(Decimal(market["takerFeeRate"])),
service_provider_fee=Token.convert_value_from_extended_decimal_format(
Decimal(market["relayerFeeShareRate"])
),
min_price_tick_size=Token.convert_value_from_extended_decimal_format(
Decimal(market["minPriceTickSize"])
),
min_quantity_tick_size=Token.convert_value_from_extended_decimal_format(
Decimal(market["minQuantityTickSize"])
),
min_notional=Token.convert_value_from_extended_decimal_format(Decimal(market["minNotional"])),
)

derivative_markets[market.id] = market
derivative_markets[derivative_market.id] = derivative_market

markets_info = (await self.fetch_binary_options_markets(market_status="active"))["markets"]
markets_info = (await self.fetch_chain_binary_options_markets(status="Active"))["markets"]
for market_info in markets_info:
quote_token = tokens_by_denom.get(market_info["quoteDenom"], None)
quote_token = self._tokens_by_denom.get(market_info["quoteDenom"])
Copy link
Contributor

@coderabbitai coderabbitai bot Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Validate quote_token in Binary Option Market Initialization

At line 3364, quote_token might be None if the token is not found. This needs handling to prevent errors during market initialization.

Add a check before initializing the market:

quote_token = self._tokens_by_denom.get(market_info["quoteDenom"])

if quote_token is None:
    # Handle missing token
    continue  # or raise an exception

market = BinaryOptionMarket(
    id=market_info["marketId"],
    status=market_info["status"],
    ticker=market_info["ticker"],
    # ... rest of the initialization
    quote_token=quote_token,
    # ... rest of the initialization
)


market = BinaryOptionMarket(
id=market_info["marketId"],
status=market_info["marketStatus"],
status=market_info["status"],
ticker=market_info["ticker"],
oracle_symbol=market_info["oracleSymbol"],
oracle_provider=market_info["oracleProvider"],
Expand All @@ -3380,21 +3374,25 @@ async def _initialize_tokens_and_markets(self):
expiration_timestamp=market_info["expirationTimestamp"],
settlement_timestamp=market_info["settlementTimestamp"],
quote_token=quote_token,
maker_fee_rate=Decimal(market_info["makerFeeRate"]),
taker_fee_rate=Decimal(market_info["takerFeeRate"]),
service_provider_fee=Decimal(market_info["serviceProviderFee"]),
min_price_tick_size=Decimal(market_info["minPriceTickSize"]),
min_quantity_tick_size=Decimal(market_info["minQuantityTickSize"]),
min_notional=Decimal(market_info["minNotional"]),
maker_fee_rate=Token.convert_value_from_extended_decimal_format(Decimal(market_info["makerFeeRate"])),
taker_fee_rate=Token.convert_value_from_extended_decimal_format(Decimal(market_info["takerFeeRate"])),
service_provider_fee=Token.convert_value_from_extended_decimal_format(
Decimal(market_info["relayerFeeShareRate"])
),
min_price_tick_size=Token.convert_value_from_extended_decimal_format(
Decimal(market_info["minPriceTickSize"])
),
min_quantity_tick_size=Token.convert_value_from_extended_decimal_format(
Decimal(market_info["minQuantityTickSize"])
),
min_notional=Token.convert_value_from_extended_decimal_format(Decimal(market_info["minNotional"])),
settlement_price=None
if market_info["settlementPrice"] == ""
else Decimal(market_info["settlementPrice"]),
else Token.convert_value_from_extended_decimal_format(Decimal(market_info["settlementPrice"])),
)

binary_option_markets[market.id] = market

self._tokens_by_denom = tokens_by_denom
self._tokens_by_symbol = tokens_by_symbol
self._spot_markets = spot_markets
self._derivative_markets = derivative_markets
self._binary_option_markets = binary_option_markets
Expand Down
103 changes: 5 additions & 98 deletions pyinjective/composer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
import json
from configparser import ConfigParser
from decimal import Decimal
from time import time
from typing import Any, Dict, List, Optional
from warnings import warn

from google.protobuf import any_pb2, json_format, timestamp_pb2

from pyinjective import constant
from pyinjective.constant import ADDITIONAL_CHAIN_FORMAT_DECIMALS, INJ_DECIMALS, INJ_DENOM
from pyinjective.core.market import BinaryOptionMarket, DerivativeMarket, SpotMarket
from pyinjective.core.token import Token
Expand Down Expand Up @@ -137,17 +135,11 @@ def __init__(

"""
self.network = network
self.spot_markets = dict()
self.derivative_markets = dict()
self.binary_option_markets = dict()
self.tokens = dict()
if spot_markets is None or derivative_markets is None or binary_option_markets is None or tokens is None:
self._initialize_markets_and_tokens_from_files()
else:
self.spot_markets = spot_markets
self.derivative_markets = derivative_markets
self.binary_option_markets = binary_option_markets
self.tokens = tokens
self.spot_markets = spot_markets or dict()
self.derivative_markets = derivative_markets or dict()
self.binary_option_markets = binary_option_markets or dict()
self.tokens = tokens or dict()

self._ofac_checker = OfacChecker()

def Coin(self, amount: int, denom: str):
Expand Down Expand Up @@ -2556,91 +2548,6 @@ def unpack_transaction_messages(transaction_data: Dict[str, Any]) -> List[Dict[s

return msgs

def _initialize_markets_and_tokens_from_files(self):
config: ConfigParser = constant.CONFIGS[self.network]
spot_markets = dict()
derivative_markets = dict()
tokens = dict()

for section_name, configuration_section in config.items():
if section_name.startswith("0x"):
description = configuration_section["description"]

quote_token = Token(
name="",
symbol="",
denom="",
address="",
decimals=int(configuration_section["quote"]),
logo="",
updated=-1,
)

if "Spot" in description:
base_token = Token(
name="",
symbol="",
denom="",
address="",
decimals=int(configuration_section["base"]),
logo="",
updated=-1,
)

market = SpotMarket(
id=section_name,
status="",
ticker=description,
base_token=base_token,
quote_token=quote_token,
maker_fee_rate=None,
taker_fee_rate=None,
service_provider_fee=None,
min_price_tick_size=Decimal(str(configuration_section["min_price_tick_size"])),
min_quantity_tick_size=Decimal(str(configuration_section["min_quantity_tick_size"])),
min_notional=Decimal(str(configuration_section.get("min_notional", "0"))),
)
spot_markets[market.id] = market
else:
market = DerivativeMarket(
id=section_name,
status="",
ticker=description,
oracle_base="",
oracle_quote="",
oracle_type="",
oracle_scale_factor=1,
initial_margin_ratio=None,
maintenance_margin_ratio=None,
quote_token=quote_token,
maker_fee_rate=None,
taker_fee_rate=None,
service_provider_fee=None,
min_price_tick_size=Decimal(str(configuration_section["min_price_tick_size"])),
min_quantity_tick_size=Decimal(str(configuration_section["min_quantity_tick_size"])),
min_notional=Decimal(str(configuration_section.get("min_notional", "0"))),
)

derivative_markets[market.id] = market

elif section_name != "DEFAULT":
token = Token(
name=section_name,
symbol=section_name,
denom=configuration_section["peggy_denom"],
address="",
decimals=int(configuration_section["decimals"]),
logo="",
updated=-1,
)

tokens[token.symbol] = token

self.tokens = tokens
self.spot_markets = spot_markets
self.derivative_markets = derivative_markets
self.binary_option_markets = dict()

def _order_mask(self, is_conditional: bool, is_buy: bool, is_market_order: bool) -> int:
order_mask = 0

Expand Down
18 changes: 0 additions & 18 deletions pyinjective/constant.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,25 +1,7 @@
import os
from configparser import ConfigParser

GAS_PRICE = 160_000_000
GAS_FEE_BUFFER_AMOUNT = 25_000
MAX_MEMO_CHARACTERS = 256
ADDITIONAL_CHAIN_FORMAT_DECIMALS = 18
TICKER_TOKENS_SEPARATOR = "/"
INJ_DENOM = "inj"
INJ_DECIMALS = 18

devnet_config = ConfigParser()
devnet_config.read(os.path.join(os.path.dirname(__file__), "denoms_devnet.ini"))

testnet_config = ConfigParser()
testnet_config.read(os.path.join(os.path.dirname(__file__), "denoms_testnet.ini"))

mainnet_config = ConfigParser()
mainnet_config.read(os.path.join(os.path.dirname(__file__), "denoms_mainnet.ini"))

CONFIGS = {
"devnet": devnet_config,
"testnet": testnet_config,
"mainnet": mainnet_config,
}
10 changes: 10 additions & 0 deletions pyinjective/core/token.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
from dataclasses import dataclass
from decimal import Decimal

from pyinjective.constant import ADDITIONAL_CHAIN_FORMAT_DECIMALS


@dataclass(eq=True, frozen=True)
class Token:
Expand All @@ -12,5 +14,13 @@ class Token:
logo: str
updated: int

@staticmethod
def convert_value_to_extended_decimal_format(value: Decimal) -> Decimal:
return value * Decimal(f"1e{ADDITIONAL_CHAIN_FORMAT_DECIMALS}")

@staticmethod
def convert_value_from_extended_decimal_format(value: Decimal) -> Decimal:
return value / Decimal(f"1e{ADDITIONAL_CHAIN_FORMAT_DECIMALS}")

def chain_formatted_value(self, human_readable_value: Decimal) -> Decimal:
return human_readable_value * Decimal(f"1e{self.decimals}")
Loading
Loading