The Player of Games is a Neural Network self improving trading strategies, using Upper Confidence Bound re-inforcement learning.

The best models are then auto-deployed so automated traders pick them up in realtime and start trading them live across different exchanges.

The overal structure is based on independant services that can run and be maintained:

UCB Neural Network Trainer
Trader for HyperLiquid Crypto Exchange
Trader for Lighter Crypto Exchange
Risk Manager
Portfolio Manager
Labs Market Research Portal
Backtester
QA Tester

UCB Trading Strategy RL Microservice

A standalone microservice for training trading strategies using PPO (Proximal Policy Optimization) reinforcement learning with UCB (Upper Confidence Bound) exploration.

Port: 9001

Features

PPO Reinforcement Learning: Episode-based policy training with GPU acceleration
UCB Optimizer: Multi-armed bandit approach for parameter space exploration
Multi-Symbol Support: Train the same strategy on different trading pairs (ETHUSDT, BTCUSDT, etc.) independently
Concurrent Training: Run multiple trainings simultaneously with tabbed UI to switch between them
Auto-Continue Training: Automatically resumes from best model on server restart
Pause/Resume: Pause training and resume later without losing progress
Real-time WebSocket Updates: Live training metrics in browser with per-training charts and trade logs
Statistics Dashboard: Comprehensive analytics with three visualization matrices:
- Product-Strategy Matrix: Shows best reward per product-strategy combination with color-coded performance
- Performance Matrix: Displays trade statistics (avg win/loss PnL, win rate, consecutive streaks) for each best model
- Product Details: Detailed breakdown with training config, model metadata, and download links
Model Storage API: Export best models for other microservices
GPU/CUDA Support: Optimized for NVIDIA GPUs

Quick Start

cd D:\PoG\UCB
python -m backend.main

The service will start on http://localhost:9001

Architecture

UCB/
├── backend/
│   ├── main.py              # FastAPI application (port 9001)
│   ├── strategy_loader.py   # Dynamic strategy loading
│   ├── exchange/
│   │   └── market_data.py   # Bybit data fetching with caching
│   ├── rl/
│   │   ├── ucb_optimizer.py # UCB + TemporalEncoder
│   │   ├── rl_trainer.py    # PPO training with SB3
│   │   └── trading_env.py   # Gymnasium trading environment
│   └── strategies/
│       └── base_strategy.py # Base class for strategies
├── frontend/
│   ├── index.html           # Main UI
│   └── static/
│       ├── styles.css       # Starlink-inspired dark theme
│       └── app.js           # Frontend application
├── strategies_library/      # Drop strategies here
├── output/                  # Trained models are saved here (multi-symbol)
│   └── {strategy_id}/
│       └── {symbol}/        # e.g., ETHUSDT, BTCUSDT
│           ├── best_model.zip
│           └── best_model_metadata.json
├── cache/
│   ├── candles/             # Cached market data
│   └── models/              # Temporary model saves
└── run.py                   # Entry point

API Endpoints

Training Control

Endpoint	Method	Description
`/api/training/start`	POST	Start training for a strategy (requires `strategy_id`, `symbol`)
`/api/training/stop`	POST	Stop active training (requires `strategy_id`, `symbol`)
`/api/training/pause`	POST	Pause training (requires `strategy_id`, `symbol`)
`/api/training/resume`	POST	Resume paused training (requires `strategy_id`, `symbol`)
`/api/training/sync`	GET	Get current training state (uses composite keys)
`/api/training/paused`	GET	List paused trainings
`/api/training/status/{strategy_id}`	GET	Get status for all symbols of a strategy
`/api/training/force-clear`	POST	Clear stuck training states

Models (Multi-Symbol)

Endpoint	Method	Description
`/api/models`	GET	List all saved models (all strategies, all symbols)
`/api/models/{strategy_id}`	GET	Get all models for a strategy (all symbols)
`/api/models/{strategy_id}/{symbol}`	GET	Get specific model info
`/api/models/{strategy_id}/{symbol}/download`	GET	Download specific model file (.zip)
`/api/models/{strategy_id}/download`	GET	Download legacy model (backward compatibility)
`/api/models/legacy`	GET	List legacy models that need migration
`/api/models/migrate`	POST	Migrate legacy models to multi-symbol structure

Strategies

Endpoint	Method	Description
`/api/strategies`	GET	List available strategies
`/api/strategies/{id}`	GET	Get strategy details
`/api/strategies/{id}/enable`	POST	Enable strategy
`/api/strategies/{id}/disable`	POST	Disable strategy

Statistics

Endpoint	Method	Description
`/api/stats`	GET	Get comprehensive statistics for all strategies and models
`/api/stats/{strategy_id}`	GET	Get detailed statistics for a specific strategy

System

Endpoint	Method	Description
`/api/status`	GET	System status and GPU info
`/api/cache/clear`	POST	Clear market data cache

WebSocket

Endpoint	Description
`WS /ws`	Real-time training updates

Connecting from Another Application

This section documents how to connect to the UCB microservice from other projects/applications to fetch and use trained models.

1. Check if Service is Running

import requests

UCB_SERVICE = "http://localhost:9001"

def is_ucb_available() -> bool:
    """Check if UCB service is running."""
    try:
        response = requests.get(f"{UCB_SERVICE}/api/status", timeout=2)
        return response.status_code == 200
    except:
        return False

2. Get Statistics Overview

The /api/stats endpoint provides comprehensive statistics about all strategies, models, and training status. This is the recommended endpoint for microservices that need an overview of the UCB service state.

import requests

UCB_SERVICE = "http://localhost:9001"

# Get all statistics
response = requests.get(f"{UCB_SERVICE}/api/stats")
stats = response.json()

# Response structure:
# {
#     "timestamp": "2024-01-15T10:30:00",
#     "service": {
#         "name": "UCB Trading Strategy RL",
#         "port": 9001,
#         "version": "2.1.0"
#     },
#     "strategies": [...],           # List of all strategies with their status
#     "training": {
#         "active": {...},           # Currently running trainings
#         "paused": {...}            # Paused trainings that can be resumed
#     },
#     "summary": {
#         "total_strategies": 5,
#         "total_models": 3,
#         "active_trainings": 1,
#         "paused_trainings": 0,
#         "best_overall_reward": 1234.56,
#         "best_model_strategy": "MomentumStrategy"
#     }
# }

# Access summary
print(f"Total strategies: {stats['summary']['total_strategies']}")
print(f"Active trainings: {stats['summary']['active_trainings']}")
print(f"Best model: {stats['summary']['best_model_strategy']}")
print(f"Best reward: {stats['summary']['best_overall_reward']}")

# Find the best model details
for strategy in stats['strategies']:
    if strategy.get('has_model') and strategy.get('is_best_overall'):
        print(f"\nBest Model Details:")
        print(f"  Strategy: {strategy['id']}")
        print(f"  Path: {strategy['model_path']}")
        print(f"  Reward: {strategy['model_info']['mean_reward']}")
        print(f"  Iteration: {strategy['model_info']['iteration']}")
        print(f"  Timesteps: {strategy['model_info']['total_timesteps']}")

3. Get Statistics for a Specific Strategy

import requests

UCB_SERVICE = "http://localhost:9001"
STRATEGY_ID = "MomentumStrategy"

# Get stats for a specific strategy
response = requests.get(f"{UCB_SERVICE}/api/stats/{STRATEGY_ID}")
if response.status_code == 200:
    strategy_stats = response.json()
    print(f"Strategy: {strategy_stats['id']}")
    print(f"Enabled: {strategy_stats['enabled']}")
    print(f"Has Model: {strategy_stats['has_model']}")

    if strategy_stats['has_model']:
        print(f"Model Path: {strategy_stats['model_path']}")
        model = strategy_stats['model_info']
        print(f"  Reward: {model['mean_reward']}")
        print(f"  Iteration: {model['iteration']}")

    if strategy_stats['is_training']:
        print(f"Currently Training: Yes")
        print(f"  Current Iteration: {strategy_stats['training_info']['current_iteration']}")
else:
    print(f"Strategy not found: {STRATEGY_ID}")

4. Check if a Model Exists (Simple)

import requests

UCB_SERVICE = "http://localhost:9001"
STRATEGY_ID = "MomentumStrategy"

# Check if model exists
response = requests.get(f"{UCB_SERVICE}/api/models/{STRATEGY_ID}")
if response.status_code == 200:
    model_info = response.json()
    print(f"Model found: {model_info}")
    print(f"  Best Reward: {model_info.get('mean_reward')}")
    print(f"  Iteration: {model_info.get('iteration')}")
    print(f"  Timesteps: {model_info.get('total_timesteps')}")
else:
    print("No model available for this strategy")

5. Download the Best Model

import requests
from pathlib import Path

UCB_SERVICE = "http://localhost:9001"
STRATEGY_ID = "MomentumStrategy"
LOCAL_MODEL_PATH = Path("./models")

def download_best_model(strategy_id: str) -> Path:
    """Download the best trained model for a strategy."""
    LOCAL_MODEL_PATH.mkdir(parents=True, exist_ok=True)

    # Download model
    response = requests.get(
        f"{UCB_SERVICE}/api/models/{strategy_id}/download",
        stream=True
    )

    if response.status_code != 200:
        raise Exception(f"Model not found: {response.status_code}")

    model_file = LOCAL_MODEL_PATH / f"{strategy_id}_best.zip"
    with open(model_file, 'wb') as f:
        for chunk in response.iter_content(chunk_size=8192):
            f.write(chunk)

    return model_file

# Usage
model_path = download_best_model("MomentumStrategy")
print(f"Model saved to: {model_path}")

6. Load and Use the Model for Trading

from stable_baselines3 import PPO
import numpy as np

# Load the downloaded model
model = PPO.load("./models/MomentumStrategy_best.zip")

# Create your trading environment (must match training env structure)
# The observation space should match what was used during training
# env = YourTradingEnvironment(candles)

# Get action from model
obs = env.reset()[0]
action, _ = model.predict(obs, deterministic=True)

# Action mapping:
# 0 = HOLD
# 1 = BUY/LONG
# 2 = SELL/SHORT
print(f"Predicted action: {action}")

7. WebSocket for Real-Time Training Monitoring

import asyncio
import websockets
import json

async def monitor_training():
    """Connect to WebSocket for real-time training updates."""
    uri = "ws://localhost:9001/ws"

    async with websockets.connect(uri) as ws:
        while True:
            message = await ws.recv()
            data = json.loads(message)

            if data["type"] == "ppo_training_update":
                print(f"Timestep: {data['timestep']}, FPS: {data.get('fps', 0)}")
            elif data["type"] == "iteration_complete":
                print(f"Iteration {data['iteration']} complete!")
                print(f"  Mean Reward: {data['mean_reward']:.2f}")
                print(f"  Best Reward: {data['best_reward']:.2f}")
                print(f"  New Best: {data['is_new_best']}")
            elif data["type"] == "training_continued":
                print(f"Continuing from iteration {data['from_iteration']}")
                print(f"  Previous best: {data['best_reward']:.2f}")
            elif data["type"] == "training_complete":
                print("Training finished!")
                break

# Run: asyncio.run(monitor_training())

8. Complete Integration Client Class

"""
UCBModelClient - Complete client for fetching trained models from UCB microservice.
Use this class from any other project/microservice to get trained RL models.
"""
import requests
from pathlib import Path
from typing import Optional, Dict, Any, List
import logging

logger = logging.getLogger(__name__)

class UCBModelClient:
    """Client for fetching trained models from UCB microservice."""

    def __init__(
        self,
        ucb_url: str = "http://localhost:9001",
        cache_dir: str = "./model_cache"
    ):
        self.ucb_url = ucb_url.rstrip('/')
        self.cache_dir = Path(cache_dir)
        self.cache_dir.mkdir(parents=True, exist_ok=True)
        self._models = {}  # Loaded model cache

    def is_available(self) -> bool:
        """Check if UCB service is running."""
        try:
            response = requests.get(f"{self.ucb_url}/api/status", timeout=2)
            return response.status_code == 200
        except Exception:
            return False

    def list_models(self) -> List[Dict[str, Any]]:
        """List all available trained models."""
        response = requests.get(f"{self.ucb_url}/api/models")
        response.raise_for_status()
        return response.json().get("models", [])

    def get_stats(self) -> Dict[str, Any]:
        """Get comprehensive statistics for all strategies and models."""
        response = requests.get(f"{self.ucb_url}/api/stats")
        response.raise_for_status()
        return response.json()

    def get_strategy_stats(self, strategy_id: str) -> Optional[Dict[str, Any]]:
        """Get detailed statistics for a specific strategy."""
        response = requests.get(f"{self.ucb_url}/api/stats/{strategy_id}")
        if response.status_code == 404:
            return None
        response.raise_for_status()
        return response.json()

    def get_best_model(self) -> Optional[Dict[str, Any]]:
        """Get the best performing model across all strategies."""
        stats = self.get_stats()
        best_strategy = stats['summary'].get('best_model_strategy')
        if not best_strategy:
            return None
        for strategy in stats['strategies']:
            if strategy['id'] == best_strategy and strategy.get('has_model'):
                return {
                    'strategy_id': strategy['id'],
                    'model_path': strategy['model_path'],
                    'model_info': strategy['model_info'],
                    'reward': strategy['model_info']['mean_reward']
                }
        return None

    def get_model_info(self, strategy_id: str) -> Optional[Dict[str, Any]]:
        """Get metadata for a specific model."""
        response = requests.get(f"{self.ucb_url}/api/models/{strategy_id}")
        if response.status_code == 404:
            return None
        response.raise_for_status()
        return response.json()

    def download_model(self, strategy_id: str, force: bool = False) -> Path:
        """
        Download model if not cached or if force=True.

        Args:
            strategy_id: The strategy identifier
            force: Re-download even if cached

        Returns:
            Path to the downloaded model file
        """
        model_path = self.cache_dir / f"{strategy_id}.zip"

        if model_path.exists() and not force:
            logger.info(f"Using cached model: {model_path}")
            return model_path

        logger.info(f"Downloading model for {strategy_id}...")
        response = requests.get(
            f"{self.ucb_url}/api/models/{strategy_id}/download",
            stream=True
        )
        response.raise_for_status()

        with open(model_path, 'wb') as f:
            for chunk in response.iter_content(chunk_size=8192):
                f.write(chunk)

        logger.info(f"Model saved to {model_path}")
        return model_path

    def load_model(self, strategy_id: str, env=None):
        """
        Download (if needed) and load a PPO model.

        Args:
            strategy_id: The strategy identifier
            env: Optional environment to attach to model

        Returns:
            Loaded PPO model
        """
        from stable_baselines3 import PPO

        if strategy_id in self._models and env is None:
            return self._models[strategy_id]

        model_path = self.download_model(strategy_id)
        model = PPO.load(str(model_path), env=env)

        if env is None:
            self._models[strategy_id] = model

        return model

    def get_action(
        self,
        strategy_id: str,
        observation,
        deterministic: bool = True
    ):
        """
        Get trading action from model given observation.

        Args:
            strategy_id: The strategy identifier
            observation: Current market observation
            deterministic: Use deterministic policy (recommended for trading)

        Returns:
            Action (0=HOLD, 1=BUY, 2=SELL)
        """
        model = self.load_model(strategy_id)
        action, _ = model.predict(observation, deterministic=deterministic)
        return action

    def start_training(
        self,
        strategy_id: str,
        symbol: str = "ETHUSDT",
        timeframe: str = "1h",
        backtest_years: int = 2,
        timesteps_per_iteration: int = 100000,
        max_iterations: int = 0
    ) -> Dict[str, Any]:
        """
        Start training for a strategy remotely.

        Args:
            strategy_id: Strategy to train
            symbol: Trading pair
            timeframe: Candle timeframe
            backtest_years: Years of historical data
            timesteps_per_iteration: PPO timesteps per iteration
            max_iterations: 0 = infinite

        Returns:
            API response
        """
        response = requests.post(
            f"{self.ucb_url}/api/training/start",
            json={
                "strategy_id": strategy_id,
                "symbol": symbol,
                "timeframe": timeframe,
                "backtest_years": backtest_years,
                "use_ppo": True,
                "timesteps_per_iteration": timesteps_per_iteration,
                "max_iterations": max_iterations
            }
        )
        response.raise_for_status()
        return response.json()

    def stop_training(self, strategy_id: str) -> Dict[str, Any]:
        """Stop training for a strategy."""
        response = requests.post(
            f"{self.ucb_url}/api/training/stop",
            json={"strategy_id": strategy_id}
        )
        response.raise_for_status()
        return response.json()


# Usage example:
if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)

    client = UCBModelClient()

    if not client.is_available():
        print("UCB service not running at localhost:9001")
        exit(1)

    # Get comprehensive statistics
    stats = client.get_stats()
    print(f"=== UCB Service Statistics ===")
    print(f"Total Strategies: {stats['summary']['total_strategies']}")
    print(f"Total Models: {stats['summary']['total_models']}")
    print(f"Active Trainings: {stats['summary']['active_trainings']}")
    print(f"Best Model: {stats['summary'].get('best_model_strategy', 'None')}")
    print(f"Best Reward: {stats['summary'].get('best_overall_reward', 'N/A')}")

    # Get the best model directly
    best = client.get_best_model()
    if best:
        print(f"\n=== Best Model Details ===")
        print(f"Strategy: {best['strategy_id']}")
        print(f"Path: {best['model_path']}")
        print(f"Reward: {best['reward']}")

    # List available models
    models = client.list_models()
    print(f"\nAvailable models: {[m['strategy_id'] for m in models]}")

    # Get model info
    info = client.get_model_info("MomentumStrategy")
    if info:
        print(f"\nModel info for MomentumStrategy:")
        print(f"  Best Reward: {info.get('mean_reward', 'N/A')}")
        print(f"  Iteration: {info.get('iteration', 'N/A')}")
        print(f"  Timesteps: {info.get('total_timesteps', 'N/A')}")

        # Download and load
        model = client.load_model("MomentumStrategy")
        print(f"\nModel loaded successfully!")
        print(f"  Policy: {model.policy}")

Model Storage Structure (Multi-Symbol)

Models are stored per strategy AND per symbol, allowing independent training on different trading pairs:

output/
├── MomentumRider-v1/
│   ├── ETHUSDT/
│   │   ├── best_model.zip           # PPO model trained on ETHUSDT
│   │   └── best_model_metadata.json
│   └── BTCUSDT/
│       ├── best_model.zip           # PPO model trained on BTCUSDT
│       └── best_model_metadata.json
├── ScalpingStrategy/
│   └── ETHUSDT/
│       ├── best_model.zip
│       └── best_model_metadata.json
└── ...

Metadata Format

{
  "strategy_id": "MomentumRider-v1",
  "symbol": "ETHUSDT",
  "mean_reward": 5460.60,
  "iteration": 12,
  "total_timesteps": 1200000,
  "saved_at": "2026-01-06T01:43:12.594895",
  "training_config": {
    "timeframe": "1h",
    "backtest_years": 2,
    "timesteps_per_iteration": 100000,
    "optimize_entry_exit": false
  },
  "trade_statistics": {
    "avg_win_pnl": 1.25,
    "avg_loss_pnl": -0.85,
    "total_pnl": 12.45,
    "win_rate": 58.5,
    "max_consecutive_wins": 7,
    "max_consecutive_losses": 4
  }
}

Note: The trade_statistics are captured from the best model only – they represent the performance of the model that achieved the highest reward, not aggregated stats across all training iterations.

Composite Keys

Internally, training sessions and models are tracked using composite keys: {strategy_id}_{symbol}

For example:

MomentumRider-v1_ETHUSDT
MomentumRider-v1_BTCUSDT

This ensures that training ETHUSDT and BTCUSDT simultaneously (or sequentially) keeps their models completely separate.

Multi-Symbol API Usage

Start Training on a Specific Symbol

import requests

UCB_SERVICE = "http://localhost:9001"

# Start training MomentumRider-v1 on ETHUSDT
response = requests.post(f"{UCB_SERVICE}/api/training/start", json={
    "strategy_id": "MomentumRider-v1",
    "symbol": "ETHUSDT",  # Required - specifies which trading pair
    "timeframe": "1h",
    "backtest_years": 2,
    "timesteps_per_iteration": 100000,
    "max_iterations": 0,  # 0 = infinite
    "use_ppo": True
})
print(response.json())

Stop/Pause Training (Must Include Symbol)

# Stop training - MUST include symbol
response = requests.post(f"{UCB_SERVICE}/api/training/stop", json={
    "strategy_id": "MomentumRider-v1",
    "symbol": "ETHUSDT"
})

# Pause training - MUST include symbol
response = requests.post(f"{UCB_SERVICE}/api/training/pause", json={
    "strategy_id": "MomentumRider-v1",
    "symbol": "ETHUSDT"
})

# Resume training - MUST include symbol
response = requests.post(f"{UCB_SERVICE}/api/training/resume", json={
    "strategy_id": "MomentumRider-v1",
    "symbol": "ETHUSDT"
})

List All Models (Multi-Symbol Response)

response = requests.get(f"{UCB_SERVICE}/api/models")
models = response.json()["models"]

# Response includes symbol for each model:
# [
#   {
#     "name": "MomentumRider-v1 (ETHUSDT)",
#     "strategy_id": "MomentumRider-v1",
#     "symbol": "ETHUSDT",
#     "path": "D:\\PoG\\UCB\\output\\MomentumRider-v1\\ETHUSDT\\best_model.zip",
#     "size_mb": 2.45,
#     "mean_reward": 5460.60,
#     "download_url": "/api/models/MomentumRider-v1/ETHUSDT/download"
#   },
#   {
#     "name": "MomentumRider-v1 (BTCUSDT)",
#     "strategy_id": "MomentumRider-v1",
#     "symbol": "BTCUSDT",
#     ...
#   }
# ]

Download a Specific Model (By Strategy + Symbol)

# Download ETHUSDT model
response = requests.get(
    f"{UCB_SERVICE}/api/models/MomentumRider-v1/ETHUSDT/download",
    stream=True
)

with open("model_ethusdt.zip", "wb") as f:
    for chunk in response.iter_content(chunk_size=8192):
        f.write(chunk)

# Download BTCUSDT model (same strategy, different symbol)
response = requests.get(
    f"{UCB_SERVICE}/api/models/MomentumRider-v1/BTCUSDT/download",
    stream=True
)

Get Statistics (Multi-Symbol)

response = requests.get(f"{UCB_SERVICE}/api/stats")
stats = response.json()

# summary now includes best_overall_symbol
print(f"Best Strategy: {stats['summary']['best_overall_strategy']}")
print(f"Best Symbol: {stats['summary']['best_overall_symbol']}")
print(f"Best Reward: {stats['summary']['best_overall_reward']}")

# Each strategy has a "models" array (not single "model")
for strategy in stats['strategies']:
    print(f"\n{strategy['name']}:")
    for model in strategy.get('models', []):
        print(f"  {model['symbol']}: reward={model.get('mean_reward', '--')}")

Legacy Model Migration

If you have models from before multi-symbol support (stored directly under output/{strategy_id}/), you can migrate them:

Check for Legacy Models

response = requests.get(f"{UCB_SERVICE}/api/models/legacy")
legacy = response.json()
print(f"Found {legacy['count']} legacy models to migrate")
for model in legacy['legacy_models']:
    print(f"  - {model['strategy_id']}: {model['path']}")

Migrate Legacy Models

# Migrate all legacy models to ETHUSDT (default symbol)
response = requests.post(
    f"{UCB_SERVICE}/api/models/migrate",
    params={"default_symbol": "ETHUSDT"}
)
result = response.json()
print(f"Migrated {result['total_migrated']} models")

This will move:

output/MomentumRider-v1/best_model.zip → output/MomentumRider-v1/ETHUSDT/best_model.zip

And update metadata to include the symbol field.

Training Continuation

When you restart the server and start training:

The service checks for existing best_model.zip in output/{strategy_id}/
If found, loads the model and reads metadata for:
- Previous iteration count
- Best reward achieved
- Total timesteps trained
Training continues from where it left off (iteration N+1)
UI shows “Continuing from saved model” notice
New models are only saved if they beat the previous best reward

WebSocket Message Types

Type	Description
`connection_established`	Initial connection with active trainings list
`training_started`	New training session started
`training_continued`	Resuming from existing model
`ppo_training_update`	Real-time training metrics (timesteps, fps, losses)
`iteration_start`	New iteration beginning
`iteration_complete`	Iteration finished with evaluation results
`training_complete`	Training session ended
`training_paused`	Training paused by user
`training_resumed`	Paused training resumed
`training_stopping`	Stop requested, gracefully shutting down
`training_error`	Error occurred during training

GPU Support

For optimal performance with GPU:

# Check GPU availability
python -c "import torch; print(f'CUDA: {torch.cuda.is_available()}')"

# Monitor GPU usage
nvidia-smi -l 1

Tested with:

NVIDIA RTX 4000 (16GB VRAM)
CUDA 11.8+
PyTorch 2.0+

Version History

v2.5.0 – Statistics page enhancements: Product-Strategy Matrix, Performance Matrix, Product Details
v2.4.0 – Multi-training UI with concurrent training support (tabbed interface)
v2.3.0 – Multi-symbol support: train same strategy on different trading pairs independently
v2.2.0 – Statistics API endpoints and Statistics UI page
v2.1.0 – Auto-continue from existing models on fresh server start
v2.0.0 – Multi-iteration PPO training with pause/resume
v1.0.0 – Initial release with UCB + PPO training

PoG Trader

Live trading microservice that executes trades on HyperLiquid using RL models trained by UCB.

Overview

PoG Trader connects to the UCB (Upper Confidence Bound) training service to fetch the best-performing reinforcement learning models and automatically executes trades on the HyperLiquid perpetual futures exchange.

Key Features

Auto-Trading: Automatically trades when new best models are detected from UCB
Single Strategy Per Product: Only the highest-reward strategy is enabled for each product (ETH, BTC, etc.)
Auto Strategy Selection: When a better strategy is imported from UCB, it automatically becomes the active strategy
Market Locking: Only one strategy can hold a position per market (perpetual futures rule)
Priority by Reward: Higher reward strategies get priority when competing for the same market
Real-time Monitoring: Live dashboard with positions, PnL, and trade history
Risk Management: Per-trade risk limits, stop losses, and kill switches
Position Sync: Detects externally closed positions and updates state accordingly
Persistent State: Positions and settings survive server restarts
Trade CSV Export: All completed trades are exported to trades.csv in the working directory

Architecture

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   UCB Service   │────▶│   PoG Trader    │────▶│   HyperLiquid   │
│   (Port 9001)   │     │   (Port 9002)   │     │    Exchange     │
└─────────────────┘     └─────────────────┘     └─────────────────┘
        │                       │
        │ Models & Signals      │ Orders & Positions
        ▼                       ▼
   RL Model Files          SQLite Database

Installation

Clone the repository:

git clone https://github.com/sirdavidjcoops/PoGTrader.git
cd PoGTrader

Install dependencies:

pip install -r requirements.txt

Copy and configure environment:

cp .env.example .env
# Edit .env with your HyperLiquid credentials

Run the service:

python run.py
# Or on Windows:
start.bat

Configuration

Environment Variables (.env)

# HyperLiquid Credentials
HL_MAIN_WALLET_ADDRESS=0x...
HL_API_WALLET_ADDRESS=0x...
HL_PRIVATE_KEY=0x...
HL_TESTNET=false

# UCB Service
UCB_SERVICE_URL=http://localhost:9001

# Risk Settings
DEFAULT_RISK_PERCENT=0.01
DEFAULT_STOP_LOSS=5
DEFAULT_MAX_STRATEGY_LOSS=50

Risk Settings

Setting	Default	Description
Risk Per Trade	1%	Percentage of equity used as margin
Stop Loss	$5	Maximum loss per trade in dollars
Max Strategy Loss	$50	Kill switch threshold per strategy

Trading Logic

Signal Processing

Models output actions:

0 = HOLD – No action
1 = BUY – Open/maintain long position
2 = SELL – Open/maintain short position
3 = EXIT – Close any open position

Market Locking (Perpetual Futures)

Since perpetual futures net positions together, only one strategy can trade each market at a time:

When a strategy opens a position, that market is “locked” to that strategy
Other strategies cannot open positions on that market until the lock is released
Higher reward strategies get priority
Lock is released when the position is closed (by signal, stop loss, or manually)

Position Sizing

Margin = Equity × Risk Per Trade (1%)
Notional = Margin × Leverage (e.g., 25x for ETH)
Size = Notional / Entry Price

Example with $1000 equity, ETH at $3500:

Margin = $1000 × 0.01 = $10
Notional = $10 × 25 = $250
Size = $250 / $3500 = 0.071 ETH

API Endpoints

System

Method	Endpoint	Description
GET	`/`	Serve dashboard UI
GET	`/api/status`	System status
GET	`/api/health`	Health check

Strategies

Method	Endpoint	Description
GET	`/api/strategies`	List all strategies
POST	`/api/strategies/{id}/{symbol}/enable`	Enable trading
POST	`/api/strategies/{id}/{symbol}/disable`	Disable trading
PUT	`/api/strategies/{id}/{symbol}/settings`	Update risk settings

Positions

Method	Endpoint	Description
GET	`/api/positions`	List open positions
POST	`/api/positions/{id}/close`	Close a position
POST	`/api/positions/close-all`	Close all positions

Trades

Method	Endpoint	Description
GET	`/api/trades`	Trade history
GET	`/api/trades/stats`	Trade statistics
POST	`/api/test-trade`	Execute test trade

Orders

Method	Endpoint	Description
GET	`/api/orders`	List working orders (stop losses, limits)
DELETE	`/api/orders/{id}?symbol=X`	Cancel an order

Equity

Method	Endpoint	Description
GET	`/api/equity`	Get equity curve snapshots
POST	`/api/equity/snapshot`	Manually record snapshot

Analytics

Method	Endpoint	Description
GET	`/api/analytics`	Comprehensive trading analytics for microservices

`/api/analytics` Response Schema

{
  "account": {
    "equity": 156.00,
    "available_balance": 150.00,
    "open_pnl": 5.00,
    "starting_capital": 156.0
  },
  "pnl": {
    "lifetime": 25.50,
    "daily": 5.00,
    "weekly": 15.00,
    "monthly": 25.50,
    "return_pct": 16.35
  },
  "performance": {
    "total_trades": 10,
    "winning_trades": 6,
    "losing_trades": 4,
    "win_rate": 60.0,
    "avg_pnl": 2.55,
    "avg_rr": 1.85,
    "best_trade_pnl": 12.50,
    "worst_trade_pnl": -5.20
  },
  "streaks": {
    "current_streak": 2,
    "max_consecutive_wins": 4,
    "max_consecutive_losses": 2,
    "consecutive_win_pnl": 18.50,
    "consecutive_loss_pnl": -8.20
  },
  "by_strategy": {
    "QuasiINVH&S-v1": {
      "total_trades": 5,
      "winning_trades": 3,
      "win_rate": 60.0,
      "avg_rr": 1.75,
      "avg_pnl": 2.10,
      "total_gain_pnl": 15.50,
      "total_loss_pnl": -5.00,
      "max_consecutive_wins": 2,
      "max_consecutive_losses": 1
    }
  }
}

## WebSocket

Connect to ws://localhost:9002/ws for real-time updates:

```javascript
// Message types
{ type: "connected", status: "ready" }
{ type: "position_opened", position: {...} }
{ type: "position_updated", position_id: 1, pnl: 45.23 }
{ type: "position_closed", trade: {...} }
{ type: "signal", strategy_id: "...", action: 1 }
{ type: "kill_switch_triggered", strategy_id: "..." }
{ type: "log", level: "INFO", message: "..." }

Database Schema

SQLite database at data/trader.db:

positions – Open and closed positions
trades – Completed trade records
strategy_configs – Strategy settings and state
action_logs – System activity logs
equity_snapshots – Hourly equity curve data

Dashboard

Access the web UI at http://localhost:9002

Features

Equity Curve – Historical equity chart with hourly snapshots
Open PnL – Total unrealized profit/loss
Closed PnL – Total realized profit/loss with % return
Avg R:R – Average risk-to-reward ratio across all trades
Active Strategies – Strategy cards with model info and settings
Open Positions – Current positions with live PnL
Working Orders – Stop loss and limit orders on the exchange
Trade History – Completed trades with stats
Action Log – Real-time system activity

Controls

Enable/Disable strategies
Adjust risk settings per strategy
Close individual or all positions
Execute test trades

Development

Project Structure

PoGTrader/
├── run.py                    # Entry point
├── requirements.txt          # Dependencies
├── backend/
│   ├── main.py              # FastAPI app
│   ├── config.py            # Configuration
│   ├── exchange/            # HyperLiquid client
│   ├── ucb/                 # UCB service client
│   ├── trading/             # Trading logic
│   └── db/                  # Database models
├── frontend/
│   ├── index.html           # Dashboard
│   └── static/
│       ├── styles.css       # Starlink theme
│       └── app.js           # Frontend app
└── data/
    ├── trader.db            # SQLite database
    ├── logs/                # Log files
    └── models/              # Downloaded models

Running Tests

# Test exchange connectivity
curl http://localhost:9002/api/test-trade -X POST

# Check system status
curl http://localhost:9002/api/status

Changelog

v1.7.0 (2026-01-08)

Analytics Enhancement: /api/analytics now includes all active strategies (even those without trades yet)
Strategy Stats: Each strategy in by_strategy now includes model_reward, symbol, and enabled status
Terminal Title: Window title now shows project name and port for easy identification
Dynamic Config: Log messages now use configured project name instead of hardcoded strings

v1.6.0 (2026-01-07)

Single Strategy Per Product: Only the highest-reward strategy is enabled per product (ETH, BTC, SOL, etc.)
Auto Strategy Selection: When UCB imports a better strategy, it automatically enables it and disables the previous best
Graceful Transition: If a lower-reward strategy has an open position, it keeps it until closed before the new strategy trades
Trade CSV Export: All completed trades are now exported to trades.csv in the working directory
UI Cleanup: Removed duplicate Active Strategies section from main dashboard (now only in sidebar)

v1.5.0 (2026-01-07)

UCB Model Sync Fix: Fixed race condition where WebSocket callback was registered after connection
Model Detection Fix: Models without exists flag are now included if they have download URL
Auto-Activation: New best models are immediately activated for trading after sync
Model Integrity: Corrupted model files are automatically detected and cleaned up
Metadata Retry: Failed downloads/loads clear metadata to ensure retry on next sync
UI Model Notifications: Frontend now shows success message when models are updated
Dashboard Reorder: Active Strategies moved below Open Positions and Working Orders
Trade History Scroll: Trade history table is now scrollable with sticky header

v1.4.0 (2026-01-07)

Analytics Endpoint: New /api/analytics endpoint for microservice consumption
Stats Overview: New dashboard section with comprehensive trading metrics
Time-Period PnL: Daily, weekly, and monthly PnL tracking
Consecutive Wins/Losses: Track max win/loss streaks with total PnL
Per-Strategy Stats: Breakdown by strategy with individual metrics
R:R Fix: Average R:R now only considers winning trades (meaningful metric)
Best/Worst Trade: Track your best and worst trades

v1.3.0 (2026-01-07)

Stop Loss Fix: Fixed trigger order format causing SDK errors
Stop Loss Retry: Added 3-attempt retry logic for stop loss placement
Software Stop Loss: Fallback stop loss monitoring if exchange order fails
Working Orders: New dashboard section showing all resting/trigger orders
Cancel Orders: Ability to cancel working orders from the UI
Closed PnL: Added realized PnL display in sidebar with % return
Avg R:R: Added average risk-to-reward ratio display
API Endpoints: Added /api/orders GET and DELETE endpoints

v1.2.0 (2026-01-07)

Equity Curve: Added hourly equity snapshots and Chart.js visualization
Risk Validation: Stop loss capped at 50% to prevent invalid negative prices
Database Migration: Improved migration to update all strategies with old risk defaults
API Endpoints: Added /api/equity and /api/equity/snapshot endpoints

v1.1.0 (2026-01-06)

Market Locking: Only one strategy per market (perpetual futures support)
Priority by Reward: Higher reward strategies get trading priority
External Close Detection: Automatically detects positions closed on exchange
Improved Position Sync: Better startup reconciliation with exchange
UI Improvements: Combined dashboard view, PnL in sidebar
Bug Fixes: Fixed Close All button, fixed observation format

v1.0.0 (Initial Release)

Core trading engine with UCB integration
HyperLiquid exchange support
Real-time monitoring dashboard
Risk management with kill switches
Position and trade persistence

TraderLighter

v1.1.0 – Enhanced Order Management & Performance

Live trading microservice that executes trades on Lighter Exchange using RL models trained by UCB.

Overview

TraderLighter is adapted from PoGTrader to trade on Lighter Exchange instead of HyperLiquid. Lighter is a ZK rollup-based perpetual futures exchange built on Ethereum L2.

Key Features

Auto-Trading: Automatically trades when new best models are detected from UCB
Single Strategy Per Product: Only the highest-reward strategy is enabled for each product (ETH, BTC, etc.)
Auto Strategy Selection: When a better strategy is imported from UCB, it automatically becomes the active strategy
Market Locking: Only one strategy can hold a position per market (perpetual futures rule)
Priority by Reward: Higher reward strategies get priority when competing for the same market
Real-time Monitoring: Live dashboard with positions, PnL, and trade history
Risk Management: Per-trade risk limits, stop losses, and kill switches
Position Sync: Detects externally closed positions and updates state accordingly
Persistent State: Positions and settings survive server restarts
Trade CSV Export: All completed trades are exported to trades.csv
Per-Market Decimal Scaling: Automatic position size scaling based on each market’s precision
Offline Model Loading: Models load from disk when UCB service is unavailable

Architecture

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   UCB Service   │────▶│  TraderLighter  │────▶│    Lighter      │
│   (Port 9001)   │     │   (Port 9102)   │     │    Exchange     │
└─────────────────┘     └─────────────────┘     └─────────────────┘
        │                       │
        │ Models & Signals      │ Orders & Positions
        ▼                       ▼
   RL Model Files          SQLite Database

Key Differences from HyperLiquid

Signing Mechanism: Lighter uses client-side transaction signing via native Go binaries (ctypes FFI)
Nonce Management: Each API key has its own nonce sequence that must increment properly
Market Indices: Markets are identified by index (0=ETH, 1=BTC, etc.) rather than symbol strings
L2 Architecture: Built on Ethereum ZK rollup for verifiable trades
Per-Market Decimals: Each market has different supported_size_decimals and supported_price_decimals

Market Decimal Precision

Lighter uses integer-based order sizes with per-market scaling:

Market	Size Decimals	Price Decimals	Size Scale
ETH	4	2	10000
BTC	5	2	100000
SOL	3	2	1000
HYPE	2	2	100
LINK	1	2	10
FIL	1	2	10

TraderLighter automatically fetches these from the API on startup and applies correct scaling.

Installation

Prerequisites

Python 3.9+
Lighter Exchange account with API keys

Setup

Clone the repository:

git clone https://github.com/sirdavidjcoops/PoGTraderLighter.git
cd PoGTraderLighter

Create virtual environment:

python -m venv venv
source venv/bin/activate  # Linux/Mac
# or
venv\Scripts\activate  # Windows

Install dependencies:

pip install -r requirements.txt

Install Lighter SDK from GitHub:

pip install git+https://github.com/elliottech/lighter-python.git

Configure environment:

cp .env.example .env
# Edit .env with your Lighter credentials

Run the application:

python run.py
# Or on Windows:
start.bat

Access the dashboard at: http://localhost:9102

Configuration

Environment Variables (.env)

# Lighter Exchange Credentials
LIGHTER_ETH_ADDRESS=0x...           # Your Ethereum wallet address
LIGHTER_ACCOUNT_INDEX=...           # Account index from Lighter
LIGHTER_API_KEY_INDEX=3             # API key index (0-254)
LIGHTER_PRIVATE_KEY=...             # API private key
LIGHTER_PUBLIC_KEY=...              # API public key
LIGHTER_READONLY_TOKEN=ro:...       # ReadOnly access token
LIGHTER_BASE_URL=https://mainnet.zklighter.elliot.ai

# UCB Service
UCB_SERVICE_URL=http://localhost:9001

# Risk Settings
DEFAULT_RISK_PERCENT=0.01           # 1% of equity per trade
DEFAULT_STOP_LOSS_DOLLARS=5.0       # $5 stop loss
DEFAULT_MAX_STRATEGY_LOSS=50.0      # $50 kill switch

Symbol Mapping

UCB models are trained on Bybit data (e.g., ETHUSDT). TraderLighter maps these to Lighter format:

UCB/Bybit	Lighter	Market Index
ETHUSDT	ETH	0
BTCUSDT	BTC	1
SOLUSDT	SOL	2
AVAXUSDT	AVAX	3
DOGEUSDT	DOGE	4
LINKUSDT	LINK	5
ARBUSDT	ARB	6
OPUSDT	OP	7

Risk Settings

Setting	Default	Description
Risk Per Trade	1%	Percentage of equity used as margin
Stop Loss	$5	Maximum loss per trade in dollars
Max Strategy Loss	$50	Kill switch threshold per strategy

Trading Logic

Signal Processing

Models output actions:

0 = HOLD – No action
1 = BUY – Open/maintain long position
2 = SELL – Open/maintain short position
3 = EXIT – Close any open position

Position Sizing

Margin = Equity × Risk Per Trade (1%)
Notional = Margin × Leverage (e.g., 25x for ETH)
Size = Notional / Entry Price

API Endpoints

System

Method	Endpoint	Description
GET	`/`	Serve dashboard UI
GET	`/api/status`	System status
GET	`/api/health`	Health check

Strategies

Method	Endpoint	Description
GET	`/api/strategies`	List all strategies
POST	`/api/strategies/{id}/{symbol}/enable`	Enable trading
POST	`/api/strategies/{id}/{symbol}/disable`	Disable trading
PUT	`/api/strategies/{id}/{symbol}/settings`	Update risk settings

Positions

Method	Endpoint	Description
GET	`/api/positions`	List open positions
POST	`/api/positions/{id}/close`	Close a position
POST	`/api/positions/close-all`	Close all positions

Trades

Method	Endpoint	Description
GET	`/api/trades`	Trade history
GET	`/api/trades/stats`	Trade statistics
GET	`/api/analytics`	Comprehensive analytics

Testing

Method	Endpoint	Description
POST	`/api/test-trade`	Execute test trade

WebSocket

Connect to ws://localhost:9102/ws for real-time updates.

Known Issues

See ClaudeCommonErrors.md for known issues and solutions.

Bug Reporting

See BugReport.md for the bug report template.

Project Structure

TraderLighter/
├── run.py                    # Entry point
├── requirements.txt          # Dependencies
├── backend/
│   ├── main.py              # FastAPI app
│   ├── config.py            # Configuration
│   ├── exchange/            # Lighter client
│   ├── ucb/                 # UCB service client
│   ├── trading/             # Trading logic
│   └── db/                  # Database models
├── frontend/
│   ├── index.html           # Dashboard
│   └── static/
│       ├── styles.css       # Styles
│       └── app.js           # Frontend app
└── data/
    ├── trader.db            # SQLite database
    ├── logs/                # Log files
    └── models/              # Downloaded models

Changelog

v1.1.0 (2026-01-08)

Clear Trade History: New DELETE /api/trades/clear endpoint to remove bad trade records (e.g., from position sizing bugs)
Stop Loss Verification Loop: Background task verifies all positions have stop losses (disabled by default due to API limitations)
Candle Caching: Reduced API calls with 55-second cache for market candles
Orders Debug Endpoint: New /api/orders/debug for detailed order troubleshooting
Optimized Order Queries: Now only queries orders for markets with open positions
Stop Loss Label: Orders now display “STOP_LOSS” type instead of generic “TRIGGER”
Terminal Title: Window title shows project name and port for easy identification
Analytics Enhancement: Active strategies included even without completed trades

v1.0.0 (2026-01-08)

Major working release with critical bug fixes:

Fixed position sizing: Now uses per-market decimal scaling from Lighter API instead of hardcoded values. Previously HYPE trades were 100x too large.
Fixed stop loss for short positions: Stop loss orders now correctly placed for SELL trades by passing position side directly instead of fetching from exchange.
Added offline model loading: Models now load from disk on startup when UCB service is unavailable.
Improved market data: Fetches supported_size_decimals and supported_price_decimals per market from Lighter API.
Better error handling: Improved logging and error messages throughout.

UCB Trading Strategy RL Microservice

Features

Quick Start

Architecture

API Endpoints

Training Control

Models (Multi-Symbol)

Strategies

Statistics

System

WebSocket

Connecting from Another Application

1. Check if Service is Running

2. Get Statistics Overview

3. Get Statistics for a Specific Strategy

4. Check if a Model Exists (Simple)

5. Download the Best Model

6. Load and Use the Model for Trading

7. WebSocket for Real-Time Training Monitoring

8. Complete Integration Client Class

Model Storage Structure (Multi-Symbol)

Metadata Format

Composite Keys

Multi-Symbol API Usage

Start Training on a Specific Symbol

Stop/Pause Training (Must Include Symbol)

List All Models (Multi-Symbol Response)

Download a Specific Model (By Strategy + Symbol)

Get Statistics (Multi-Symbol)

Legacy Model Migration

Check for Legacy Models

Migrate Legacy Models

Training Continuation

WebSocket Message Types

GPU Support

Version History

PoG Trader

Overview

Key Features

Architecture

Installation

Configuration

Environment Variables (.env)

Risk Settings

Trading Logic

Signal Processing

Market Locking (Perpetual Futures)

Position Sizing

API Endpoints

System

Strategies

Positions

Trades

Orders

Equity

Analytics

/api/analytics Response Schema

Database Schema

Dashboard

Features

Controls

Development

Project Structure

Running Tests

Changelog

v1.7.0 (2026-01-08)

v1.6.0 (2026-01-07)

v1.5.0 (2026-01-07)

v1.4.0 (2026-01-07)

v1.3.0 (2026-01-07)

v1.2.0 (2026-01-07)

v1.1.0 (2026-01-06)

v1.0.0 (Initial Release)

TraderLighter

Overview

Key Features

Architecture

Key Differences from HyperLiquid

Market Decimal Precision

Installation

`/api/analytics` Response Schema