bft-crdt-experiment/docs/oracle-deployment-guide.md

# BFT-CRDT Oracle Network Deployment Guide

## Overview

This guide walks through deploying a production-ready BFT-CRDT oracle network that provides decentralized, manipulation-resistant price feeds without consensus overhead.

## Architecture Overview

```
┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   Oracle Node   │     │   Oracle Node   │     │   Oracle Node   │
│   (Region A)    │◄───►│   (Region B)    │◄───►│   (Region C)    │
└────────┬────────┘     └────────┬────────┘     └────────┬────────┘
         │                       │                       │
         ▼                       ▼                       ▼
┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│  Data Sources   │     │  Data Sources   │     │  Data Sources   │
│ • Binance       │     │ • Coinbase      │     │ • Kraken        │
│ • Uniswap       │     │ • Curve         │     │ • dYdX          │
└─────────────────┘     └─────────────────┘     └─────────────────┘
         
                    ▼ All Attestations ▼
                    
┌──────────────────────────────────────────────────────────────────┐
│                        BFT-CRDT Network                           │
│  • No consensus required                                          │
│  • Byzantine fault tolerant                                       │
│  • Eventual consistency                                           │
└──────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌──────────────────────────────────────────────────────────────────┐
│                      Smart Contracts                              │
│  • Aggregate prices on-demand                                     │
│  • Custom aggregation strategies                                  │
│  • Outlier detection                                              │
└──────────────────────────────────────────────────────────────────┘
```

## Prerequisites

### Hardware Requirements

**Minimum Requirements per Oracle Node:**
- CPU: 4 cores @ 2.5GHz
- RAM: 8GB
- Storage: 100GB SSD
- Network: 100Mbps dedicated bandwidth

**Recommended Requirements:**
- CPU: 8 cores @ 3.0GHz
- RAM: 16GB
- Storage: 500GB NVMe SSD
- Network: 1Gbps dedicated bandwidth

### Software Requirements

- Rust 1.70+ (for oracle node)
- Docker 20.10+ (optional, for containerized deployment)
- PostgreSQL 14+ (for local state persistence)
- Node.js 18+ (for monitoring dashboard)

## Step 1: Oracle Node Setup

### 1.1 Install Dependencies

```bash
# Ubuntu/Debian
sudo apt update
sudo apt install -y build-essential pkg-config libssl-dev postgresql postgresql-contrib

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env
```

### 1.2 Clone and Build Oracle Node

```bash
git clone https://github.com/your-org/bft-crdt-oracle
cd bft-crdt-oracle
cargo build --release
```

### 1.3 Generate Oracle Identity

```bash
# Generate new oracle keypair
./target/release/oracle-node keygen --output oracle-key.json

# Extract public key for registration
./target/release/oracle-node show-pubkey --key oracle-key.json
```

### 1.4 Configure Oracle Node

Create `config.toml`:

```toml
[oracle]
id = "oracle_prod_1"
key_file = "./oracle-key.json"

[network]
# P2P settings
listen_addr = "0.0.0.0:9000"
bootstrap_peers = [
    "/ip4/oracle1.network.com/tcp/9000/p2p/QmPeerId1...",
    "/ip4/oracle2.network.com/tcp/9000/p2p/QmPeerId2...",
    "/ip4/oracle3.network.com/tcp/9000/p2p/QmPeerId3..."
]

[data_sources]
# Exchange APIs
[[data_sources.exchanges]]
name = "binance"
api_url = "https://api.binance.com/api/v3"
weight = 25

[[data_sources.exchanges]]
name = "coinbase"
api_url = "https://api.coinbase.com/v2"
api_key = "${COINBASE_API_KEY}"
api_secret = "${COINBASE_API_SECRET}"
weight = 25

# On-chain sources
[[data_sources.on_chain]]
name = "uniswap_v3"
chain = "ethereum"
rpc_url = "${ETH_RPC_URL}"
weight = 20

[[data_sources.on_chain]]
name = "curve"
chain = "ethereum"
rpc_url = "${ETH_RPC_URL}"
weight = 15

# Other oracles (for cross-validation)
[[data_sources.oracles]]
name = "chainlink"
chain = "ethereum"
contract = "0x5f4eC3Df9cbd43714FE2740f5E3616155c5b8419"
weight = 15

[submission]
# Price submission settings
min_sources = 3
max_price_age_seconds = 30
submission_interval_seconds = 5
confidence_threshold = 0.95

[monitoring]
# Metrics and monitoring
metrics_port = 9091
log_level = "info"
```

### 1.5 Set Up Database

```bash
# Create database
sudo -u postgres createdb oracle_node
sudo -u postgres createuser oracle_user -P

# Initialize schema
psql -U oracle_user -d oracle_node < schema.sql
```

## Step 2: Data Source Integration

### 2.1 Exchange API Integration

Create `src/data_sources/binance.rs`:

```rust
use async_trait::async_trait;
use reqwest::Client;
use serde::Deserialize;

#[derive(Deserialize)]
struct BinanceTickerResponse {
    symbol: String,
    price: String,
}

pub struct BinanceClient {
    client: Client,
    api_url: String,
}

#[async_trait]
impl DataSource for BinanceClient {
    async fn fetch_price(&self, pair: &str) -> Result<PriceData, Error> {
        let symbol = convert_pair_format(pair); // ETH/USD -> ETHUSDT
        let url = format!("{}/ticker/price?symbol={}", self.api_url, symbol);
        
        let resp: BinanceTickerResponse = self.client
            .get(&url)
            .send()
            .await?
            .json()
            .await?;
            
        Ok(PriceData {
            source: "binance",
            price: parse_price(&resp.price)?,
            volume: self.fetch_volume(&symbol).await?,
            timestamp: current_timestamp(),
        })
    }
}
```

### 2.2 On-Chain Data Source

```rust
use ethers::prelude::*;

pub struct UniswapV3Client {
    provider: Provider<Http>,
    pool_address: Address,
}

impl UniswapV3Client {
    async fn fetch_price(&self, pair: &str) -> Result<PriceData, Error> {
        // Get pool slot0 for current price
        let pool = IUniswapV3Pool::new(self.pool_address, self.provider.clone());
        let slot0 = pool.slot_0().call().await?;
        
        // Calculate price from sqrtPriceX96
        let price = calculate_price_from_sqrt(slot0.0, decimals0, decimals1);
        
        Ok(PriceData {
            source: "uniswap_v3",
            price,
            volume: self.fetch_24h_volume().await?,
            timestamp: current_timestamp(),
        })
    }
}
```

## Step 3: Smart Contract Deployment

### 3.1 Deploy Oracle Registry

```solidity
// Deploy OracleRegistry.sol
contract OracleRegistry {
    mapping(address => OracleInfo) public oracles;
    
    struct OracleInfo {
        string peerId;
        uint256 stake;
        uint256 reputation;
        bool active;
    }
    
    function registerOracle(string memory peerId) external payable {
        require(msg.value >= MIN_STAKE, "Insufficient stake");
        oracles[msg.sender] = OracleInfo({
            peerId: peerId,
            stake: msg.value,
            reputation: INITIAL_REPUTATION,
            active: true
        });
    }
}
```

### 3.2 Deploy Price Aggregator

```solidity
// Deploy PriceAggregator.sol with oracle network interface
contract PriceAggregator {
    IOracleNetwork public oracleNetwork;
    
    constructor(address _oracleNetwork) {
        oracleNetwork = IOracleNetwork(_oracleNetwork);
    }
    
    // ... aggregation logic
}
```

## Step 4: Running the Oracle Network

### 4.1 Start Oracle Node

```bash
# Set environment variables
export DATABASE_URL="postgresql://oracle_user:password@localhost/oracle_node"
export COINBASE_API_KEY="your-api-key"
export ETH_RPC_URL="https://eth-mainnet.g.alchemy.com/v2/your-key"

# Run oracle node
./target/release/oracle-node run --config config.toml
```

### 4.2 Docker Deployment

Create `Dockerfile`:

```dockerfile
FROM rust:1.70 as builder
WORKDIR /app
COPY . .
RUN cargo build --release

FROM debian:bullseye-slim
RUN apt-get update && apt-get install -y libssl1.1 ca-certificates
COPY --from=builder /app/target/release/oracle-node /usr/local/bin/
COPY config.toml /etc/oracle/
CMD ["oracle-node", "run", "--config", "/etc/oracle/config.toml"]
```

Deploy with Docker Compose:

```yaml
version: '3.8'

services:
  oracle:
    build: .
    environment:
      - DATABASE_URL=postgresql://oracle:password@db/oracle_node
      - COINBASE_API_KEY=${COINBASE_API_KEY}
      - ETH_RPC_URL=${ETH_RPC_URL}
    ports:
      - "9000:9000"  # P2P
      - "9091:9091"  # Metrics
    depends_on:
      - db
    restart: unless-stopped

  db:
    image: postgres:14
    environment:
      - POSTGRES_DB=oracle_node
      - POSTGRES_USER=oracle
      - POSTGRES_PASSWORD=password
    volumes:
      - oracle_data:/var/lib/postgresql/data

volumes:
  oracle_data:
```

## Step 5: Monitoring and Maintenance

### 5.1 Set Up Monitoring

```yaml
# prometheus.yml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'oracle-nodes'
    static_configs:
      - targets: 
        - 'oracle1:9091'
        - 'oracle2:9091'
        - 'oracle3:9091'
```

### 5.2 Key Metrics to Monitor

- **Attestation Rate**: Should be consistent (e.g., 0.2-1 Hz)
- **Data Source Availability**: Track failures per source
- **Price Deviation**: Monitor for outliers
- **Network Connectivity**: P2P peer count
- **CRDT Merge Rate**: Should see regular merges
- **Memory Usage**: CRDTs grow over time

### 5.3 Alerting Rules

```yaml
groups:
  - name: oracle_alerts
    rules:
      - alert: OracleOffline
        expr: up{job="oracle-nodes"} == 0
        for: 5m
        
      - alert: LowDataSources
        expr: oracle_active_sources < 3
        for: 10m
        
      - alert: HighPriceDeviation
        expr: oracle_price_deviation > 0.05
        for: 5m
```

## Step 6: Security Best Practices

### 6.1 Key Management

- Use hardware security modules (HSM) for production keys
- Implement key rotation every 90 days
- Never expose private keys in logs or configs

### 6.2 API Security

- Use API rate limiting for all data sources
- Implement circuit breakers for failing sources
- Validate all external data with multiple sources

### 6.3 Network Security

- Enable TLS for all P2P connections
- Implement peer authentication
- Use firewalls to restrict access

### 6.4 Operational Security

```bash
# Regular security audit
./oracle-node audit --check-sources --verify-attestations

# Backup critical data
pg_dump -U oracle_user oracle_node > backup_$(date +%Y%m%d).sql

# Monitor for anomalies
tail -f /var/log/oracle/oracle.log | grep -E "(ERROR|WARN|ANOMALY)"
```

## Step 7: Scaling Considerations

### 7.1 Horizontal Scaling

- Add more oracle nodes in different regions
- Use GeoDNS for regional data source routing
- Implement sharding by asset pairs if needed

### 7.2 Performance Optimization

```toml
[performance]
# Tune for high throughput
attestation_batch_size = 100
merge_interval_ms = 500
cache_ttl_seconds = 30
max_memory_gb = 8
```

### 7.3 Cost Optimization

- Cache frequently accessed prices
- Batch RPC calls to reduce costs
- Use websocket connections where available
- Implement adaptive submission frequency

## Troubleshooting

### Common Issues

1. **Oracle not submitting prices**
   - Check data source connectivity
   - Verify API keys are valid
   - Ensure minimum sources threshold is met

2. **High memory usage**
   - Implement CRDT pruning for old attestations
   - Adjust cache sizes
   - Monitor for memory leaks

3. **Network partition**
   - Ensure bootstrap nodes are accessible
   - Check firewall rules
   - Verify P2P port is open

## Conclusion

A properly deployed BFT-CRDT oracle network provides:
- Decentralized price feeds without consensus overhead
- Byzantine fault tolerance
- High availability and partition tolerance
- Cost-effective operation

Regular monitoring and maintenance ensure reliable operation for DeFi protocols depending on accurate price data.