Building production-ready crypto price feeds with DexPaprika: handling disconnects, backpressure, and spikes

Streaming setups work perfectly in development. Load tests pass. Everything looks solid. Then production hits, and within an hour, users report stale prices. Connections drop. Nothing changed in your code, yet everything breaks.

This tutorial will teach you to build price feeds that survive production. You will implement real-time cryptocurrency streaming with DexPaprika while handling the six most common production failures: reconnection storms, proxy timeouts, memory leaks, mobile disconnections, corporate firewalls, and cost optimization.

What you will learn

In this tutorial, you will:

• Implement reconnection logic that prevents infrastructure collapse
• Configure nginx and heartbeats to prevent proxy timeouts
• Build bounded buffers that prevent memory leaks
• Optimize streaming connections for mobile networks
• Add SSE fallback for corporate firewall environments
• Optimize cloud infrastructure costs for real-time streaming

Prerequisites

To follow this tutorial, you will need:

• Node.js 18+ installed
• Basic understanding of EventSource (SSE) or WebSocket APIs
• Familiarity with async/await JavaScript patterns
• No API key required (DexPaprika streaming is completely free)

Quick start

Get live WETH price streaming in 30 seconds:

// Connect to DexPaprika's free streaming endpoint
import { EventSource } from 'eventsource';

const wethStream = new EventSource(
  'https://streaming.dexpaprika.com/stream?method=t_p&chain=ethereum&address=0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2'
);

wethStream.onmessage = (event) => {
  const data = JSON.parse(event.data);
  console.log(`WETH: $${data.price} (${data.price_change_24h}%)`);
};

wethStream.onerror = (error) => {
  console.error('Stream error:', error);
  wethStream.close();
};

This basic implementation connects to DexPaprika's WETH price stream. The endpoint updates approximately every second with current price data from Ethereum DEXs. No API key, no credit card, no rate limits.

However, this code will fail in production under six specific conditions. The following sections will transform this basic stream into a production-ready implementation that handles real-world failures.

Section 1: Preventing reconnection storms with exponential backoff

The immediate reconnection pattern destroys infrastructure during deployments. When 10,000 clients disconnect simultaneously (server restart, network hiccup), they all attempt to reconnect at the exact same moment. The load balancer receives 10,000 connection requests in under 100 milliseconds. New servers spawn and crash from the load. Auto-scaling spawns more. Those crash too. The loop continues until manual intervention.

Understanding the failure pattern

Server restarts cause mass disconnections. All 10,000 clients disconnect simultaneously. Each client attempts immediate reconnection. The server receives 10,000 requests in 100ms and crashes. Clients reconnect immediately again. The crash loop continues.

Slack experienced this at scale with 2 million concurrent clients. A server hiccup triggered disconnection. All 2 million clients attempted reconnection in under 10 seconds. CPU spiked to 100% and stayed there for 15 minutes.

Implementing exponential backoff

You will implement exponential backoff with random jitter to spread reconnections over time:

// Production-ready DexPaprika stream with exponential backoff
import { EventSource } from 'eventsource';

class DexPaprikaStream {
  constructor(chain, address) {
    this.chain = chain;
    this.address = address;
    this.url = `https://streaming.dexpaprika.com/stream?method=t_p&chain=${chain}&address=${address}`;
    this.reconnectDelay = 1000; // Start at 1 second
    this.stream = null;
  }

  connect() {
    this.stream = new EventSource(this.url);

    this.stream.onopen = () => {
      console.log(`Connected to ${this.chain} token ${this.address}`);
      this.reconnectDelay = 1000; // Reset on successful connection
    };

    this.stream.onmessage = (event) => {
      const data = JSON.parse(event.data);
      this.handlePriceUpdate(data);
    };

    this.stream.onerror = (error) => {
      console.error('Stream error, reconnecting...');
      this.stream.close();

      // Random jitter prevents synchronized reconnections
      const jitter = Math.random() * 1000;

      setTimeout(() => {
        this.connect();
        // Double delay, cap at 30 seconds
        this.reconnectDelay = Math.min(this.reconnectDelay * 2, 30000);
      }, this.reconnectDelay + jitter);
    };
  }

  handlePriceUpdate(data) {
    console.log(`Price: $${data.price} (${data.price_change_24h}%)`);
  }
}

// Usage: Stream WETH prices
const wethFeed = new DexPaprikaStream('ethereum', '0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2');
wethFeed.connect();

This implementation spreads 10,000 reconnections over 30 seconds instead of 100 milliseconds. Servers survive the reconnection wave. Users reconnect smoothly without infrastructure collapse.

The exponential backoff pattern (1s → 2s → 4s → 8s → 16s → 30s max) combined with random jitter (0-1000ms variance) prevents synchronized reconnection attempts. When the connection succeeds, the delay resets to 1 second for the next failure.

Checklist for reconnection safety

Before deploying, ensure:

• Exponential backoff implemented (1s → 2s → 4s → 8s → 16s → 30s max)
• Random jitter added (0-1000ms variance)
• Delay resets on successful connection
• Load test simulates mass disconnections (kill all servers, watch recovery)
• Monitoring tracks reconnection rate (alert when exceeding 100/second)

Section 2: Configuring nginx and heartbeats to prevent timeout disconnections

SSE streams work locally and on staging. Deploy to production behind nginx, and connections drop after exactly 60 seconds. Every single time. Users refresh. The connection drops again at 60 seconds.

Understanding nginx timeout behavior

Nginx's default proxy_read_timeout is 60 seconds. When the backend doesn't send data for 60 seconds, nginx kills the connection silently. Your server thinks the connection remains alive. Your client thinks it's connected. Nginx drops everything between them.

This affects low-volume tokens particularly hard. High-volume tokens like WETH and USDC update frequently (roughly every second on DexPaprika), keeping the connection alive. Low-volume tokens might go 2-3 minutes without price changes. Nginx kills those connections at exactly 60 seconds.

Configuring nginx for long-lived connections

You will configure nginx to handle long-lived SSE connections:

# nginx configuration for DexPaprika streaming
location /stream {
    proxy_pass https://streaming.dexpaprika.com;
    proxy_http_version 1.1;
    proxy_set_header Connection "";

    # Prevent timeout disconnections
    proxy_read_timeout 3600s;      # 1 hour timeout
    proxy_connect_timeout 10s;     # Initial connection timeout
    proxy_send_timeout 60s;        # Timeout for sending to backend

    # SSE-specific configuration
    proxy_buffering off;           # Disable buffering for real-time
    proxy_cache off;               # Disable caching
    chunked_transfer_encoding off; # Prevent chunked encoding issues

    # Critical: Disable Nginx buffering completely
    proxy_set_header X-Accel-Buffering no;
}

The proxy_read_timeout 3600s setting allows connections to remain idle for up to 1 hour without nginx terminating them. The proxy_buffering off directive ensures price updates flow through immediately without buffering delays.

Checklist for timeout prevention

Before deploying, verify:

• proxy_read_timeout set to 3600s or higher in nginx
• 30-second heartbeat messages implemented (if proxying DexPaprika)
• Test with connections receiving zero data for 5+ minutes
• Monitor connection duration (alert if max duration < 10 minutes)
• proxy_buffering off configured in nginx for real-time updates

Section 3: Managing memory with bounded buffers

Memory usage climbs slowly over days. Eventually, the process crashes with out-of-memory (OOM) errors. Restart it, and the problem returns. Memory starts at 500MB with 20,000 concurrent connections. After 24 hours: 2GB. After 48 hours: 4GB. At 72 hours, the process crashes.

Implementing bounded circular buffers

You will implement bounded buffers that cap memory usage:

// Production-ready bounded buffer implementation
class BoundedBuffer {
  constructor(maxSize = 100) {
    this.buffer = [];
    this.maxSize = maxSize;
  }

  push(message) {
    this.buffer.push(message);

    // Drop oldest message when buffer exceeds limit
    if (this.buffer.length > this.maxSize) {
      this.buffer.shift();
    }
  }

  getAll() {
    return this.buffer;
  }

  clear() {
    this.buffer = [];
  }

  size() {
    return this.buffer.length;
  }
}

When the buffer reaches 100 messages, the oldest message drops automatically. If a client falls that far behind, they're likely disconnected. Better to lose old messages than crash the server with OOM errors.

Checklist for memory safety

Before deploying, verify:

• Buffers capped at 100-1000 messages maximum
• Oldest messages drop when buffer fills
• Per-connection memory usage monitored
• Force-close connections idle > 30 seconds
• Memory leak detection enabled (heap snapshots in production)

Section 4: Optimizing for mobile networks

Desktop users experience 99.9% connection stability. Mobile users? 60%. Same code, different environment. Mobile networks cycle radio power states: active (full power, millisecond latency), idle (low power, 50-100ms latency), and dormant (radio off, 2-3 second wake-up).

Implementing mobile-optimized streaming

You will implement 30-second heartbeats with aggressive reconnection detection:

// Mobile-optimized DexPaprika stream
class MobileDexPaprikaStream {
  constructor(chain, address, options = {}) {
    this.chain = chain;
    this.address = address;
    this.url = `https://streaming.dexpaprika.com/stream?method=t_p&chain=${chain}&address=${address}`;
    this.lastMessageTime = Date.now();
    this.heartbeatInterval = options.heartbeatInterval || 30000; // 30 seconds
    this.timeoutThreshold = options.timeoutThreshold || 45000;   // 45 seconds
    this.stream = null;
    this.pingInterval = null;
  }

  connect() {
    this.stream = new EventSource(this.url);

    this.stream.onopen = () => {
      console.log('Connected to DexPaprika');
      this.startHeartbeatMonitor();
    };

    this.stream.onmessage = (event) => {
      this.lastMessageTime = Date.now();
      const data = JSON.parse(event.data);
      this.handlePriceUpdate(data);
    };

    this.stream.onerror = (error) => {
      console.error('Stream error');
      this.cleanup();
      this.reconnect();
    };
  }

  startHeartbeatMonitor() {
    this.pingInterval = setInterval(() => {
      const timeSinceLastMessage = Date.now() - this.lastMessageTime;

      // Connection considered dead if no message in 45 seconds
      if (timeSinceLastMessage > this.timeoutThreshold) {
        console.warn(`No data for ${timeSinceLastMessage}ms - reconnecting`);
        this.cleanup();
        this.reconnect();
      }
    }, this.heartbeatInterval);
  }

  handlePriceUpdate(data) {
    console.log(`Price: $${data.price} (${data.price_change_24h}%)`);
  }

  cleanup() {
    if (this.pingInterval) {
      clearInterval(this.pingInterval);
      this.pingInterval = null;
    }
    if (this.stream) {
      this.stream.close();
      this.stream = null;
    }
  }

  reconnect() {
    setTimeout(() => {
      this.connect();
    }, 2000);
  }

  close() {
    this.cleanup();
  }
}

The 30-second interval allows the mobile radio to power down between checks while keeping connections alive through most carrier networks. Setting the timeout threshold at 45 seconds (1.5× heartbeat interval) provides buffer for network latency.

Checklist for mobile optimization

Before deploying, verify:

• 30-second heartbeat interval (not faster - kills battery)
• Page visibility detection implemented with immediate reconnect
• Connection timeout set at 45 seconds (1.5× heartbeat interval)
• Testing completed on real 3G/4G networks (not WiFi only)
• Mobile vs desktop connection stability monitored separately

Section 5: Adding SSE fallback for corporate firewalls

Enterprise customers represent significant revenue. They also create nightmares for WebSocket connections. Corporate proxies like SophosXG, WatchGuard, and Fortinet block WebSocket protocol upgrades. The connection fails silently with no error message indicating why.

Understanding corporate proxy behavior

Corporate security teams often refuse to whitelist WebSocket connections. Policy restricts to HTTP/HTTPS only. The protocol upgrade handshake fails silently from the client's perspective. No error message. No indication why. The connection simply never establishes.

SSE works because it's standard HTTP. No special protocol. No upgrade handshake. Just a long-lived HTTP response. Proxies understand and allow it.

In enterprise customer bases:

• 30% of enterprise customers block WebSocket
• 5% block SSE (custom proxy configs)
• 1% block both (extreme lockdown)

If you're targeting enterprise customers with WebSocket-only implementation, you're losing 30% of potential revenue.

Checklist for corporate compatibility

Before deploying to enterprise customers, verify:

• SSE transport implemented (standard HTTP, works through proxies)
• Polling fallback available for extreme lockdown scenarios
• Connection type detection added to analytics
• Transport usage monitored in production
• Documentation provided for security teams (ports/protocols required)

Section 6: Optimizing cloud infrastructure costs

Cloud providers charge for streaming connections. The billing model can destroy budgets. AWS API Gateway supports WebSocket, but the pricing model creates million-dollar surprises.

Understanding API Gateway pricing trap

API Gateway charges per message and per connection minute. For real-time price feeds sending updates every second, you're paying for billions of tiny messages.

API Gateway charges approximately $142,000/month for 10,000 concurrent users with 1 update/second. DexPaprika's streaming endpoint has no per-message costs, no per-minute costs, and no rate limits.

Checklist for cost optimization

Before deploying, verify:

• API Gateway costs calculated with real traffic projections
• CloudFront + Lambda@Edge evaluated for SSE distribution
• Direct client connection to DexPaprika considered (zero infrastructure cost)
• Per-message costs monitored in billing dashboard
• Billing alerts configured (prevent surprise charges)

Testing your implementation

You will verify your production-ready implementation works correctly across all failure scenarios.

Test 1: Reconnection storm simulation

Verify exponential backoff spreads reconnections:

node test-reconnection-storm.js

Expected result: 100 clients reconnect over 30+ seconds, not simultaneously.

Test 2: Nginx timeout detection

Verify connections survive nginx's 60-second timeout:

node test-nginx-timeout.js

Expected result: Connection remains active for 5+ minutes without drops at 60 seconds.

Troubleshooting

Connection drops immediately

You will check:

• Network connectivity to streaming.dexpaprika.com
• Firewall rules allow HTTPS outbound connections
• EventSource polyfill installed (Node.js requires eventsource package)

No data received

You will verify:

• Token address is correct (use checksummed format: 0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2)
• Chain parameter matches token's blockchain (ethereum, bsc, polygon, etc.)
• Stream URL format: https://streaming.dexpaprika.com/stream?method=t_p&chain=CHAIN&address=ADDRESS

Connection dies at exactly 60 seconds

You will fix:

• Configure nginx proxy_read_timeout to 3600s (Section 2)
• Enable proxy_buffering off in nginx
• Implement 30-second heartbeat if proxying DexPaprika

Next steps

You have successfully implemented production-ready cryptocurrency price streaming with DexPaprika. Your implementation handles reconnection storms, proxy timeouts, memory leaks, mobile network instability, corporate firewalls, and cost optimization.

To extend this implementation:

Add database persistence: Store price history to PostgreSQL or TimescaleDB for historical analysis.

Implement price alerts: Notify users when prices cross thresholds.

Scale to hundreds of tokens: DexPaprika supports 1,300+ cryptocurrencies across 28+ blockchains with no rate limits.

Deploy to production: Use CloudFront + Lambda@Edge for global distribution at minimal cost (Section 6), or allow clients to connect directly to DexPaprika's streaming endpoint for zero infrastructure costs.

Related tutorials

• SSE vs WebSockets: choosing the right transport - Protocol comparison and decision framework for real-time crypto applications
• Why 1-second polling doesn't scale - Understanding when to migrate from polling to streaming architecture
• Server-Sent Events (SSE) explained for crypto apps - Deep dive into SSE implementation patterns