CubeWatchTS Interfaces Documentation¶

Welcome¶

Welcome to the CubeWatchTS Interfaces documentation. This project powers the data pipeline that feeds CubeWatchTS, an energy transaction surveillance platform used to detect market abuse and ensure regulatory compliance across European energy markets.

Whether you're a compliance officer investigating suspicious trading patterns, a developer implementing a new data interface, or someone new to energy trading, this documentation will help you understand how market data flows from exchanges into our surveillance system.

Who Is This For?¶

🎓 New to Energy Trading?¶

Start here to understand fundamental concepts:

• Terms & Definitions - Baseload, peak, spot, futures, options, and more
• Order Types - Market orders, limit orders, stop orders, iceberg orders
• Order Book - How order books work and what market depth means
• Spreads - Calendar, location, and commodity spread trading strategies
• Power Auctions - How EPEX day-ahead and intraday auctions work
• OTC Trading - Over-the-counter trading explained
• Market Abuse Patterns - Wash trading, spoofing, and other manipulation tactics
• Energy Capacity - NTC, ATC, and capacity hoarding

👮 Compliance & Surveillance Users¶

Learn how to interpret CubeWatchTS data:

• Value Tables - Reference codes used in CubeWatchTS (TRAN_INS_TYPE, etc.)
• Common Patterns - Typical trading patterns and what to look for
• Market Abuse Patterns - Detailed explanations of manipulation techniques

🔧 Developers & Technical Users¶

Understand the technical architecture:

• Architecture Overview - System design and data flow
• Technology Stack - Frameworks, databases, and tools
• Common Patterns - Design patterns used across interfaces
• Contributing Guide - How to contribute to this documentation

What is CubeWatchTS?¶

CubeWatchTS is an energy transaction surveillance platform that monitors trading activity to detect potential market abuse and support regulatory compliance. It analyzes:

• Orders: Limit orders, market orders, amendments, cancellations
• Trades: Executed transactions with price, volume, and counterparty information
• Market Events: Auction results, capacity allocations, reference data updates
• Market Abuse Indicators: Unusual patterns that may indicate manipulation

The platform relies on clean, structured input data governed by the CubeWatchTS Data Structure specification.

How Data Flows Into CubeWatchTS¶

The Interfaces Team builds and maintains specialized data integration services that collect, validate, and transform market data from various exchanges and data providers. Here's how data flows through the system:

graph TD
    A[Exchange/Data Source] -->|1. Connect & Collect| B[Interface Service]
    B -->|2. Validate & Transform| C[Interface Processor]
    C -->|3. Write Staged Data| D[CLIENT_ORDERS<br/>CLIENT_TRADES<br/>CLIENT_REFERENCE]
    C -->|4. Signal Completion| E[CLIENT_HANDSHAKE]
    E -->|5. Trigger EOD| F[CubeWatchTS EOD Process]
    F -->|6. Copy & Enrich| G[ORDERS<br/>TRADES<br/>REFERENCE]
    G -->|7. Run Surveillance| H[Detection Rules]
    H -->|8. Create Alerts| I[Compliance Review]

    style B fill:#e1f5ff
    style F fill:#fff4e1
    style H fill:#ffe1e1

The Data Pipeline¶

Why Separate Tables?¶

The separation between CLIENT_ tables (interface-owned) and production tables* (CubeWatchTS-owned) provides:

• Clean boundaries: Interfaces can't corrupt production surveillance data
• Reprocessing capability: Re-run interfaces without affecting historical analysis
• Data quality gates: CubeWatchTS validates data before accepting it
• Independent deployment: Interface updates don't require CubeWatchTS downtime

Available Interfaces¶

EPEX - European Power Exchange¶

Real-time streaming interface for EPEX spot market

• Protocol: RabbitMQ message queue
• Data: Day-ahead and intraday orders, trades, auction results
• Markets: German, French, Belgian, Dutch, Austrian power markets
• Frequency: Real-time (seconds)

View EPEX Documentation →

NordPool - Nordic Power Exchange¶

Real-time streaming interface for Nordic electricity markets

• Protocol: WebSocket + REST API
• Data: Orders, trades, market depth, reference data
• Markets: Norwegian, Swedish, Finnish, Danish, Baltic power markets
• Architecture: Three-service pipeline using Parquet files
• Frequency: Real-time (sub-second)

View NordPool Documentation →

Refinitiv - Market Data Enrichment¶

Batch enrichment service for global commodity markets

• Protocol: REST API (DataScope Select)
• Data: Public market trades for CME, ICE, LME commodities
• Purpose: Enriches private OTC trades with public market context
• Architecture: Multi-tenant service supporting multiple CubeWatchTS instances
• Frequency: Daily batch jobs

View Refinitiv Documentation →

MAR - Market Abuse Regulation Reports¶

Regulatory compliance data from Trayport

• Protocol: SFTP file transfer
• Data: Transaction reports, order reports, organized market data
• Purpose: Regulatory reporting under EU Market Abuse Regulation
• Frequency: Daily scheduled downloads

View MAR Documentation →

Interface Comparison¶

Key Technologies¶

All interfaces share a common technology foundation:

• Platform: Windows Services (.NET 6+)
• Database: SQL Server with EF Core
• Concurrency: Channel-based multi-threading
• Logging: Serilog → File, EventLog, ElasticSearch/Kibana
• Configuration: Encrypted app settings (connection strings, API keys)
• Monitoring: Health checks, performance counters, error alerts

Learn more about the technology stack →

Common Design Patterns¶

Each interface follows proven patterns for reliability and maintainability:

Data Processing Pipeline¶

External Source → Download → Parse → Validate → Transform → Stage → Handshake

Error Handling Strategy¶

• Transient errors: Retry with exponential backoff
• Data errors: Log, skip record, continue processing
• Fatal errors: Stop processing, alert operations team
• Partial failures: Process what's valid, report what failed

Configuration Management¶

• Encrypted credentials stored in appsettings.json
• Environment-specific settings (Dev, Test, Prod)
• Feature flags for gradual rollouts
• Version-controlled configuration templates

Learn more about common patterns →

Getting Started¶

For Compliance Users¶

1. Start with Terms & Definitions to understand energy trading basics
2. Read about Market Abuse Patterns to recognize suspicious activity
3. Review Value Tables to interpret CubeWatchTS codes

For Developers¶

1. Review Architecture Overview to understand system design
2. Read interface-specific documentation (EPEX, NordPool, etc.)
3. Check the Contributing Guide to contribute to documentation

For Operations Teams¶

1. Understand the Data Flow to troubleshoot issues
2. Review interface-specific configuration guides
3. Set up monitoring and alerting using health check endpoints

Support & Contribution¶

Need Help?¶

• Technical Issues: Contact the Interfaces Team
• CubeWatchTS Questions: Refer to the Data Structure Specification
• Documentation Feedback: See the Contributing Guide

Contributing to Documentation¶

This documentation is maintained by the team and welcomes contributions! Whether you're:

• Fixing typos or broken links
• Adding examples and clarifications
• Writing new sections for missing topics
• Improving explanations for beginners

Check out our Contributing Guide - designed for contributors with no prior Git or Markdown experience.

Quick Reference¶

Essential Links¶

• Trading Glossary - Key terminology explained
• Architecture Diagrams - Visual system overview
• Value Tables - CubeWatchTS reference codes
• Contributing Guide - How to contribute

Navigation Tips¶

• Use the top navigation to browse by interface (EPEX, NordPool, Refinitiv, MAR)
• Use the search (press / or click search icon) to find specific topics
• Check the table of contents on the right side of each page for quick navigation
• All code examples include a copy button for easy use

Recent Updates¶