Jupiter Platform

The Problem

Tecnel MC had a trading platform for backtesting strategies on historical data. The problem was time: each backtest over three years of data took four and a half hours in Node.js. With thousands of strategies to validate and parameters to optimize, iteration was impossible. A researcher could test at most 5-6 strategies per day.

Code analysis showed the root cause: nested for loops reading millions of CSV rows one at a time. Each market tick was loaded from disk, parsed from string to number, then processed individually. Three layers of inefficiency stacked on top of each other.

The Solution

I rewrote the calculation engine in Python with three fundamental optimizations.

First: I replaced Python loops with NumPy vectorized operations. Instead of processing one tick at a time, NumPy works on the entire array in a single operation using the processor's SIMD instructions. For the most critical sections I used Numba, which JIT-compiles Python code to machine code at runtime: close to C speed without leaving Python.

Second: I moved from CSV to the Parquet columnar format. A CSV file loads everything into memory even if you only need 3 columns out of 50. Parquet reads only the requested columns, reducing I/O by 70%. Historical data compressed in Parquet occupies a fraction of the original CSV size.

Third: parallel processing. Each strategy runs on a separate CPU core. The system can test 10 strategies simultaneously on the same hardware.

The system integrates with IG.com for historical data and live trading. It calculates drawdown, Sharpe ratio, and margin call simulations in real time during each test, and uses Monte Carlo to validate the statistical robustness of strategies before running them with real money.

The Result

From four and a half hours to under 10 minutes per strategy. 95% time saved. A researcher can test dozens of strategies a day instead of 5 or 6. I/O dropped 70% thanks to Parquet.