Data Preparation
To fully utilize the power of HftBacktest, it requires to input Tick-by-Tick full order book and trade feed data. Unfortunately, free Tick-by-Tick full order book and trade feed data for HFT is not available unlike daily bar data provided by platforms like Yahoo Finance. However, in the case of cryptocurrency, you can collect the full raw feed yourself.
Getting started from Binance Futures’ raw feed data
You can collect Binance Futures feed yourself using https://github.com/nkaz001/collect-binancefutures
[1]:
import gzip
with gzip.open('usdm/btcusdt_20230404.dat.gz', 'r') as f:
for i in range(20):
line = f.readline()
print(line)
b'1680652700423575 {"stream":"btcusdt@bookTicker","data":{"e":"bookTicker","u":2710246762461,"s":"BTCUSDT","b":"28145.10","B":"3.868","a":"28145.20","A":"6.887","T":1680652700430,"E":1680652700435}}\n'
b'1680652700441533 {"stream":"btcusdt@trade","data":{"e":"trade","E":1680652700455,"T":1680652700452,"s":"BTCUSDT","t":3535186032,"p":"28145.10","q":"0.002","X":"MARKET","m":true}}\n'
b'1680652700441685 {"stream":"btcusdt@trade","data":{"e":"trade","E":1680652700455,"T":1680652700452,"s":"BTCUSDT","t":3535186033,"p":"28145.10","q":"0.020","X":"MARKET","m":true}}\n'
b'1680652700441725 {"stream":"btcusdt@trade","data":{"e":"trade","E":1680652700455,"T":1680652700452,"s":"BTCUSDT","t":3535186034,"p":"28145.10","q":"0.020","X":"MARKET","m":true}}\n'
b'1680652700442528 {"stream":"btcusdt@trade","data":{"e":"trade","E":1680652700455,"T":1680652700452,"s":"BTCUSDT","t":3535186035,"p":"28145.10","q":"0.008","X":"MARKET","m":true}}\n'
b'1680652700442569 {"stream":"btcusdt@bookTicker","data":{"e":"bookTicker","u":2710246762974,"s":"BTCUSDT","b":"28145.10","B":"3.818","a":"28145.20","A":"6.887","T":1680652700452,"E":1680652700455}}\n'
b'1680652700454910 {"stream":"btcusdt@trade","data":{"e":"trade","E":1680652700468,"T":1680652700462,"s":"BTCUSDT","t":3535186036,"p":"28145.20","q":"0.002","X":"MARKET","m":false}}\n'
b'1680652700455070 {"stream":"btcusdt@trade","data":{"e":"trade","E":1680652700468,"T":1680652700462,"s":"BTCUSDT","t":3535186037,"p":"28145.20","q":"0.008","X":"MARKET","m":false}}\n'
b'1680652700455110 {"stream":"btcusdt@bookTicker","data":{"e":"bookTicker","u":2710246763198,"s":"BTCUSDT","b":"28145.10","B":"3.818","a":"28145.20","A":"6.907","T":1680652700462,"E":1680652700468}}\n'
b'1680652700458611 {"stream":"btcusdt@bookTicker","data":{"e":"bookTicker","u":2710246763205,"s":"BTCUSDT","b":"28145.10","B":"3.818","a":"28145.20","A":"6.927","T":1680652700462,"E":1680652700469}}\n'
b'1680652700461970 {"stream":"btcusdt@bookTicker","data":{"e":"bookTicker","u":2710246763256,"s":"BTCUSDT","b":"28145.10","B":"3.818","a":"28145.20","A":"6.917","T":1680652700462,"E":1680652700470}}\n'
b'1680652700462351 {"stream":"btcusdt@bookTicker","data":{"e":"bookTicker","u":2710246763281,"s":"BTCUSDT","b":"28145.10","B":"3.818","a":"28145.20","A":"6.925","T":1680652700463,"E":1680652700471}}\n'
b'1680652700487340 {"stream":"btcusdt@bookTicker","data":{"e":"bookTicker","u":2710246763977,"s":"BTCUSDT","b":"28145.10","B":"3.818","a":"28145.20","A":"6.954","T":1680652700498,"E":1680652700501}}\n'
b'1680652700566269 {"stream":"btcusdt@bookTicker","data":{"e":"bookTicker","u":2710246765398,"s":"BTCUSDT","b":"28145.10","B":"3.818","a":"28145.20","A":"6.833","T":1680652700575,"E":1680652700579}}\n'
b'1680652700573952 {"stream":"btcusdt@bookTicker","data":{"e":"bookTicker","u":2710246765530,"s":"BTCUSDT","b":"28145.10","B":"3.818","a":"28145.20","A":"6.953","T":1680652700583,"E":1680652700588}}\n'
b'1680652700574554 {"stream":"btcusdt@bookTicker","data":{"e":"bookTicker","u":2710246765551,"s":"BTCUSDT","b":"28145.10","B":"3.818","a":"28145.20","A":"6.961","T":1680652700585,"E":1680652700588}}\n'
b'1680652700599351 {"lastUpdateId": 2710246765483, "E": 1680652700590, "T": 1680652700580, "bids": [["28145.10", "3.818"], ["28145.00", "0.413"], ["28144.70", "0.002"], ["28144.60", "0.023"], ["28144.50", "0.003"], ["28144.40", "2.430"], ["28144.20", "0.022"], ["28144.10", "0.055"], ["28144.00", "0.160"], ["28143.80", "0.049"], ["28143.70", "0.002"], ["28143.60", "1.151"], ["28143.50", "0.001"], ["28143.40", "0.009"], ["28143.30", "0.008"], ["28143.20", "0.713"], ["28143.10", "0.006"], ["28143.00", "1.633"], ["28142.90", "0.005"], ["28142.80", "0.027"], ["28142.70", "0.180"], ["28142.60", "0.066"], ["28142.50", "0.895"], ["28142.40", "1.654"], ["28142.30", "0.004"], ["28142.20", "0.470"], ["28142.10", "0.054"], ["28142.00", "0.631"], ["28141.90", "1.312"], ["28141.80", "0.046"], ["28141.70", "0.044"], ["28141.60", "0.067"], ["28141.50", "0.110"], ["28141.40", "1.124"], ["28141.30", "4.160"], ["28141.20", "3.110"], ["28141.10", "0.012"], ["28141.00", "0.010"], ["28140.90", "0.201"], ["28140.80", "0.054"], ["28140.70", "0.011"], ["28140.60", "0.324"], ["28140.50", "0.509"], ["28140.40", "0.071"], ["28140.30", "0.207"], ["28140.20", "0.983"], ["28140.10", "0.039"], ["28140.00", "1.007"], ["28139.80", "0.046"], ["28139.70", "0.382"], ["28139.60", "1.200"], ["28139.50", "0.413"], ["28139.40", "0.222"], ["28139.30", "0.406"], ["28139.20", "1.656"], ["28139.10", "0.030"], ["28139.00", "2.014"], ["28138.90", "0.017"], ["28138.80", "0.004"], ["28138.70", "0.099"], ["28138.60", "1.580"], ["28138.50", "5.951"], ["28138.40", "0.537"], ["28138.30", "0.002"], ["28138.20", "0.603"], ["28138.10", "0.004"], ["28138.00", "0.010"], ["28137.90", "0.196"], ["28137.80", "0.178"], ["28137.70", "1.191"], ["28137.60", "2.539"], ["28137.50", "0.013"], ["28137.40", "0.903"], ["28137.30", "0.002"], ["28137.20", "1.023"], ["28137.10", "1.874"], ["28137.00", "4.064"], ["28136.90", "2.793"], ["28136.80", "0.008"], ["28136.70", "0.553"], ["28136.60", "0.017"], ["28136.50", "0.010"], ["28136.40", "0.005"], ["28136.30", "0.022"], ["28136.20", "0.282"], ["28136.10", "8.013"], ["28136.00", "0.278"], ["28135.90", "0.045"], ["28135.80", "2.203"], ["28135.70", "2.601"], ["28135.60", "1.460"], ["28135.50", "0.050"], ["28135.40", "0.345"], ["28135.30", "0.256"], ["28135.20", "0.005"], ["28135.10", "0.267"], ["28135.00", "0.160"], ["28134.90", "0.002"], ["28134.80", "1.929"], ["28134.70", "0.090"], ["28134.60", "1.176"], ["28134.50", "0.715"], ["28134.40", "3.618"], ["28134.30", "15.128"], ["28134.20", "2.789"], ["28134.10", "0.032"], ["28134.00", "0.334"], ["28133.90", "0.048"], ["28133.80", "1.652"], ["28133.60", "0.054"], ["28133.50", "0.002"], ["28133.40", "0.315"], ["28133.30", "1.448"], ["28133.20", "0.036"], ["28133.10", "0.004"], ["28133.00", "0.200"], ["28132.90", "0.760"], ["28132.80", "0.476"], ["28132.70", "0.063"], ["28132.60", "0.449"], ["28132.40", "0.007"], ["28132.30", "2.406"], ["28132.20", "10.661"], ["28132.10", "0.008"], ["28132.00", "0.512"], ["28131.90", "0.794"], ["28131.80", "0.054"], ["28131.70", "1.402"], ["28131.60", "1.305"], ["28131.50", "1.800"], ["28131.40", "0.201"], ["28131.20", "0.054"], ["28131.10", "2.420"], ["28131.00", "0.296"], ["28130.90", "0.008"], ["28130.80", "0.011"], ["28130.70", "0.102"], ["28130.60", "0.001"], ["28130.50", "0.078"], ["28130.40", "0.002"], ["28130.30", "0.004"], ["28130.20", "1.234"], ["28130.10", "0.060"], ["28130.00", "0.146"], ["28129.80", "0.013"], ["28129.70", "0.055"], ["28129.60", "0.819"], ["28129.50", "0.890"], ["28129.40", "2.296"], ["28129.30", "1.003"], ["28129.20", "3.000"], ["28129.10", "0.080"], ["28129.00", "1.844"], ["28128.90", "2.610"], ["28128.80", "10.346"], ["28128.70", "2.683"], ["28128.60", "1.148"], ["28128.30", "0.064"], ["28128.20", "0.001"], ["28128.10", "13.066"], ["28128.00", "0.316"], ["28127.90", "0.001"], ["28127.70", "3.002"], ["28127.60", "0.008"], ["28127.50", "0.468"], ["28127.40", "2.042"], ["28127.30", "0.061"], ["28127.20", "2.208"], ["28127.10", "0.086"], ["28127.00", "0.109"], ["28126.90", "0.386"], ["28126.80", "0.132"], ["28126.70", "7.302"], ["28126.60", "10.660"], ["28126.30", "0.045"], ["28126.10", "4.328"], ["28126.00", "1.750"], ["28125.90", "1.007"], ["28125.80", "0.004"], ["28125.70", "0.038"], ["28125.60", "0.192"], ["28125.50", "0.001"], ["28125.40", "0.706"], ["28125.30", "0.003"], ["28125.20", "0.097"], ["28125.10", "8.451"], ["28125.00", "25.287"], ["28124.90", "1.010"], ["28124.60", "0.004"], ["28124.50", "0.012"], ["28124.40", "0.413"], ["28124.20", "0.466"], ["28124.10", "1.475"], ["28124.00", "0.531"], ["28123.90", "0.020"], ["28123.80", "0.001"], ["28123.70", "2.028"], ["28123.50", "0.002"], ["28123.40", "0.036"], ["28123.30", "0.208"], ["28123.20", "0.839"], ["28123.10", "0.009"], ["28123.00", "2.830"], ["28122.90", "0.530"], ["28122.80", "0.087"], ["28122.70", "0.215"], ["28122.60", "0.251"], ["28122.50", "0.308"], ["28122.40", "0.062"], ["28122.30", "1.864"], ["28122.20", "0.027"], ["28122.10", "0.002"], ["28122.00", "0.103"], ["28121.90", "0.811"], ["28121.80", "8.052"], ["28121.70", "0.806"], ["28121.50", "0.064"], ["28121.40", "1.047"], ["28121.30", "0.002"], ["28121.20", "0.232"], ["28121.10", "0.002"], ["28121.00", "0.772"], ["28120.90", "0.135"], ["28120.80", "0.042"], ["28120.70", "0.212"], ["28120.60", "0.011"], ["28120.50", "3.413"], ["28120.10", "0.156"], ["28120.00", "3.143"], ["28119.80", "0.605"], ["28119.60", "0.919"], ["28119.40", "0.404"], ["28119.20", "1.333"], ["28119.10", "0.215"], ["28119.00", "3.003"], ["28118.90", "0.767"], ["28118.80", "3.326"], ["28118.70", "0.010"], ["28118.60", "0.153"], ["28118.30", "0.033"], ["28118.10", "0.007"], ["28117.90", "1.959"], ["28117.60", "2.722"], ["28117.50", "11.443"], ["28117.40", "0.007"], ["28117.30", "2.235"], ["28117.10", "0.744"], ["28117.00", "0.223"], ["28116.90", "3.895"], ["28116.80", "1.510"], ["28116.70", "0.003"], ["28116.60", "0.003"], ["28116.50", "4.452"], ["28116.40", "6.051"], ["28116.30", "2.006"], ["28116.20", "3.300"], ["28116.10", "2.584"], ["28116.00", "0.067"], ["28115.80", "0.030"], ["28115.70", "0.001"], ["28115.60", "0.062"], ["28115.50", "0.727"], ["28115.40", "1.641"], ["28115.30", "3.191"], ["28115.20", "0.065"], ["28115.10", "0.011"], ["28115.00", "3.178"], ["28114.90", "0.001"], ["28114.80", "0.107"], ["28114.70", "0.133"], ["28114.50", "4.029"], ["28114.40", "3.876"], ["28114.20", "0.827"], ["28114.10", "0.077"], ["28114.00", "0.796"], ["28113.90", "0.409"], ["28113.80", "0.266"], ["28113.70", "3.197"], ["28113.60", "3.032"], ["28113.50", "0.068"], ["28113.40", "1.207"], ["28113.30", "3.000"], ["28113.20", "0.213"], ["28113.10", "3.016"], ["28113.00", "0.472"], ["28112.70", "6.047"], ["28112.60", "2.034"], ["28112.50", "1.369"], ["28112.40", "1.452"], ["28112.30", "3.105"], ["28112.20", "0.212"], ["28112.00", "5.929"], ["28111.90", "0.951"], ["28111.80", "3.270"], ["28111.60", "13.388"], ["28111.50", "0.055"], ["28111.40", "0.005"], ["28111.30", "2.748"], ["28111.20", "0.495"], ["28111.10", "6.307"], ["28111.00", "1.337"], ["28110.90", "11.898"], ["28110.70", "3.402"], ["28110.60", "0.213"], ["28110.40", "0.019"], ["28110.20", "0.388"], ["28110.10", "0.701"], ["28110.00", "6.217"], ["28109.90", "0.002"], ["28109.80", "0.102"], ["28109.60", "1.655"], ["28109.50", "0.010"], ["28109.40", "0.032"], ["28109.30", "0.001"], ["28109.10", "0.243"], ["28109.00", "0.037"], ["28108.90", "1.974"], ["28108.80", "1.373"], ["28108.70", "1.302"], ["28108.60", "4.321"], ["28108.50", "0.102"], ["28108.40", "2.026"], ["28108.30", "0.030"], ["28108.20", "5.421"], ["28108.10", "0.223"], ["28108.00", "0.091"], ["28107.90", "0.075"], ["28107.80", "0.015"], ["28107.60", "0.200"], ["28107.50", "0.002"], ["28107.40", "3.469"], ["28107.30", "0.284"], ["28107.20", "1.545"], ["28107.10", "6.866"], ["28107.00", "0.213"], ["28106.90", "0.524"], ["28106.80", "3.141"], ["28106.70", "0.007"], ["28106.60", "3.101"], ["28106.50", "0.369"], ["28106.40", "0.258"], ["28106.30", "0.785"], ["28106.20", "0.111"], ["28106.10", "0.553"], ["28106.00", "1.088"], ["28105.60", "0.953"], ["28105.30", "0.533"], ["28105.20", "2.129"], ["28105.10", "0.003"], ["28105.00", "0.978"], ["28104.80", "0.030"], ["28104.70", "0.001"], ["28104.60", "0.209"], ["28104.50", "0.002"], ["28104.40", "0.317"], ["28104.30", "0.917"], ["28104.20", "0.008"], ["28104.10", "0.048"], ["28104.00", "0.939"], ["28103.90", "0.232"], ["28103.80", "0.548"], ["28103.70", "0.270"], ["28103.60", "0.010"], ["28103.40", "0.012"], ["28103.30", "0.006"], ["28103.20", "0.213"], ["28103.10", "0.351"], ["28103.00", "0.251"], ["28102.90", "0.884"], ["28102.80", "6.814"], ["28102.70", "1.471"], ["28102.60", "30.273"], ["28102.50", "1.525"], ["28102.40", "0.049"], ["28102.30", "7.455"], ["28102.20", "0.072"], ["28102.10", "1.428"], ["28102.00", "0.204"], ["28101.90", "6.973"], ["28101.80", "0.001"], ["28101.70", "18.887"], ["28101.60", "0.247"], ["28101.50", "2.869"], ["28101.40", "0.268"], ["28101.30", "30.271"], ["28101.20", "0.062"], ["28101.10", "0.660"], ["28101.00", "0.113"], ["28100.90", "0.205"], ["28100.70", "0.124"], ["28100.60", "0.181"], ["28100.50", "1.703"], ["28100.40", "0.275"], ["28100.30", "0.001"], ["28100.20", "0.212"], ["28100.10", "1.573"], ["28100.00", "2.727"], ["28099.90", "6.474"], ["28099.70", "0.099"], ["28099.60", "0.056"], ["28099.50", "0.360"], ["28099.40", "0.996"], ["28099.20", "0.406"], ["28099.10", "0.478"], ["28099.00", "3.469"], ["28098.90", "1.864"], ["28098.80", "1.804"], ["28098.70", "0.092"], ["28098.60", "0.002"], ["28098.50", "0.119"], ["28098.40", "0.093"], ["28098.30", "0.002"], ["28098.20", "0.729"], ["28098.10", "0.070"], ["28098.00", "1.096"], ["28097.90", "0.054"], ["28097.80", "2.228"], ["28097.70", "0.448"], ["28097.60", "0.070"], ["28097.50", "1.361"], ["28097.40", "1.100"], ["28097.30", "0.446"], ["28097.20", "1.831"], ["28097.10", "4.399"], ["28097.00", "0.074"], ["28096.90", "0.327"], ["28096.80", "0.226"], ["28096.70", "0.244"], ["28096.60", "0.312"], ["28096.50", "1.530"], ["28096.40", "0.166"], ["28096.30", "0.208"], ["28096.20", "5.612"], ["28096.10", "0.587"], ["28096.00", "0.139"], ["28095.90", "2.851"], ["28095.80", "0.129"], ["28095.70", "0.101"], ["28095.60", "0.088"], ["28095.40", "3.064"], ["28095.30", "1.375"], ["28095.20", "0.187"], ["28095.10", "0.225"], ["28095.00", "0.811"], ["28094.90", "0.801"], ["28094.80", "0.174"], ["28094.70", "0.295"], ["28094.60", "0.421"], ["28094.50", "0.060"], ["28094.40", "0.660"], ["28094.30", "0.371"], ["28094.20", "0.169"], ["28094.10", "2.136"], ["28094.00", "0.183"], ["28093.90", "0.130"], ["28093.80", "5.429"], ["28093.70", "13.389"], ["28093.60", "0.093"], ["28093.40", "0.159"], ["28093.30", "0.005"], ["28093.20", "0.164"], ["28093.10", "0.038"], ["28093.00", "3.091"], ["28092.80", "0.003"], ["28092.70", "3.147"], ["28092.60", "0.117"], ["28092.50", "1.473"], ["28092.40", "8.439"], ["28092.30", "2.853"], ["28092.20", "1.371"], ["28092.10", "10.690"], ["28092.00", "0.058"], ["28091.90", "0.024"], ["28091.70", "4.074"], ["28091.60", "0.004"], ["28091.50", "0.231"], ["28091.30", "0.618"], ["28091.20", "1.096"], ["28091.00", "0.542"], ["28090.90", "0.123"], ["28090.80", "0.108"], ["28090.70", "0.350"], ["28090.60", "0.138"], ["28090.50", "1.754"], ["28090.40", "2.112"], ["28090.30", "0.181"], ["28090.20", "5.214"], ["28090.10", "0.422"], ["28090.00", "0.254"], ["28089.90", "2.105"], ["28089.80", "19.973"], ["28089.60", "5.150"], ["28089.50", "0.256"], ["28089.40", "1.079"], ["28089.30", "1.593"], ["28089.20", "5.242"], ["28089.10", "0.312"], ["28089.00", "1.047"], ["28088.90", "4.891"], ["28088.80", "0.166"], ["28088.70", "0.001"], ["28088.60", "0.144"], ["28088.50", "0.140"], ["28088.40", "2.132"], ["28088.20", "0.068"], ["28088.10", "6.753"], ["28088.00", "1.447"], ["28087.90", "0.412"], ["28087.80", "4.234"], ["28087.70", "0.311"], ["28087.60", "1.810"], ["28087.50", "0.031"], ["28087.40", "1.345"], ["28087.30", "0.267"], ["28087.20", "0.324"], ["28087.10", "0.003"], ["28087.00", "0.650"], ["28086.90", "5.378"], ["28086.80", "0.656"], ["28086.70", "2.352"], ["28086.60", "0.131"], ["28086.50", "0.265"], ["28086.40", "0.174"], ["28086.30", "0.300"], ["28086.20", "0.207"], ["28086.10", "0.401"], ["28086.00", "1.076"], ["28085.90", "0.475"], ["28085.80", "3.487"], ["28085.70", "5.414"], ["28085.60", "0.086"], ["28085.50", "0.144"], ["28085.40", "3.595"], ["28085.30", "0.289"], ["28085.20", "2.669"], ["28085.10", "0.007"], ["28085.00", "14.256"], ["28084.90", "0.343"], ["28084.80", "0.421"], ["28084.70", "0.481"], ["28084.60", "0.390"], ["28084.50", "0.187"], ["28084.40", "0.224"], ["28084.20", "0.085"], ["28084.10", "0.059"], ["28084.00", "7.846"], ["28083.90", "3.144"], ["28083.80", "3.763"], ["28083.70", "0.006"], ["28083.60", "0.082"], ["28083.50", "0.296"], ["28083.40", "0.124"], ["28083.30", "9.287"], ["28083.10", "29.720"], ["28083.00", "125.094"], ["28082.80", "0.091"], ["28082.70", "0.536"], ["28082.60", "0.014"], ["28082.50", "0.091"], ["28082.40", "0.086"], ["28082.30", "3.703"], ["28082.20", "0.032"], ["28082.10", "3.001"], ["28082.00", "0.024"], ["28081.90", "0.307"], ["28081.80", "0.114"], ["28081.70", "0.037"], ["28081.60", "0.141"], ["28081.50", "0.138"], ["28081.40", "3.000"], ["28081.30", "0.210"], ["28081.20", "0.211"], ["28081.10", "0.199"], ["28081.00", "0.146"], ["28080.90", "0.632"], ["28080.70", "0.362"], ["28080.60", "0.164"], ["28080.50", "0.138"], ["28080.40", "1.070"], ["28080.30", "0.212"], ["28080.20", "3.000"], ["28080.00", "4.756"], ["28079.90", "0.002"], ["28079.80", "1.560"], ["28079.70", "6.080"], ["28079.60", "0.416"], ["28079.50", "0.023"], ["28079.40", "0.173"], ["28079.20", "0.017"], ["28079.10", "2.076"], ["28079.00", "1.575"], ["28078.90", "0.004"], ["28078.80", "0.084"], ["28078.70", "0.019"], ["28078.60", "0.108"], ["28078.50", "0.071"], ["28078.40", "6.548"], ["28078.30", "0.981"], ["28078.20", "3.954"], ["28078.10", "0.202"], ["28078.00", "0.920"], ["28077.90", "0.201"], ["28077.80", "0.107"], ["28077.70", "0.466"], ["28077.60", "16.112"], ["28077.50", "0.099"], ["28077.40", "0.080"], ["28077.30", "0.136"], ["28077.20", "0.081"], ["28077.10", "0.297"], ["28077.00", "7.175"], ["28076.90", "0.347"], ["28076.80", "1.552"], ["28076.70", "0.284"], ["28076.60", "1.143"], ["28076.50", "3.103"], ["28076.30", "2.133"], ["28076.10", "0.501"], ["28076.00", "0.229"], ["28075.70", "0.117"], ["28075.60", "0.091"], ["28075.50", "0.276"], ["28075.40", "0.116"], ["28075.30", "0.119"], ["28075.20", "0.910"], ["28075.10", "0.141"], ["28075.00", "0.424"], ["28074.90", "0.176"], ["28074.80", "3.707"], ["28074.70", "0.204"], ["28074.60", "0.085"], ["28074.50", "0.075"], ["28074.40", "0.006"], ["28074.30", "0.088"], ["28074.20", "0.625"], ["28074.10", "0.421"], ["28074.00", "0.592"], ["28073.90", "0.410"], ["28073.80", "0.018"], ["28073.70", "0.176"], ["28073.60", "2.122"], ["28073.50", "0.351"], ["28073.40", "0.001"], ["28073.30", "0.001"], ["28073.10", "0.069"], ["28073.00", "0.858"], ["28072.90", "0.345"], ["28072.80", "0.316"], ["28072.70", "0.054"], ["28072.60", "0.001"], ["28072.50", "0.215"], ["28072.40", "0.116"], ["28072.20", "1.316"], ["28072.10", "1.531"], ["28072.00", "4.635"], ["28071.90", "5.776"], ["28071.80", "0.184"], ["28071.70", "0.001"], ["28071.60", "0.016"], ["28071.40", "0.217"], ["28071.30", "1.938"], ["28071.20", "0.089"], ["28071.10", "0.357"], ["28071.00", "14.024"], ["28070.90", "0.021"], ["28070.80", "0.423"], ["28070.70", "0.146"], ["28070.60", "10.724"], ["28070.50", "0.093"], ["28070.40", "2.891"], ["28070.30", "0.343"], ["28070.20", "0.106"], ["28070.10", "0.051"], ["28070.00", "45.977"], ["28069.90", "0.270"], ["28069.80", "0.116"], ["28069.70", "0.057"], ["28069.60", "0.080"], ["28069.40", "1.521"], ["28069.30", "0.085"], ["28069.20", "1.365"], ["28069.10", "0.058"], ["28069.00", "0.112"], ["28068.90", "0.195"], ["28068.80", "2.878"], ["28068.70", "0.271"], ["28068.60", "0.200"], ["28068.50", "0.352"], ["28068.40", "0.077"], ["28068.30", "0.368"], ["28068.20", "0.182"], ["28068.10", "0.315"], ["28068.00", "0.116"], ["28067.90", "0.106"], ["28067.80", "0.249"], ["28067.70", "0.003"], ["28067.60", "0.003"], ["28067.50", "8.164"], ["28067.40", "0.177"], ["28067.30", "0.116"], ["28067.20", "0.393"], ["28067.10", "0.240"], ["28067.00", "5.245"], ["28066.90", "0.129"], ["28066.80", "0.373"], ["28066.70", "0.166"], ["28066.60", "0.021"], ["28066.50", "1.518"], ["28066.40", "0.041"], ["28066.30", "0.167"], ["28066.20", "0.007"], ["28066.10", "0.125"], ["28066.00", "0.337"], ["28065.80", "0.135"], ["28065.70", "0.085"], ["28065.60", "0.255"], ["28065.50", "0.059"], ["28065.40", "0.415"], ["28065.30", "2.592"], ["28065.20", "0.086"], ["28065.10", "0.407"], ["28065.00", "0.615"], ["28064.90", "0.308"], ["28064.80", "0.457"], ["28064.70", "4.348"], ["28064.60", "0.016"], ["28064.50", "0.004"], ["28064.40", "1.835"], ["28064.20", "0.089"], ["28064.10", "0.320"], ["28064.00", "0.145"], ["28063.90", "0.002"], ["28063.70", "0.594"], ["28063.60", "0.621"], ["28063.50", "0.001"], ["28063.40", "0.082"], ["28063.30", "0.256"], ["28063.20", "0.166"], ["28063.10", "3.124"], ["28063.00", "2.869"], ["28062.90", "0.201"], ["28062.80", "0.424"], ["28062.70", "0.007"], ["28062.60", "0.133"], ["28062.50", "0.080"], ["28062.40", "0.199"], ["28062.20", "0.292"], ["28062.10", "0.056"], ["28062.00", "0.163"], ["28061.90", "0.180"], ["28061.80", "0.053"], ["28061.70", "0.082"], ["28061.60", "0.001"], ["28061.50", "0.537"], ["28061.40", "1.035"], ["28061.30", "0.239"], ["28061.20", "0.085"], ["28061.10", "0.083"], ["28061.00", "0.407"], ["28060.90", "0.087"], ["28060.80", "0.473"], ["28060.70", "0.822"], ["28060.60", "0.205"], ["28060.50", "2.651"], ["28060.40", "0.281"], ["28060.30", "0.348"], ["28060.20", "0.127"], ["28060.10", "0.129"], ["28060.00", "12.187"], ["28059.90", "0.337"], ["28059.80", "1.211"], ["28059.70", "0.350"], ["28059.60", "1.515"], ["28059.50", "3.055"], ["28059.40", "0.217"], ["28059.30", "0.441"], ["28059.20", "1.583"], ["28059.10", "0.011"], ["28059.00", "0.254"], ["28058.90", "0.152"], ["28058.80", "0.828"], ["28058.70", "0.074"], ["28058.60", "0.033"], ["28058.50", "0.121"], ["28058.40", "0.550"], ["28058.30", "1.555"], ["28058.20", "0.155"], ["28058.10", "0.103"], ["28058.00", "6.335"], ["28057.90", "0.042"], ["28057.80", "0.873"], ["28057.70", "0.004"], ["28057.60", "0.001"], ["28057.50", "0.301"], ["28057.30", "0.551"], ["28057.20", "1.538"], ["28057.10", "0.195"], ["28057.00", "0.560"], ["28056.90", "0.101"], ["28056.80", "0.003"], ["28056.70", "0.002"], ["28056.60", "0.011"], ["28056.50", "0.085"], ["28056.40", "0.299"], ["28056.30", "0.006"], ["28056.20", "0.030"], ["28056.10", "0.553"], ["28056.00", "1.433"], ["28055.90", "0.039"], ["28055.80", "0.490"], ["28055.70", "0.085"], ["28055.60", "0.454"], ["28055.50", "0.263"], ["28055.40", "4.539"], ["28055.30", "0.085"], ["28055.20", "0.255"], ["28055.10", "0.175"], ["28055.00", "3.113"], ["28054.90", "0.173"], ["28054.80", "0.001"], ["28054.70", "0.514"], ["28054.60", "0.222"], ["28054.50", "0.307"], ["28054.40", "0.514"], ["28054.30", "0.099"], ["28054.20", "0.060"], ["28054.10", "0.121"], ["28054.00", "0.587"], ["28053.90", "9.321"], ["28053.80", "0.185"], ["28053.70", "0.001"], ["28053.60", "0.242"], ["28053.50", "0.022"], ["28053.40", "0.035"], ["28053.30", "0.007"], ["28053.10", "0.105"], ["28053.00", "0.907"], ["28052.90", "0.216"], ["28052.80", "7.400"], ["28052.60", "0.059"], ["28052.50", "0.191"], ["28052.40", "0.365"], ["28052.30", "0.131"], ["28052.20", "3.907"], ["28052.10", "0.120"], ["28052.00", "0.805"], ["28051.90", "172.732"], ["28051.80", "0.016"], ["28051.70", "0.100"], ["28051.60", "0.192"], ["28051.50", "1.835"], ["28051.40", "0.099"], ["28051.20", "0.097"], ["28051.10", "0.153"], ["28051.00", "9.564"], ["28050.90", "0.177"], ["28050.80", "0.030"], ["28050.70", "3.525"], ["28050.60", "10.026"], ["28050.50", "0.212"], ["28050.40", "0.098"], ["28050.30", "0.012"], ["28050.10", "1.148"], ["28050.00", "38.892"], ["28049.90", "0.115"], ["28049.80", "10.540"], ["28049.70", "5.713"], ["28049.60", "0.288"], ["28049.50", "0.001"], ["28049.40", "0.001"], ["28049.30", "0.103"], ["28049.20", "1.513"], ["28049.10", "0.215"], ["28049.00", "0.331"], ["28048.80", "0.034"], ["28048.70", "0.531"], ["28048.60", "0.004"], ["28048.50", "1.171"], ["28048.40", "0.175"], ["28048.30", "0.206"], ["28048.10", "8.352"], ["28048.00", "2.224"], ["28047.90", "0.120"], ["28047.80", "0.549"], ["28047.70", "0.214"], ["28047.60", "0.106"], ["28047.50", "0.178"], ["28047.40", "0.240"], ["28047.30", "0.012"], ["28047.20", "0.038"], ["28047.10", "0.091"], ["28047.00", "0.611"], ["28046.90", "0.242"], ["28046.80", "0.073"], ["28046.70", "0.101"], ["28046.60", "0.329"], ["28046.30", "0.014"], ["28046.20", "0.002"], ["28046.10", "0.382"], ["28046.00", "0.535"], ["28045.90", "0.135"], ["28045.60", "0.011"], ["28045.50", "0.815"], ["28045.40", "0.119"], ["28045.30", "0.230"], ["28045.20", "0.003"], ["28045.10", "0.155"], ["28045.00", "1.935"], ["28044.90", "0.123"], ["28044.80", "12.323"], ["28044.70", "0.306"], ["28044.60", "0.141"], ["28044.50", "0.005"], ["28044.40", "0.007"], ["28044.30", "0.198"], ["28044.20", "0.101"], ["28044.10", "0.071"], ["28044.00", "3.365"], ["28043.90", "0.001"], ["28043.80", "1.073"], ["28043.70", "0.585"], ["28043.60", "0.001"], ["28043.50", "0.002"], ["28043.30", "0.034"], ["28043.20", "0.088"], ["28043.10", "0.006"], ["28043.00", "0.257"], ["28042.90", "0.324"], ["28042.70", "0.123"], ["28042.60", "0.042"], ["28042.50", "0.238"], ["28042.40", "0.002"], ["28042.30", "10.905"], ["28042.20", "0.042"], ["28042.10", "0.187"], ["28042.00", "0.324"], ["28041.90", "0.083"], ["28041.80", "0.056"], ["28041.70", "0.006"], ["28041.60", "0.080"], ["28041.50", "0.018"], ["28041.40", "0.002"], ["28041.30", "0.003"], ["28041.20", "0.427"], ["28041.10", "0.534"], ["28041.00", "0.163"], ["28040.90", "0.142"], ["28040.80", "0.007"], ["28040.60", "0.119"], ["28040.50", "0.003"], ["28040.40", "0.691"], ["28040.30", "0.135"], ["28040.20", "0.158"], ["28040.10", "0.047"], ["28040.00", "57.404"], ["28039.90", "0.081"], ["28039.80", "0.017"], ["28039.70", "0.217"], ["28039.60", "0.256"], ["28039.50", "0.718"], ["28039.40", "0.243"], ["28039.30", "0.027"], ["28039.20", "1.464"], ["28039.10", "0.364"], ["28039.00", "0.771"], ["28038.90", "0.082"], ["28038.80", "1.601"], ["28038.70", "0.001"], ["28038.60", "0.322"], ["28038.40", "0.209"], ["28038.30", "0.041"], ["28038.20", "0.004"], ["28038.00", "4.476"], ["28037.90", "3.860"], ["28037.80", "0.068"], ["28037.70", "0.542"], ["28037.60", "0.249"], ["28037.50", "0.301"], ["28037.40", "4.364"], ["28037.30", "0.055"], ["28037.20", "0.103"], ["28037.10", "3.944"], ["28037.00", "0.014"], ["28036.90", "0.320"], ["28036.80", "0.002"], ["28036.70", "0.070"], ["28036.60", "0.088"], ["28036.50", "0.015"], ["28036.40", "0.343"], ["28036.30", "0.100"], ["28036.20", "0.228"], ["28036.10", "0.002"], ["28036.00", "0.662"], ["28035.90", "0.260"], ["28035.80", "0.210"], ["28035.70", "2.550"], ["28035.60", "0.129"], ["28035.50", "0.354"], ["28035.40", "0.003"], ["28035.30", "0.103"]], "asks": [["28145.20", "6.833"], ["28145.30", "0.009"], ["28145.40", "0.008"], ["28145.60", "0.002"], ["28145.70", "0.003"], ["28145.90", "0.009"], ["28146.00", "0.002"], ["28146.10", "0.009"], ["28146.20", "0.045"], ["28146.40", "0.139"], ["28146.50", "0.047"], ["28146.60", "0.027"], ["28146.70", "0.023"], ["28146.80", "0.380"], ["28146.90", "0.254"], ["28147.00", "0.027"], ["28147.10", "0.084"], ["28147.20", "0.002"], ["28147.30", "0.177"], ["28147.50", "0.001"], ["28147.70", "0.177"], ["28147.80", "0.105"], ["28147.90", "0.138"], ["28148.00", "0.419"], ["28148.10", "0.426"], ["28148.20", "0.008"], ["28148.30", "0.155"], ["28148.40", "0.088"], ["28148.50", "0.260"], ["28148.60", "0.006"], ["28148.70", "0.871"], ["28148.80", "0.595"], ["28148.90", "2.000"], ["28149.00", "1.147"], ["28149.10", "0.370"], ["28149.20", "1.754"], ["28149.40", "0.628"], ["28149.50", "5.833"], ["28149.60", "1.963"], ["28149.70", "0.001"], ["28149.80", "1.704"], ["28149.90", "0.480"], ["28150.00", "2.712"], ["28150.10", "0.345"], ["28150.20", "0.002"], ["28150.40", "0.202"], ["28150.50", "0.072"], ["28150.60", "3.905"], ["28150.70", "0.212"], ["28150.90", "0.374"], ["28151.00", "1.357"], ["28151.10", "0.074"], ["28151.20", "0.176"], ["28151.30", "0.001"], ["28151.40", "0.160"], ["28151.60", "0.135"], ["28152.00", "0.568"], ["28152.10", "1.432"], ["28152.20", "0.007"], ["28152.30", "0.010"], ["28152.40", "0.110"], ["28152.60", "0.012"], ["28152.70", "0.110"], ["28152.80", "0.070"], ["28152.90", "1.146"], ["28153.00", "5.070"], ["28153.10", "4.020"], ["28153.20", "12.494"], ["28153.30", "2.211"], ["28153.50", "0.321"], ["28153.60", "1.553"], ["28153.70", "0.248"], ["28153.80", "0.008"], ["28153.90", "0.080"], ["28154.00", "0.339"], ["28154.10", "0.415"], ["28154.20", "0.001"], ["28154.30", "0.100"], ["28154.40", "0.385"], ["28154.50", "0.001"], ["28154.60", "0.177"], ["28154.70", "0.001"], ["28154.80", "0.445"], ["28154.90", "0.717"], ["28155.00", "1.370"], ["28155.10", "1.489"], ["28155.30", "0.035"], ["28155.40", "0.129"], ["28155.60", "0.391"], ["28155.70", "1.065"], ["28155.80", "0.501"], ["28155.90", "1.464"], ["28156.00", "0.800"], ["28156.10", "0.393"], ["28156.20", "0.046"], ["28156.30", "1.067"], ["28156.40", "1.634"], ["28156.50", "0.001"], ["28156.60", "1.411"], ["28156.70", "0.089"], ["28156.80", "1.159"], ["28157.00", "2.333"], ["28157.10", "0.357"], ["28157.20", "0.575"], ["28157.30", "0.253"], ["28157.40", "0.977"], ["28157.50", "0.062"], ["28157.60", "1.290"], ["28157.70", "0.034"], ["28157.80", "0.279"], ["28157.90", "0.067"], ["28158.00", "0.782"], ["28158.10", "0.224"], ["28158.20", "0.533"], ["28158.30", "0.801"], ["28158.40", "0.872"], ["28158.50", "1.372"], ["28158.60", "1.956"], ["28158.70", "0.451"], ["28158.80", "0.356"], ["28158.90", "0.267"], ["28159.00", "2.356"], ["28159.10", "0.747"], ["28159.20", "1.102"], ["28159.30", "5.502"], ["28159.40", "1.647"], ["28159.50", "0.662"], ["28159.60", "0.726"], ["28159.70", "2.441"], ["28159.80", "1.814"], ["28159.90", "1.407"], ["28160.00", "0.995"], ["28160.10", "0.006"], ["28160.20", "2.176"], ["28160.30", "0.432"], ["28160.40", "0.489"], ["28160.60", "0.001"], ["28160.70", "2.037"], ["28160.80", "0.181"], ["28160.90", "1.832"], ["28161.00", "0.004"], ["28161.10", "0.595"], ["28161.20", "0.200"], ["28161.30", "1.191"], ["28161.40", "0.004"], ["28161.50", "0.003"], ["28161.60", "0.325"], ["28161.80", "0.195"], ["28161.90", "0.054"], ["28162.00", "0.871"], ["28162.20", "0.751"], ["28162.30", "0.057"], ["28162.40", "0.302"], ["28162.50", "0.004"], ["28162.70", "0.004"], ["28162.80", "0.099"], ["28162.90", "1.662"], ["28163.10", "0.502"], ["28163.30", "0.054"], ["28163.40", "0.911"], ["28163.50", "7.023"], ["28163.60", "0.095"], ["28163.70", "0.003"], ["28163.80", "0.648"], ["28163.90", "0.083"], ["28164.00", "0.492"], ["28164.10", "0.595"], ["28164.20", "5.000"], ["28164.40", "3.216"], ["28164.50", "0.584"], ["28164.60", "0.002"], ["28164.70", "2.085"], ["28164.80", "0.563"], ["28164.90", "5.215"], ["28165.00", "4.531"], ["28165.10", "0.750"], ["28165.20", "0.980"], ["28165.30", "0.356"], ["28165.40", "0.301"], ["28165.50", "0.303"], ["28165.60", "0.002"], ["28165.70", "0.007"], ["28165.80", "0.383"], ["28165.90", "1.105"], ["28166.00", "0.161"], ["28166.20", "4.622"], ["28166.30", "0.202"], ["28166.40", "0.440"], ["28166.50", "1.251"], ["28166.60", "0.034"], ["28166.70", "0.232"], ["28166.90", "0.370"], ["28167.00", "0.107"], ["28167.10", "0.399"], ["28167.20", "0.059"], ["28167.30", "0.040"], ["28167.40", "0.003"], ["28167.60", "0.325"], ["28167.70", "0.024"], ["28167.80", "0.777"], ["28167.90", "0.022"], ["28168.00", "0.633"], ["28168.10", "2.091"], ["28168.20", "0.396"], ["28168.30", "3.324"], ["28168.40", "0.362"], ["28168.50", "0.107"], ["28168.60", "2.304"], ["28168.80", "10.144"], ["28169.00", "0.055"], ["28169.10", "0.063"], ["28169.20", "0.070"], ["28169.30", "0.003"], ["28169.40", "3.075"], ["28169.50", "12.127"], ["28169.60", "3.905"], ["28169.70", "0.108"], ["28169.80", "0.072"], ["28169.90", "1.229"], ["28170.00", "2.082"], ["28170.10", "0.028"], ["28170.20", "0.001"], ["28170.30", "0.129"], ["28170.50", "0.011"], ["28170.60", "0.216"], ["28170.80", "1.498"], ["28170.90", "5.049"], ["28171.10", "0.011"], ["28171.20", "0.352"], ["28171.50", "1.278"], ["28171.70", "0.471"], ["28171.80", "1.966"], ["28171.90", "0.011"], ["28172.00", "0.010"], ["28172.10", "1.060"], ["28172.20", "1.669"], ["28172.30", "7.416"], ["28172.60", "0.886"], ["28172.70", "0.001"], ["28172.80", "0.055"], ["28172.90", "0.053"], ["28173.00", "0.585"], ["28173.10", "0.010"], ["28173.30", "0.014"], ["28173.40", "0.002"], ["28173.50", "3.029"], ["28173.70", "0.344"], ["28173.80", "6.066"], ["28173.90", "0.004"], ["28174.00", "0.001"], ["28174.10", "4.263"], ["28174.20", "0.709"], ["28174.30", "0.096"], ["28174.40", "0.384"], ["28174.60", "0.005"], ["28174.70", "3.405"], ["28174.80", "0.003"], ["28175.00", "3.116"], ["28175.10", "0.123"], ["28175.20", "1.063"], ["28175.40", "3.993"], ["28175.50", "2.137"], ["28175.60", "1.139"], ["28175.70", "0.002"], ["28175.80", "2.647"], ["28176.00", "0.071"], ["28176.10", "0.050"], ["28176.20", "0.011"], ["28176.40", "5.880"], ["28176.50", "0.050"], ["28176.60", "0.001"], ["28176.70", "3.832"], ["28176.80", "3.693"], ["28177.10", "0.001"], ["28177.20", "1.047"], ["28177.30", "0.001"], ["28177.40", "3.001"], ["28177.60", "0.037"], ["28177.80", "3.684"], ["28177.90", "0.040"], ["28178.10", "0.341"], ["28178.20", "3.195"], ["28178.40", "0.846"], ["28178.50", "0.003"], ["28178.60", "2.880"], ["28178.70", "0.763"], ["28178.80", "0.388"], ["28178.90", "0.077"], ["28179.10", "0.003"], ["28179.20", "1.981"], ["28179.30", "3.001"], ["28179.40", "0.029"], ["28179.50", "0.517"], ["28179.60", "0.216"], ["28179.80", "0.450"], ["28179.90", "0.410"], ["28180.00", "4.999"], ["28180.10", "0.074"], ["28180.20", "0.452"], ["28180.30", "3.017"], ["28180.50", "0.037"], ["28180.70", "0.500"], ["28180.90", "3.001"], ["28181.00", "0.180"], ["28181.20", "0.028"], ["28181.30", "0.213"], ["28181.40", "0.054"], ["28181.60", "0.096"], ["28181.70", "0.321"], ["28181.80", "0.082"], ["28181.90", "0.004"], ["28182.00", "5.527"], ["28182.10", "1.942"], ["28182.20", "0.138"], ["28182.30", "3.115"], ["28182.40", "8.624"], ["28182.50", "3.128"], ["28182.60", "0.324"], ["28182.70", "0.554"], ["28182.90", "1.942"], ["28183.00", "3.001"], ["28183.10", "0.219"], ["28183.20", "0.006"], ["28183.30", "0.005"], ["28183.50", "0.052"], ["28183.70", "1.187"], ["28184.00", "0.051"], ["28184.10", "1.392"], ["28184.20", "2.828"], ["28184.30", "2.053"], ["28184.40", "0.093"], ["28184.50", "0.062"], ["28184.60", "0.010"], ["28184.70", "1.937"], ["28184.80", "0.178"], ["28184.90", "0.482"], ["28185.00", "1.207"], ["28185.10", "0.207"], ["28185.20", "0.058"], ["28185.40", "0.056"], ["28185.50", "0.067"], ["28185.70", "0.124"], ["28185.80", "0.002"], ["28185.90", "0.112"], ["28186.00", "0.677"], ["28186.20", "0.233"], ["28186.30", "0.064"], ["28186.40", "0.019"], ["28186.50", "0.050"], ["28186.60", "2.438"], ["28186.70", "0.696"], ["28186.80", "2.121"], ["28186.90", "0.096"], ["28187.00", "0.655"], ["28187.10", "0.075"], ["28187.20", "3.055"], ["28187.30", "0.138"], ["28187.40", "0.490"], ["28187.50", "0.261"], ["28187.60", "0.057"], ["28187.70", "0.249"], ["28187.80", "0.093"], ["28187.90", "0.139"], ["28188.00", "0.511"], ["28188.10", "6.819"], ["28188.20", "4.594"], ["28188.30", "2.910"], ["28188.50", "30.115"], ["28188.60", "0.094"], ["28188.70", "0.301"], ["28188.80", "0.132"], ["28189.00", "0.211"], ["28189.10", "0.035"], ["28189.20", "4.035"], ["28189.30", "0.046"], ["28189.40", "0.077"], ["28189.50", "0.131"], ["28189.60", "5.334"], ["28189.70", "0.017"], ["28189.80", "1.417"], ["28189.90", "0.102"], ["28190.00", "0.331"], ["28190.10", "6.807"], ["28190.20", "1.500"], ["28190.30", "6.361"], ["28190.40", "0.024"], ["28190.50", "0.656"], ["28190.60", "0.395"], ["28190.70", "0.002"], ["28190.80", "0.053"], ["28190.90", "0.512"], ["28191.00", "0.326"], ["28191.10", "0.208"], ["28191.20", "0.014"], ["28191.30", "0.001"], ["28191.40", "0.365"], ["28191.50", "0.316"], ["28191.70", "0.070"], ["28191.80", "3.992"], ["28191.90", "0.053"], ["28192.00", "0.223"], ["28192.10", "0.078"], ["28192.20", "10.683"], ["28192.30", "0.001"], ["28192.40", "0.054"], ["28192.50", "0.010"], ["28192.60", "0.011"], ["28192.70", "0.121"], ["28192.80", "0.367"], ["28192.90", "0.264"], ["28193.00", "0.025"], ["28193.10", "0.059"], ["28193.20", "0.108"], ["28193.30", "0.101"], ["28193.40", "0.948"], ["28193.50", "4.147"], ["28193.60", "0.265"], ["28193.70", "0.014"], ["28193.80", "0.011"], ["28194.00", "2.695"], ["28194.10", "3.476"], ["28194.20", "0.559"], ["28194.30", "0.014"], ["28194.40", "0.387"], ["28194.50", "0.100"], ["28194.60", "0.108"], ["28194.70", "0.111"], ["28194.80", "0.912"], ["28194.90", "0.115"], ["28195.00", "1.881"], ["28195.20", "0.117"], ["28195.30", "0.166"], ["28195.40", "4.603"], ["28195.50", "0.056"], ["28195.60", "0.257"], ["28195.70", "0.005"], ["28195.80", "0.099"], ["28195.90", "0.035"], ["28196.00", "2.935"], ["28196.10", "0.008"], ["28196.20", "0.012"], ["28196.30", "0.113"], ["28196.40", "0.168"], ["28196.50", "3.003"], ["28196.60", "0.060"], ["28196.70", "0.990"], ["28196.80", "4.424"], ["28197.00", "0.170"], ["28197.10", "0.106"], ["28197.20", "2.753"], ["28197.30", "3.056"], ["28197.40", "0.085"], ["28197.50", "0.001"], ["28197.60", "0.433"], ["28197.70", "1.963"], ["28197.80", "1.724"], ["28197.90", "0.090"], ["28198.00", "0.414"], ["28198.10", "0.083"], ["28198.20", "0.436"], ["28198.30", "0.473"], ["28198.40", "1.423"], ["28198.50", "0.123"], ["28198.70", "0.396"], ["28198.80", "0.186"], ["28198.90", "0.011"], ["28199.00", "7.475"], ["28199.10", "2.453"], ["28199.20", "0.380"], ["28199.30", "3.340"], ["28199.40", "4.688"], ["28199.50", "0.153"], ["28199.60", "0.023"], ["28199.70", "0.315"], ["28199.90", "0.014"], ["28200.00", "18.031"], ["28200.10", "1.041"], ["28200.20", "0.505"], ["28200.40", "0.054"], ["28200.50", "0.084"], ["28200.60", "1.081"], ["28200.70", "0.182"], ["28200.80", "4.406"], ["28200.90", "0.409"], ["28201.00", "0.016"], ["28201.10", "0.414"], ["28201.20", "1.624"], ["28201.30", "0.111"], ["28201.40", "2.080"], ["28201.50", "0.268"], ["28201.60", "0.083"], ["28201.70", "0.081"], ["28201.80", "0.186"], ["28201.90", "0.209"], ["28202.00", "3.496"], ["28202.10", "0.055"], ["28202.20", "3.021"], ["28202.30", "0.433"], ["28202.40", "0.219"], ["28202.50", "5.000"], ["28202.60", "0.001"], ["28202.70", "0.255"], ["28202.80", "22.064"], ["28202.90", "0.084"], ["28203.00", "11.714"], ["28203.10", "0.001"], ["28203.20", "0.092"], ["28203.30", "0.233"], ["28203.40", "4.383"], ["28203.50", "19.015"], ["28203.60", "3.235"], ["28203.70", "0.269"], ["28203.80", "0.109"], ["28203.90", "0.296"], ["28204.00", "0.337"], ["28204.10", "0.142"], ["28204.20", "0.087"], ["28204.30", "0.004"], ["28204.40", "6.686"], ["28204.50", "2.388"], ["28204.60", "0.426"], ["28204.70", "0.444"], ["28204.80", "1.497"], ["28204.90", "8.441"], ["28205.00", "0.627"], ["28205.10", "4.597"], ["28205.20", "0.001"], ["28205.30", "0.011"], ["28205.40", "0.064"], ["28205.50", "0.411"], ["28205.60", "0.295"], ["28205.70", "0.213"], ["28205.80", "0.084"], ["28205.90", "1.043"], ["28206.00", "0.227"], ["28206.10", "0.004"], ["28206.20", "0.406"], ["28206.30", "1.735"], ["28206.40", "0.018"], ["28206.50", "2.383"], ["28206.70", "0.973"], ["28206.80", "3.162"], ["28206.90", "3.793"], ["28207.00", "10.365"], ["28207.10", "0.295"], ["28207.20", "0.085"], ["28207.30", "0.196"], ["28207.40", "2.647"], ["28207.50", "4.286"], ["28207.70", "0.506"], ["28207.80", "0.085"], ["28207.90", "1.995"], ["28208.00", "3.232"], ["28208.10", "0.050"], ["28208.20", "0.105"], ["28208.30", "0.071"], ["28208.50", "1.316"], ["28208.60", "0.085"], ["28208.70", "0.849"], ["28208.80", "1.434"], ["28208.90", "2.712"], ["28209.00", "9.793"], ["28209.10", "12.185"], ["28209.20", "0.720"], ["28209.40", "9.574"], ["28209.50", "1.182"], ["28209.60", "0.145"], ["28209.70", "0.292"], ["28209.80", "0.256"], ["28209.90", "0.120"], ["28210.00", "1.127"], ["28210.10", "6.834"], ["28210.20", "30.180"], ["28210.30", "1.563"], ["28210.40", "6.369"], ["28210.50", "0.081"], ["28210.60", "0.167"], ["28210.70", "0.002"], ["28210.90", "0.091"], ["28211.00", "1.032"], ["28211.10", "0.359"], ["28211.20", "4.299"], ["28211.30", "2.474"], ["28211.40", "6.268"], ["28211.50", "0.200"], ["28211.60", "0.289"], ["28211.70", "0.107"], ["28211.80", "3.086"], ["28211.90", "1.842"], ["28212.00", "4.012"], ["28212.10", "3.379"], ["28212.20", "2.258"], ["28212.30", "0.081"], ["28212.40", "2.738"], ["28212.50", "0.015"], ["28212.60", "4.417"], ["28212.70", "0.080"], ["28212.80", "4.372"], ["28212.90", "0.328"], ["28213.00", "0.075"], ["28213.10", "0.910"], ["28213.20", "0.117"], ["28213.30", "3.360"], ["28213.40", "0.379"], ["28213.50", "0.943"], ["28213.60", "0.564"], ["28213.70", "0.009"], ["28213.80", "3.940"], ["28213.90", "0.024"], ["28214.00", "4.199"], ["28214.10", "0.085"], ["28214.20", "0.012"], ["28214.30", "0.175"], ["28214.40", "0.306"], ["28214.50", "0.742"], ["28214.60", "0.170"], ["28214.70", "0.325"], ["28214.80", "0.080"], ["28214.90", "0.994"], ["28215.00", "0.617"], ["28215.10", "0.001"], ["28215.20", "0.561"], ["28215.30", "0.215"], ["28215.40", "0.007"], ["28215.50", "3.623"], ["28215.60", "0.138"], ["28215.70", "0.020"], ["28215.80", "0.317"], ["28215.90", "1.659"], ["28216.00", "0.628"], ["28216.10", "1.021"], ["28216.30", "0.220"], ["28216.40", "1.549"], ["28216.50", "0.211"], ["28216.60", "0.122"], ["28216.70", "0.238"], ["28216.80", "5.845"], ["28216.90", "0.279"], ["28217.00", "1.474"], ["28217.20", "0.129"], ["28217.30", "3.198"], ["28217.40", "0.237"], ["28217.50", "6.120"], ["28217.60", "0.747"], ["28217.70", "0.385"], ["28217.80", "0.001"], ["28217.90", "2.335"], ["28218.00", "0.679"], ["28218.10", "0.331"], ["28218.20", "2.191"], ["28218.30", "0.764"], ["28218.40", "0.341"], ["28218.50", "0.162"], ["28218.60", "2.987"], ["28218.70", "0.152"], ["28218.80", "0.932"], ["28218.90", "1.451"], ["28219.00", "0.386"], ["28219.10", "3.103"], ["28219.20", "0.115"], ["28219.30", "0.247"], ["28219.40", "6.307"], ["28219.50", "0.122"], ["28219.60", "0.041"], ["28219.70", "0.963"], ["28219.80", "0.129"], ["28219.90", "4.433"], ["28220.00", "6.985"], ["28220.10", "0.460"], ["28220.20", "0.372"], ["28220.30", "0.424"], ["28220.40", "1.460"], ["28220.50", "1.451"], ["28220.70", "0.238"], ["28220.80", "2.689"], ["28220.90", "0.289"], ["28221.00", "0.098"], ["28221.10", "0.011"], ["28221.20", "0.001"], ["28221.30", "1.032"], ["28221.40", "0.845"], ["28221.50", "0.148"], ["28221.60", "0.113"], ["28221.70", "0.359"], ["28221.80", "0.055"], ["28222.00", "2.391"], ["28222.10", "0.159"], ["28222.20", "0.012"], ["28222.30", "0.035"], ["28222.40", "0.252"], ["28222.50", "1.670"], ["28222.60", "0.051"], ["28222.70", "0.090"], ["28222.80", "0.005"], ["28222.90", "3.115"], ["28223.00", "0.941"], ["28223.10", "0.211"], ["28223.20", "0.011"], ["28223.30", "0.086"], ["28223.40", "3.336"], ["28223.60", "1.432"], ["28223.70", "11.014"], ["28223.80", "0.052"], ["28223.90", "0.064"], ["28224.00", "15.905"], ["28224.10", "0.186"], ["28224.20", "0.244"], ["28224.30", "0.492"], ["28224.40", "0.125"], ["28224.50", "0.002"], ["28224.60", "0.135"], ["28224.70", "0.026"], ["28224.80", "0.862"], ["28224.90", "0.138"], ["28225.00", "2.189"], ["28225.10", "1.957"], ["28225.20", "0.005"], ["28225.30", "0.035"], ["28225.40", "0.088"], ["28225.50", "0.305"], ["28225.70", "0.191"], ["28225.80", "0.001"], ["28225.90", "2.999"], ["28226.00", "0.361"], ["28226.10", "0.084"], ["28226.20", "0.182"], ["28226.30", "11.900"], ["28226.40", "0.570"], ["28226.50", "1.386"], ["28226.60", "0.002"], ["28226.70", "0.059"], ["28226.80", "0.118"], ["28226.90", "0.115"], ["28227.00", "0.181"], ["28227.10", "0.004"], ["28227.20", "0.221"], ["28227.30", "0.320"], ["28227.40", "0.163"], ["28227.50", "0.080"], ["28227.60", "0.630"], ["28227.80", "0.065"], ["28227.90", "0.279"], ["28228.00", "2.303"], ["28228.10", "0.380"], ["28228.20", "2.645"], ["28228.40", "6.630"], ["28228.50", "0.386"], ["28228.60", "0.370"], ["28228.70", "0.336"], ["28228.80", "0.005"], ["28228.90", "0.159"], ["28229.00", "28.807"], ["28229.10", "0.001"], ["28229.20", "0.004"], ["28229.30", "0.187"], ["28229.40", "0.001"], ["28229.50", "0.901"], ["28229.60", "1.410"], ["28229.70", "0.083"], ["28229.80", "1.477"], ["28229.90", "1.876"], ["28230.00", "6.240"], ["28230.10", "0.311"], ["28230.20", "2.843"], ["28230.30", "0.207"], ["28230.40", "5.485"], ["28230.50", "0.208"], ["28230.60", "3.067"], ["28230.70", "0.235"], ["28230.80", "0.092"], ["28230.90", "0.117"], ["28231.00", "0.041"], ["28231.10", "0.021"], ["28231.20", "0.002"], ["28231.30", "0.006"], ["28231.40", "1.011"], ["28231.50", "0.014"], ["28231.70", "0.252"], ["28231.80", "2.973"], ["28231.90", "0.007"], ["28232.00", "1.155"], ["28232.10", "0.110"], ["28232.20", "0.135"], ["28232.30", "0.050"], ["28232.40", "0.149"], ["28232.50", "0.270"], ["28232.60", "0.203"], ["28232.70", "0.015"], ["28232.80", "0.197"], ["28232.90", "0.145"], ["28233.00", "0.074"], ["28233.20", "0.122"], ["28233.30", "2.320"], ["28233.50", "0.137"], ["28233.60", "0.152"], ["28233.80", "0.196"], ["28233.90", "0.905"], ["28234.00", "3.679"], ["28234.10", "0.169"], ["28234.20", "0.084"], ["28234.30", "0.488"], ["28234.40", "0.126"], ["28234.50", "0.209"], ["28234.60", "0.596"], ["28234.70", "0.357"], ["28234.80", "2.721"], ["28234.90", "0.150"], ["28235.00", "3.057"], ["28235.10", "2.129"], ["28235.20", "0.118"], ["28235.40", "0.165"], ["28235.50", "0.139"], ["28235.60", "0.422"], ["28235.70", "0.329"], ["28235.80", "0.024"], ["28235.90", "0.085"], ["28236.00", "2.251"], ["28236.10", "0.354"], ["28236.20", "0.076"], ["28236.30", "0.099"], ["28236.40", "0.221"], ["28236.50", "8.147"], ["28236.60", "0.166"], ["28236.70", "0.313"], ["28236.90", "1.496"], ["28237.00", "0.084"], ["28237.10", "0.001"], ["28237.20", "0.074"], ["28237.30", "0.096"], ["28237.40", "0.102"], ["28237.50", "5.147"], ["28237.60", "0.066"], ["28237.70", "0.253"], ["28237.90", "0.099"], ["28238.00", "0.046"], ["28238.10", "0.108"], ["28238.20", "1.551"], ["28238.30", "0.286"], ["28238.40", "5.336"], ["28238.50", "0.663"], ["28238.60", "0.009"], ["28238.70", "1.416"], ["28238.80", "2.560"], ["28238.90", "0.233"], ["28239.00", "2.213"], ["28239.10", "0.408"], ["28239.20", "0.281"], ["28239.30", "0.195"], ["28239.40", "0.162"], ["28239.60", "0.188"], ["28239.70", "0.002"], ["28239.80", "0.367"], ["28239.90", "6.932"], ["28240.00", "13.593"], ["28240.10", "0.036"], ["28240.20", "0.118"], ["28240.30", "0.009"], ["28240.40", "10.040"], ["28240.50", "12.444"], ["28240.60", "3.424"], ["28240.70", "2.113"], ["28240.90", "0.362"], ["28241.00", "0.404"], ["28241.20", "0.086"], ["28241.30", "0.093"], ["28241.40", "3.048"], ["28241.60", "0.174"], ["28241.70", "0.296"], ["28241.80", "0.155"], ["28241.90", "0.741"], ["28242.00", "0.040"], ["28242.10", "0.094"], ["28242.20", "0.010"], ["28242.30", "0.188"], ["28242.40", "0.171"], ["28242.50", "0.067"], ["28242.60", "0.144"], ["28242.70", "0.231"], ["28242.80", "0.001"], ["28242.90", "3.156"], ["28243.00", "0.586"], ["28243.10", "8.162"], ["28243.20", "0.030"], ["28243.30", "0.224"], ["28243.40", "3.938"], ["28243.50", "12.026"], ["28243.60", "0.002"], ["28243.70", "0.214"], ["28243.80", "21.460"], ["28243.90", "0.196"], ["28244.00", "0.652"], ["28244.10", "94.981"], ["28244.20", "0.427"], ["28244.30", "0.285"], ["28244.40", "0.180"], ["28244.50", "2.342"], ["28244.60", "0.410"], ["28244.70", "0.003"], ["28244.80", "0.088"], ["28244.90", "6.304"], ["28245.00", "0.848"], ["28245.10", "0.315"], ["28245.20", "1.991"], ["28245.30", "0.366"], ["28245.40", "0.106"], ["28245.50", "0.223"], ["28245.60", "0.223"], ["28245.70", "0.002"], ["28245.90", "0.227"], ["28246.00", "0.316"], ["28246.10", "0.343"], ["28246.20", "2.305"], ["28246.40", "0.021"], ["28246.50", "0.224"], ["28246.60", "0.003"], ["28246.70", "0.025"], ["28246.80", "0.038"], ["28246.90", "0.001"], ["28247.00", "0.486"], ["28247.10", "0.434"], ["28247.20", "0.215"], ["28247.30", "0.119"], ["28247.40", "0.226"], ["28247.50", "0.060"], ["28247.60", "0.225"], ["28247.70", "0.536"], ["28247.80", "0.147"], ["28247.90", "0.080"], ["28248.00", "1.657"], ["28248.10", "0.093"], ["28248.20", "0.317"], ["28248.30", "0.223"], ["28248.40", "0.161"], ["28248.50", "0.954"], ["28248.60", "0.403"], ["28248.70", "0.080"], ["28248.80", "0.005"], ["28248.90", "0.256"], ["28249.00", "3.201"], ["28249.10", "0.309"], ["28249.20", "0.005"], ["28249.30", "0.078"], ["28249.40", "0.268"], ["28249.50", "4.679"], ["28249.60", "0.130"], ["28249.70", "0.210"], ["28249.80", "15.197"], ["28249.90", "4.501"], ["28250.00", "20.080"], ["28250.10", "0.133"], ["28250.20", "0.148"], ["28250.30", "0.106"], ["28250.40", "0.150"], ["28250.50", "0.036"], ["28250.60", "0.044"], ["28250.70", "0.481"], ["28250.80", "0.002"], ["28250.90", "1.351"], ["28251.00", "5.915"], ["28251.10", "0.004"], ["28251.20", "0.176"], ["28251.30", "0.215"], ["28251.40", "0.145"], ["28251.50", "0.202"], ["28251.60", "0.611"], ["28251.80", "0.835"], ["28251.90", "0.288"], ["28252.00", "0.261"], ["28252.10", "0.084"], ["28252.20", "0.005"], ["28252.30", "0.119"], ["28252.60", "0.055"], ["28252.70", "0.005"], ["28252.80", "0.002"], ["28252.90", "0.463"], ["28253.00", "0.070"], ["28253.10", "0.231"], ["28253.20", "0.149"], ["28253.30", "2.858"], ["28253.40", "1.579"], ["28253.50", "0.070"], ["28253.60", "0.103"], ["28253.70", "0.007"], ["28253.80", "0.295"], ["28253.90", "0.001"], ["28254.00", "0.252"], ["28254.10", "0.073"], ["28254.20", "0.020"], ["28254.30", "0.002"], ["28254.40", "0.636"], ["28254.50", "0.171"], ["28254.60", "0.097"], ["28254.70", "0.001"], ["28254.80", "0.007"], ["28255.00", "3.013"], ["28255.10", "0.100"], ["28255.20", "5.642"], ["28255.30", "0.172"], ["28255.40", "0.085"], ["28255.50", "0.131"]]}\n'
b'1680652700600379 {"stream":"btcusdt@depth@0ms","data":{"e":"depthUpdate","E":1680652700594,"T":1680652700586,"s":"BTCUSDT","U":2710246765373,"u":2710246765597,"pu":2710246765326,"b":[["5000.00","14.368"],["28018.90","0.000"],["28049.60","0.288"],["28080.20","3.000"]],"a":[["28145.20","6.961"],["28173.30","0.015"],["28220.30","0.424"]]}}\n'
b'1680652700600379 {"stream":"btcusdt@depth@0ms","data":{"e":"depthUpdate","E":1680652700610,"T":1680652700603,"s":"BTCUSDT","U":2710246765629,"u":2710246765882,"pu":2710246765597,"b":[["28018.90","0.080"],["28080.20","3.080"],["28117.00","0.223"]],"a":[["28220.20","0.452"],["28250.90","1.431"],["28281.50","0.081"]]}}\n'
b'1680652700618593 {"stream":"btcusdt@depth@0ms","data":{"e":"depthUpdate","E":1680652700628,"T":1680652700614,"s":"BTCUSDT","U":2710246765966,"u":2710246766123,"pu":2710246765882,"b":[],"a":[["28173.30","0.014"],["28989.60","0.650"]]}}\n'
The first token of the line is timestamp received by local.
The data needs to be converted to normalized data that can be fed into HftBacktest.
convert method also attempts to correct timestamps by reordering the rows.[2]:
import numpy as np
from hftbacktest.data.utils import binancefutures
data = binancefutures.convert('usdm/btcusdt_20230404.dat.gz')
np.savez('btcusdt_20230404', data=data)
local_timestamp is ahead of exch_timestamp by 18836.0
found 542 rows that exch_timestamp is ahead of the previous exch_timestamp
Correction is done.
You can save the data directly to a file by providing output_filename.
[3]:
binancefutures.convert('usdm/btcusdt_20230405.dat.gz', output_filename='btcusdt_20230405')
local_timestamp is ahead of exch_timestamp by 26932.0
found 6555 rows that exch_timestamp is ahead of the previous exch_timestamp
Correction is done.
Saving to btcusdt_20230405
[3]:
array([[ 1.00000000e+00, 1.68065280e+15, 1.68065280e+15,
1.00000000e+00, 2.23000000e+04, 2.78800000e+00],
[ 1.00000000e+00, 1.68065280e+15, 1.68065280e+15,
1.00000000e+00, 2.75774000e+04, 0.00000000e+00],
[ 1.00000000e+00, 1.68065280e+15, 1.68065280e+15,
1.00000000e+00, 2.80238000e+04, 1.63800000e+00],
...,
[ 1.00000000e+00, 1.68065321e+15, 1.68065321e+15,
-1.00000000e+00, 2.81499000e+04, 1.53200000e+00],
[ 1.00000000e+00, 1.68065321e+15, 1.68065321e+15,
-1.00000000e+00, 2.85725000e+04, 1.83000000e-01],
[ 1.00000000e+00, 1.68065321e+15, 1.68065321e+15,
-1.00000000e+00, 2.89844000e+04, 1.00000000e-03]])
Normalized data as follows. You can find more details on Data.
[4]:
import pandas as pd
df = pd.DataFrame(data, columns=['event', 'exch_timestamp', 'local_timestamp', 'side', 'price', 'qty'])
df['event'] = df['event'].astype(int)
df['exch_timestamp'] = df['exch_timestamp'].astype(int)
df['local_timestamp'] = df['local_timestamp'].astype(int)
df['side'] = df['side'].astype(int)
df
[4]:
| event | exch_timestamp | local_timestamp | side | price | qty | |
|---|---|---|---|---|---|---|
| 0 | 2 | 1680652700452000 | 1680652700460369 | -1 | 28145.1 | 0.002 |
| 1 | 2 | 1680652700452000 | 1680652700460521 | -1 | 28145.1 | 0.020 |
| 2 | 2 | 1680652700452000 | 1680652700460561 | -1 | 28145.1 | 0.020 |
| 3 | 2 | 1680652700452000 | 1680652700461364 | -1 | 28145.1 | 0.008 |
| 4 | 2 | 1680652700462000 | 1680652700473746 | 1 | 28145.2 | 0.002 |
| ... | ... | ... | ... | ... | ... | ... |
| 71014 | 1 | 1680652799975000 | 1680652799977784 | -1 | 28182.7 | 0.441 |
| 71015 | 1 | 1680652799975000 | 1680652799977784 | -1 | 28186.9 | 0.054 |
| 71016 | 1 | 1680652799975000 | 1680652799977784 | -1 | 28225.5 | 3.213 |
| 71017 | 1 | 1680652799975000 | 1680652799977784 | -1 | 28231.7 | 0.356 |
| 71018 | 1 | 1680652799975000 | 1680652799977784 | -1 | 28251.4 | 0.262 |
71019 rows × 6 columns
Creating a market depth snapshot
As Binance Futures exchange runs 24/7, you need the initial snapshot to get the complete(almost) market depth.
collect-binancefutures fetches the snapshot only when it makes the connection, so you need build the initial snapshot from the start of the collected feed data.[5]:
from hftbacktest.data.utils import create_last_snapshot
# Build 20230404 End of Day snapshot. It will be used for the initial snapshot for 20230405.
data = create_last_snapshot('btcusdt_20230404.npz', tick_size=0.01, lot_size=0.001)
np.savez('btcusdt_20230404_eod.npz', data=data)
# Build 20230405 End of Day snapshot.
# Due to the file size limitation, btcusdt_20230405.npz does not contain data for the entire day.
create_last_snapshot(
'btcusdt_20230405.npz',
tick_size=0.01,
lot_size=0.001,
initial_snapshot='btcusdt_20230404_eod.npz',
output_snapshot_filename='btcusdt_20230405_eod'
)
Load btcusdt_20230404.npz
Load btcusdt_20230405.npz
[5]:
array([[ 4.00000000e+00, 1.68065321e+15, -1.00000000e+00,
1.00000000e+00, 2.81401000e+04, 8.25100000e+00],
[ 4.00000000e+00, 1.68065321e+15, -1.00000000e+00,
1.00000000e+00, 2.81400000e+04, 1.62000000e-01],
[ 4.00000000e+00, 1.68065321e+15, -1.00000000e+00,
1.00000000e+00, 2.81399000e+04, 4.00000000e-03],
...,
[ 4.00000000e+00, 1.68065321e+15, -1.00000000e+00,
-1.00000000e+00, 3.09404800e+05, 2.00000000e-03],
[ 4.00000000e+00, 1.68065321e+15, -1.00000000e+00,
-1.00000000e+00, 3.09425600e+05, 7.00000000e-03],
[ 4.00000000e+00, 1.68065321e+15, -1.00000000e+00,
-1.00000000e+00, 3.09443200e+05, 5.00000000e-03]])
[6]:
import pandas as pd
df = pd.DataFrame(data, columns=['event', 'exch_timestamp', 'local_timestamp', 'side', 'price', 'qty'])
df['event'] = df['event'].astype(int)
df['exch_timestamp'] = df['exch_timestamp'].astype(int)
df['local_timestamp'] = df['local_timestamp'].astype(int)
df['side'] = df['side'].astype(int)
df
[6]:
| event | exch_timestamp | local_timestamp | side | price | qty | |
|---|---|---|---|---|---|---|
| 0 | 4 | 1680652799977784 | -1 | 1 | 28155.1 | 0.060 |
| 1 | 4 | 1680652799977784 | -1 | 1 | 28155.0 | 0.004 |
| 2 | 4 | 1680652799977784 | -1 | 1 | 28154.9 | 0.001 |
| 3 | 4 | 1680652799977784 | -1 | 1 | 28154.8 | 0.001 |
| 4 | 4 | 1680652799977784 | -1 | 1 | 28154.7 | 0.002 |
| ... | ... | ... | ... | ... | ... | ... |
| 4092 | 4 | 1680652799977784 | -1 | -1 | 30827.5 | 1.620 |
| 4093 | 4 | 1680652799977784 | -1 | -1 | 31500.0 | 33.077 |
| 4094 | 4 | 1680652799977784 | -1 | -1 | 33500.0 | 11.648 |
| 4095 | 4 | 1680652799977784 | -1 | -1 | 33752.3 | 0.001 |
| 4096 | 4 | 1680652799977784 | -1 | -1 | 33783.5 | 12.417 |
4097 rows × 6 columns
Getting started from Tardis.dev data
Few vendors offer tick-by-tick full market depth data along with snapshot and trade data, and Tardis.dev is among them.
[ ]:
# https://docs.tardis.dev/historical-data-details/binance-futures
# Download sample Binance futures BTCUSDT trades
!wget https://datasets.tardis.dev/v1/binance-futures/trades/2020/02/01/BTCUSDT.csv.gz -O BTCUSDT_trades.csv.gz
# Download sample Binance futures BTCUSDT book
!wget https://datasets.tardis.dev/v1/binance-futures/incremental_book_L2/2020/02/01/BTCUSDT.csv.gz -O BTCUSDT_book.csv.gz
[8]:
from hftbacktest.data.utils import tardis
data = tardis.convert(['BTCUSDT_trades.csv.gz', 'BTCUSDT_book.csv.gz'])
np.savez('btcusdt_20200201.npz', data=data)
Reading BTCUSDT_trades.csv.gz
Reading BTCUSDT_book.csv.gz
Merging
found 20948 rows that exch_timestamp is ahead of the previous exch_timestamp
Correction is done.
You can save the data directly to a file by providing output_filename. If there are too many rows, you need to increase buffer_size.
[9]:
tardis.convert(
['BTCUSDT_trades.csv.gz', 'BTCUSDT_book.csv.gz'],
output_filename='btcusdt_20200201.npz',
buffer_size=200_000_000
)
Reading BTCUSDT_trades.csv.gz
Reading BTCUSDT_book.csv.gz
Merging
found 20948 rows that exch_timestamp is ahead of the previous exch_timestamp
Correction is done.
Saving to btcusdt_20200201.npz
[9]:
array([[ 2.0000000e+00, 1.5805152e+15, 1.5805152e+15, 1.0000000e+00,
9.3645100e+03, 1.1970000e+00],
[ 2.0000000e+00, 1.5805152e+15, 1.5805152e+15, 1.0000000e+00,
9.3656700e+03, 2.0000000e-02],
[ 2.0000000e+00, 1.5805152e+15, 1.5805152e+15, 1.0000000e+00,
9.3658600e+03, 1.0000000e-02],
...,
[ 1.0000000e+00, 1.5806016e+15, 1.5806016e+15, 1.0000000e+00,
9.3514700e+03, 3.9140000e+00],
[ 1.0000000e+00, 1.5806016e+15, 1.5806016e+15, -1.0000000e+00,
9.3977800e+03, 1.0000000e-01],
[ 1.0000000e+00, 1.5806016e+15, 1.5806016e+15, 1.0000000e+00,
9.3481400e+03, 3.9800000e+00]])
You can also build the snapshot in the same way as described above.