Data Validation
- correct_event_order(data, sorted_exch_index, sorted_local_index)[source]
Corrects exchange timestamps that are reversed by splitting each row into separate events. These events are then ordered by both exchange and local timestamps through duplication. See the data for details.
- Parameters:
data (ndarray[Any, dtype([('ev', '<u8'), ('exch_ts', '<i8'), ('local_ts', '<i8'), ('px', '<f8'), ('qty', '<f8'), ('order_id', '<u8'), ('ival', '<i8'), ('fval', '<f8')], align=True)]) – Data to be corrected.
sorted_exch_index (ndarray[Any, dtype[_ScalarType_co]]) – Index of data sorted by exchange timestamp.
sorted_local_index (ndarray[Any, dtype[_ScalarType_co]]) – Index of data sorted by local timestamp.
- Returns:
Data with the corrected event order.
- Return type:
ndarray[Any, dtype([(‘ev’, ‘<u8’), (‘exch_ts’, ‘<i8’), (‘local_ts’, ‘<i8’), (‘px’, ‘<f8’), (‘qty’, ‘<f8’), (‘order_id’, ‘<u8’), (‘ival’, ‘<i8’), (‘fval’, ‘<f8’)], align=True)]
- correct_local_timestamp(data, base_latency)[source]
Adjusts the local timestamp in place if the feed latency is negative by offsetting it by the maximum negative latency value as follows:
feed_latency = local_timestamp - exch_timestamp adjusted_local_timestamp = local_timestamp + min(feed_latency, 0) + base_latency
- Parameters:
data (ndarray[Any, dtype([('ev', '<u8'), ('exch_ts', '<i8'), ('local_ts', '<i8'), ('px', '<f8'), ('qty', '<f8'), ('order_id', '<u8'), ('ival', '<i8'), ('fval', '<f8')], align=True)]) – Data to be corrected.
base_latency (float) – Due to discrepancies in system time between the exchange and the local machine, latency may be measured inaccurately, resulting in negative latency values. The conversion process automatically adjusts for positive latency but may still produce zero latency cases. By adding
base_latency
, more realistic values can be obtained. Unit should be the same as the feed data’s timestamp unit.
- Returns:
Data with the corrected timestamps.
- Return type:
ndarray[Any, dtype([(‘ev’, ‘<u8’), (‘exch_ts’, ‘<i8’), (‘local_ts’, ‘<i8’), (‘px’, ‘<f8’), (‘qty’, ‘<f8’), (‘order_id’, ‘<u8’), (‘ival’, ‘<i8’), (‘fval’, ‘<f8’)], align=True)]
- validate_event_order(data)[source]
Validates that the order of events is correct. If the data contains an incorrect event order, a
ValueError
will be raised.