Data Validation

correct_event_order(data, sorted_exch_index, sorted_local_index)[source]

Corrects exchange timestamps that are reversed by splitting each row into separate events. These events are then ordered by both exchange and local timestamps through duplication. See the data for details.

Parameters:
  • data (ndarray[Any, dtype([('ev', '<u8'), ('exch_ts', '<i8'), ('local_ts', '<i8'), ('px', '<f8'), ('qty', '<f8'), ('order_id', '<u8'), ('ival', '<i8'), ('fval', '<f8')], align=True)]) – Data to be corrected.

  • sorted_exch_index (ndarray[Any, dtype[_ScalarType_co]]) – Index of data sorted by exchange timestamp.

  • sorted_local_index (ndarray[Any, dtype[_ScalarType_co]]) – Index of data sorted by local timestamp.

Returns:

Data with the corrected event order.

Return type:

ndarray[Any, dtype([(‘ev’, ‘<u8’), (‘exch_ts’, ‘<i8’), (‘local_ts’, ‘<i8’), (‘px’, ‘<f8’), (‘qty’, ‘<f8’), (‘order_id’, ‘<u8’), (‘ival’, ‘<i8’), (‘fval’, ‘<f8’)], align=True)]

correct_local_timestamp(data, base_latency)[source]

Adjusts the local timestamp in place if the feed latency is negative by offsetting it by the maximum negative latency value as follows:

feed_latency = local_timestamp - exch_timestamp
adjusted_local_timestamp = local_timestamp + min(feed_latency, 0) + base_latency
Parameters:
  • data (ndarray[Any, dtype([('ev', '<u8'), ('exch_ts', '<i8'), ('local_ts', '<i8'), ('px', '<f8'), ('qty', '<f8'), ('order_id', '<u8'), ('ival', '<i8'), ('fval', '<f8')], align=True)]) – Data to be corrected.

  • base_latency (float) – Due to discrepancies in system time between the exchange and the local machine, latency may be measured inaccurately, resulting in negative latency values. The conversion process automatically adjusts for positive latency but may still produce zero latency cases. By adding base_latency, more realistic values can be obtained. Unit should be the same as the feed data’s timestamp unit.

Returns:

Data with the corrected timestamps.

Return type:

ndarray[Any, dtype([(‘ev’, ‘<u8’), (‘exch_ts’, ‘<i8’), (‘local_ts’, ‘<i8’), (‘px’, ‘<f8’), (‘qty’, ‘<f8’), (‘order_id’, ‘<u8’), (‘ival’, ‘<i8’), (‘fval’, ‘<f8’)], align=True)]

validate_event_order(data)[source]

Validates that the order of events is correct. If the data contains an incorrect event order, a ValueError will be raised.

Parameters:

data (ndarray[Any, dtype([('ev', '<u8'), ('exch_ts', '<i8'), ('local_ts', '<i8'), ('px', '<f8'), ('qty', '<f8'), ('order_id', '<u8'), ('ival', '<i8'), ('fval', '<f8')], align=True)]) – Data to validate.

Return type:

None