4. Input

An input is the entry node of a robot: a named, duration-stamped cache of a processed time series, built over a set of contracts on one market-data adapter. Raw ticks arrive at the adapter; the input hands each tick to your data processor; whatever the processor stores becomes the series that patterns observe (the Pattern chapter). Every input belongs to an account, is addressed by its unique label, and states its duration explicitly — the engine never derives it.

Your secret sauce

The data processor is the one place in the whole pipeline where your ideas live, and the engine treats it as a sealed box. What crosses the boundary is deliberately minimal: a tick comes in, the processor may push a (timestamp, value) pair into the input's storage, and a single readiness flag goes back. Nothing about the computation itself is visible to the engine — it holds a function pointer with a fixed signature and an opaque userData pointer, and it calls them.

The consequence is full interchangeability. A three-line moving average, a gradient-boosted model, a neural network scoring news sentiment, or a call out to an external service are all the same thing to the engine: a callable that sometimes stores a number and reports whether the input is ready. Swapping one for the other changes no wiring — the same builders, the same patterns, the same rules keep working. The engine supplies the machinery (caching, contract routing, timestamp alignment, threading); the processor supplies the edge.

Building inputs

Six input builders exist. Four are declared in tse_input.h — tse_add_input_ohlcv, tse_add_input_bidask, tse_add_input_trade, tse_add_input_executed. The remaining two, tse_add_input_book and tse_add_input_book_imbalance, live in tse_book.h with the rest of the order-book surface (the order-book chapter). All four tse_input.h builders share one shape and differ only in the processor typedef and tick type.

tse_add_input_bidask takes a TseBidAskInputProcessor over TseTickBidAsk const*, tse_add_input_trade a TseTradeInputProcessor over TseTickTrade const*, and tse_add_input_executed a TseExecutedInputProcessor over TseRetained const* — everything else is identical. Note the OHLCV typedef name: ABI version 6 renamed the old TseInputProcessor to TseOhlcvInputProcessor, making every processor typedef name its feed. The C++ wrapper kept the short alias tse::InputProcessor for its OHLCV std::function form.

Every builder returns TseStatus; on tse_error the reason is available through tse_last_error. The C++ wrapper checks the status for you and throws tse::Error; Python raises tse.TseError.

The builders take the same parameters, whatever the feed:

typedef int (*TseOhlcvInputProcessor)
(
    TseStorageHandle    storage,
    char const*         contractId,
    TseTickOHLCV const* tick,
    void*               userData
);

TseStatus tse_add_input_ohlcv
(
    TseAccountHandle       account,
    char const*            label,
    int32_t                dataLength,
    TseDuration            duration,
    TseOhlcvInputProcessor processor,
    void*                  userData,
    TseMarketHandle        market,
    int                    coreId,
    char const* const*     contractSymbols,
    size_t                 contractCount
);

void Account::addInputOhlcv   (std::string label, std::int32_t dataLength, Duration duration, InputProcessor processor,         Market const& market, std::vector<std::string> contractSymbols, int coreId = -1) &;
void Account::addInputBidAsk  (std::string label, std::int32_t dataLength, Duration duration, BidAskInputProcessor processor,   Market const& market, std::vector<std::string> contractSymbols, int coreId = -1) &;
void Account::addInputTrade   (std::string label, std::int32_t dataLength, Duration duration, TradeInputProcessor processor,    Market const& market, std::vector<std::string> contractSymbols, int coreId = -1) &;
void Account::addInputExecuted(std::string label, std::int32_t dataLength, Duration duration, ExecutedInputProcessor processor, Market const& market, std::vector<std::string> contractSymbols, int coreId = -1) &;

Account.add_input_ohlcv(label, data_length, duration, processor, market, contract_symbols, core_id=-1)
Account.add_input_bidask(label, data_length, duration, processor, market, contract_symbols, core_id=-1)
Account.add_input_trade(label, data_length, duration, processor, market, contract_symbols, core_id=-1)
Account.add_input_executed(label, data_length, duration, processor, market, contract_symbols, core_id=-1)

Data length and capacity

dataLength is the requested capacity of the input's value storage — the series your processor pushes into and patterns read from. The storage is a ring: its actual capacity is the requested length rounded up to the next power of two, and once it is full each new push overwrites the oldest element. A dataLength of 15 therefore yields a buffer of 16, which changes only how much history the node retains — the numbers your processor computes are unaffected. Choose dataLength from what your computation needs to look back on; readiness is entirely your call — the engine imposes no minimum fill.

The processor contract

The processor receives four things: the storage handle (valid only for the duration of the call), the symbol of the contract whose tick is being processed, a pointer to the typed tick, and your userData. Inside the call, two functions operate on the storage.

The return value is the readiness flag. Returning non-zero (C), true (C++) or a truthy value (Python) declares the input ready: the newest stored value is published to every pattern observing the input. Returning zero declares the input not ready: whatever was pushed stays in the storage, but nothing is published and no pattern sees an update — the engine records the not-ready outcome in its log and moves on. There is no error-code channel out of a processor: not-ready is the only outward signal. The C++ and Python trampolines enforce this — any exception escaping your callable is caught at the boundary and converted into the not-ready return, so exceptions never cross the C ABI.

void   tse_storage_push(TseStorageHandle storage, int64_t tsNanoseconds, double value);
size_t tse_storage_size(TseStorageHandle storage);

class Storage final {
    void push(std::int64_t tsNanoseconds, double value) const noexcept;
    std::size_t size() const noexcept;
};

Storage.push(ts_nanoseconds, value)
Storage.size()

A minimal Python processor — a simple moving average that becomes ready once the window is full:

def make_sma(period):
    window = []
    def processor(storage, contract_id, tick):
        window.append(tick.close)
        del window[:-period]
        storage.push(tick.tsNanoseconds, sum(window) / len(window))
        return storage.size() >= period
    return processor

Feed types

Five feeds can drive an input; each pairs one adapter type with one tick record and one processor typedef.

One asymmetry of the trade feed is worth knowing: the engine's internal representation of the trade channel keeps the traded price only. You push a full TseTickTrade (price, volume, side), but a trade-input processor receives the price with volume equal to zero and side equal to tse_side_undefined.

The executed channel

The executed feed turns your own fills into an input: a robot can observe the stream of executed trades exactly the way it observes quotes. The channel has no internal producer — the caller pushes each executed trade into an adapter of type tse_md_executed, addressed by contract name or id.

The executed feed and the book feed are pure input sources: they carry no price the portfolio could mark against and no venue the simulator could fill on, so a simulator cannot be attached to them and a portfolio subscription on them is refused with a diagnostic.

The record is TseRetained — the same POD that external trade booking uses (tse_trades.h):

TseStatus tse_market_push_executed_by_name(TseMarketHandle market, char const* contractName, TseRetained const* trade);
TseStatus tse_market_push_executed_by_id(TseMarketHandle market, uint64_t contractId, TseRetained const* trade);

void Market::pushExecuted(std::string contractName, Retained const& trade) const &;
void Market::pushExecuted(std::uint64_t contractId, Retained const& trade) const &;

Market.push_executed_by_name(contract_name, trade)
Market.push_executed_by_id(contract_id, trade)

Book inputs

tse_add_input_book runs your processor over every TseBookMessage reaching the bound (adapter, contract) pair; to maintain an actual book inside the processor, create one with tse_book_create and apply each message via tse_book_apply. tse_add_input_book_imbalance is the compiled-in alternative: it feeds every message into a book you supply and stores the whole-book imbalance per message, with zero callback crossings. Both are covered with the rest of the book surface in the order-book chapter.

Contract binding

Contracts are declared on the account (tse_add_contract) independently of any feed; an input's contract list is what places them on an adapter. Building an input registers each named contract on that adapter and wires the input to observe exactly that subset — there is no standalone "add contract to adapter" call, and pushing a tick for a contract not registered on the adapter is an error. Three roles must not be confused:

An input bound to one contract never sees another contract's ticks, even when both flow through the same adapter. Trading a contract does not feed any input: a robot may observe contract B while its rules trade contract A — the motivating case for the split — provided A is registered on the adapter, which is what tse_portfolio_subscribe is for when no input binds it. Subscription gives the portfolio its marking stream and the simulator its fill stream without routing a single tick into a processor.

The one hard placement requirement is enforced at start time. Every contract a robot trades must be carried by at least one price-carrying market adapter — an adapter of the OHLCV, bid/ask or trade type; the executed and book channels carry no price a portfolio could mark against. tse_start walks the started robot's traded contracts, attaches the simulator to every carrying adapter and subscribes the portfolio on the price-carrying ones by itself; when the execution is the simulator and no price-carrying adapter carries a traded contract, the start fails with the error traded contract "…" is on no price-carrying market adapter; the Simulator needs market data to fill it. Binding an input to the symbol is therefore sufficient — the input places the contract on its adapter, and the start does the rest; an explicit tse_portfolio_subscribe remains the manual tool for contracts outside any input's list.

The contract list is validated at build time, and a failed builder leaves the account fully usable:

• an empty list is rejected — at least one contract must be specified;
• an empty symbol string is rejected;
• a symbol not declared on the account is rejected — declare contracts before building inputs;
• a duplicated symbol within the list is rejected by the core.

Finally, an input that no pattern references is inert. tse_start claims exactly the started robot's dependency component — robot, its rules, their patterns, and the inputs those patterns observe. A bare input outside that component is never attached: it receives nothing, computes nothing, and costs nothing until some started robot's pattern names it.

Persistence keys

A processor is a function pointer, and a function pointer cannot be serialized. The plain builders above therefore produce a robot that runs but cannot be saved. To make an input savable, register the processor under a string key on the account and build the input with the keyed variant — the recipe then stores the key, and any process that registers the same key can rebuild the input.

A saved recipe stores each input with its market-data type, duration, data length, contract subset and processor key — but bound to no adapter. After tse_load on a fresh account that has registered the same keys, every loaded input is unbound: create the market adapter(s) and execution you want, then bind each input with tse_bind_input(account, inputLabel, market). Binding checks that the adapter's type matches the input's saved market-data type and registers the input's contract subset on the new adapter. Starting a robot while any loaded input is still unbound is an error. The save/load machinery itself — recipes, keys and the storage format — belongs to the environment chapter.

The registration and keyed-builder families cover the four tse_input.h feeds:

The book inputs have no keyed form. Pattern formulas follow the same scheme with tse_register_formula_processor and tse_add_pattern_formula_by_key (the Pattern chapter).

TseStatus tse_register_input_processor_ohlcv(TseAccountHandle account, char const* key, TseOhlcvInputProcessor processor, void* userData);
TseStatus tse_add_input_ohlcv_by_key(TseAccountHandle account, char const* label, int32_t dataLength, TseDuration duration, char const* key, TseMarketHandle market, char const* const* contractSymbols, size_t contractCount);

void Account::registerInputProcessor(std::string key, InputProcessor processor) &;
void Account::addInputOhlcvByKey(std::string label, std::int32_t dataLength, Duration duration, std::string key, Market const& market, std::vector<std::string> contractSymbols) &;
void Account::bindInput(std::string inputLabel, Market const& market) &;

Account.register_input_processor(key, processor)
Account.add_input_ohlcv_by_key(label, data_length, duration, key, market, contract_symbols)
Account.bind_input(input_label, market)

Version 5.0.0.0