Skip to content

IBKR TWS wire-protocol notes

Reference notes for the subset of the TWS API we implement in IbkrBroker. Companion to docs/ibkr-connectivity.md (which covers the user-side setup) and docs/live-trading-safety.md (which covers what we do with the connection).

Why we don't vendor IBKR's official TWSAPI

IBKR publishes a C++ client (TwsApi/source/cppclient/client/) that decodes the protocol. We're not vendoring it. Reasons:

  • License. The TWS API license forbids redistribution without written permission. Checking it into this repo is the wrong move.
  • Size. ~50 generated .h / .cpp files, plus a build system that doesn't fit CMake without invasive surgery.
  • API mismatch. It assumes asynchronous callback dispatch via EWrapper. Our IBroker interface is synchronous request / response (qe::Result<T>). Adapting their callback model means wrapping every method in a "send, block on condvar, unblock from callback" pattern — which we can write directly against the wire protocol with less ceremony.
  • Scope. We need ~10 message types. Their client supports hundreds.

So: hand-rolled minimal client. Wire-protocol notes follow.

Connection layer

TWS / IB Gateway listens on a TCP port (see connectivity doc). The handshake is:

  1. Client opens TCP connection.
  2. Client sends an API version banner: API\0 followed by a 4-byte big-endian length prefix and the string "v<min>..<max>" (e.g. "v100..151").
  3. Server responds with <server-version>\0<connection-time>\0 (newline-separated ASCII).
  4. Client sends a START_API message (id 71) containing client id
  5. optional capabilities.
  6. Server may send back account list (managedAccts), next-valid-order-id (nextValidId), then is ready for requests.

All messages after the banner use the same framing:

+----+----+----+----+--------+
| 4-byte BE length | payload |
+----+----+----+----+--------+

Payload is ASCII fields separated by \0 (null bytes), ending with a trailing \0. Every field is a string; the protocol does not distinguish ints from strings at the wire level — both sides agree by position.

Each message starts with a numeric msg_id and (for most incoming messages) a version field.

Message types we need

Direction id Name Why
→ server 71 START_API Handshake completion
→ server 3 REQ_ACCOUNT_UPDATES Subscribe to account snapshot
→ server 61 REQ_POSITIONS Position list
→ server 62 CANCEL_POSITIONS Stop position stream
→ server 3 PLACE_ORDER (id 3) Submit an order
→ server 4 CANCEL_ORDER Cancel by orderId
→ server 5 REQ_OPEN_ORDERS One-shot open-order snapshot
← server 5 OPEN_ORDER Open order row (one per)
← server 53 OPEN_ORDER_END End of open-order batch
← server 3 ORDER_STATUS Order state transition
← server 11 EXEC_DETAILS Fill notification
← server 61 POSITION_DATA Position row
← server 62 POSITION_END End of position batch
← server 6 ACCT_VALUE Account field key/value
← server 9 NEXT_VALID_ID First usable orderId
← server 4 ERR_MSG Error / warning

That's it. Charts, market data, options-chain queries, etc. live behind separate ids that the dashboard already covers via Yahoo and Alpaca — we don't double up on IBKR for those.

Order ids

IBKR's orderId is a monotonically-increasing per-client integer the client picks, not server-assigned. The server returns the next-valid starting id during handshake; we use that as our seed and increment per submission. Our client_order_id (idempotency key) is sent in the order's orderRef field — IBKR echoes it back in OPEN_ORDER rows.

Error handling

ERR_MSG (id 4) carries (reqId, errorCode, errorMessage). Numeric error codes 100–449 are warnings, 1100–2110 are connection events, 2000+ are warnings/notices, 10000+ are order-side rejects. Our adapter maps:

  • 1100 / 1101 / 1102 → connection state change (badge OFFLINE)
  • 200, 201, 202, 203 → order reject (Result<> Error in submit_order)
  • 502, 503, 504 → bad client connection (IoError)
  • Everything else → log + ignore (or surface to UI via the reconcile modal)

Threading

Read side runs on a dedicated worker thread that:

  1. Reads 4-byte length, then payload, indefinitely.
  2. Decodes msg_id + version + fields, dispatches to a per-id handler.
  3. Handlers either update shared state (open orders, positions, account) or signal a request's pending condvar.

Write side is mutex-guarded — any caller can submit_order, cancel_order, etc. concurrently; sends are serialized.

Request/response correlation: server only knows requestIds we gave it. We map requestId → pending future internally.

Reconnect

If the read thread sees EOF or a socket error:

  1. Mark connection state OFFLINE.
  2. Cancel all pending request futures with IoError.
  3. Don't auto-reconnect. Per safety doc: "We never auto-restart TWS." The user re-launches Gateway and (eventually) the dashboard's Reconnect IBKR action triggers a new IbkrBroker::connect().

Test posture

Pure protocol — no network — tests cover:

  • Banner roundtrip: encode v100..151, decode server version.
  • Message framing: length-prefixed null-separated fields encode + decode roundtrip for each message type we use.
  • Field encoding: int / double / bool / string serializers.
  • Error-code → ErrorCode mapping table.

Network tests live behind the same ALPACA_PAPER_LIVE_TESTS gate pattern (IBKR_PAPER_LIVE_TESTS=1 env var) so CI doesn't try to hit a Gateway that isn't there.