Skip to main content

Reconnect and reconcile durable state

A reconnected socket starts a new connection epoch. It does not prove that every order, execution, account, contract, and subscription invariant has been recovered. Keep application readiness false until fresh broker observations have been matched against durable intent and the reconciliation result itself is durable.

There is no single “orders from the last 24 hours” call. Every recovery surface has a different ownership, update, and history boundary.

Visibility matrix

SurfaceWhat it containsImportant boundary
IB.trades()A copy of all Trade objects currently known to this IB instanceSession projection only; wrapper reset erases it. Completed-order requests can add previously unknown trades.
IB.openTrades() / IB.openOrders()Session trades/orders whose status is not in the pinned library's done-state setPreferred maintained cache after successful startup synchronization; visibility still depends on client/master configuration.
IB.reqOpenOrdersAsync()Active orders placed with this exact client IDClient 0 also binds current manual TWS orders, changing their API IDs; that binding is rejected in API read-only mode. The pinned library warns the returned snapshot can be stale.
IB.reqAllOpenOrdersAsync()One current snapshot of open orders in associated accountsIt does not start a subscription, keep other clients' orders synchronized, transfer ownership, or grant modification rights.
IB.reqCompletedOrdersAsync(apiOnly)Current-day orders no longer modifiableExecuted, rejected, and cancelled orders; not an unlimited or rolling 24-hour history.
IB.reqExecutionsAsync(filter)Executions in the supported history/filter scopeCurrent-day by default; TWS configuration can extend its trade-log scope, while IB Gateway remains current-day. Deduplicate exact execId values.
Flex reportsReporting-period activity statementsBack-office control and longer history, not a live order-control surface; use the version 3 request lifecycle to retrieve a report instance.

The same client ID reconnecting can recover activity that remains in that scope. A configured Master API Client ID can receive other API clients' order and trade data. Client ID 0 can additionally receive TWS and FIX activity. Broader visibility is not broader authority: modification and cancellation rules still apply, and a foreign or manual order must be explicitly classified by policy.

Pinned request ownership

IB.reqOpenOrdersAsync() and IB.reqAllOpenOrdersAsync() both register the constant wrapper request key "openOrders". Never overlap them. A second call replaces the first call's future/result ownership, so even identical-looking recovery requests need one serialized connection owner.

During an open-order request, pinned ib_async updates the session trade projection and appends each trade to the request result; it does not emit the ordinary live openOrderEvent for those result callbacks. openOrderEnd completes the shared request. reqCompletedOrdersAsync() owns a separate "completedOrders" slot; its callbacks return Trade objects and add a previously unknown positive permId to the session projection.

Use a finite deadline, retain the returned future, and on timeout or caller cancellation cancel that local future and recycle the connection epoch. The pinned async implementations register wrapper-owned futures rather than returning request IDs that this workflow can cancel through the documented public API. Continuing after an ambiguous partial snapshot can create false absence and duplicate-order risk.

Recovery order

For a new local socket epoch:

  1. Connect with the intended client ID, order-visibility configuration, and raiseSyncErrors=True.
  2. Verify the exact accessible-versus-authorized account set again.
  3. Use the post-startup openTrades() cache for the primary open-order view. Do not immediately re-request reqOpenOrdersAsync() merely for reassurance.
  4. If policy needs a broader one-shot audit, call reqAllOpenOrdersAsync() serially and classify every foreign/manual order without assuming ownership.
  5. Request completed orders, then executions. Ordering executions after orders improves permId attachment in the pinned wrapper.
  6. Persist open, completed, execution, commission, and request-scope observations under one reconciliation-run ID.
  7. Match durable intent to observations, fail on ambiguous identity, and mark absence as unknown rather than cancelled.
  8. Rebuild every subscription for a new local epoch and prove its own readiness. A previous upstream 1102 does not preserve wrapper subscriptions across a local disconnect/reset.
  9. Persist the reconciliation decision before publishing application readiness.

The default connectAsync startup flags already request open orders, completed orders, and executions, with executions after the order requests. This is useful only when the configured scope matches the application. If a startup request times out and raiseSyncErrors=False, the library can log the error and still emit connectedEvent; production recovery should use raiseSyncErrors=True and additional application invariants.

Correlation and fail-closed outcomes

Correlate in this order, retaining every identifier:

  1. Positive permId within the authorized account scope.
  2. Exact account plus unique durable orderRef/application intent ID.
  3. Exact client ID plus API order ID when the application knows that identity was assigned; preserve zero values rather than treating them as missing.

Every durable order record needs a nonempty application intentId, and that ID must be unique within the reconciliation input. One durable intent matching multiple broker records, two durable intents matching one broker record, or conflicting payloads for one exact execution ID is an identity conflict. Every execution must attach through the same ordered account/permId/orderRef/API-identity checks to exactly one reconciled durable order. Stop readiness and investigate; do not select the “closest” record.

Classify each durable outstanding order as:

  • open: one current open record matches;
  • completed: one current-day completed record matches, with executions reconciled separately;
  • unknown: absent from all bounded surfaces or outside their retention/visibility scope;
  • conflict: ambiguous identity or incompatible account/order data.

Unknown is not cancelled, rejected, or safe to resubmit. A transmitted order can be working outside the current visibility window, and a fill can have occurred while the application was offline. Reconcile through broader controlled surfaces or operator review; never call placeOrder solely because an intent is absent.

Minimal paper-account collection

This example performs no order placement, modification, or cancellation. It deliberately excludes completed orders and executions from startup so those bounded snapshots can be collected serially with explicit ownership. Use a nonzero client ID to avoid binding manual TWS orders.

import asyncio
from collections.abc import Awaitable
from typing import Any

from ib_async import ExecutionFilter, IB, StartupFetch


async def bounded(ib: IB, request: Awaitable[Any], timeout: float) -> Any:
future = asyncio.ensure_future(request)
try:
return await asyncio.wait_for(asyncio.shield(future), timeout=timeout)
except BaseException:
future.cancel()
ib.disconnect()
raise


async def main() -> None:
ib = IB()
startup = (
StartupFetch.POSITIONS
| StartupFetch.ORDERS_OPEN
| StartupFetch.ACCOUNT_UPDATES
| StartupFetch.SUB_ACCOUNT_UPDATES
)
try:
await ib.connectAsync(
host="127.0.0.1",
port=4002,
clientId=21,
timeout=10,
readonly=False,
raiseSyncErrors=True,
fetchFields=startup,
)
accounts = frozenset(ib.managedAccounts())
if accounts != frozenset({"DU1234567"}):
raise PermissionError("paper account scope changed")

open_trades = tuple(ib.openTrades())
completed = await bounded(ib, ib.reqCompletedOrdersAsync(True), 10)
fills = await bounded(
ib,
ib.reqExecutionsAsync(ExecutionFilter(acctCode="DU1234567")),
10,
)
print(len(open_trades), len(completed), len(fills))
finally:
ib.disconnect()


asyncio.run(main())

readonly=False permits order-state synchronization; this example still sends no order command. For client ID 0, connectAsync automatically requests manual-order binding, so use that identity only under an explicit binding policy.

Production readiness gate

The collection layer should first pass each fill through the durable execution/commission ledger described in Record fills and commissions. The gate below then reconciles stable order identities, exact execution IDs, and subscription replay. It persists a blocked or ready decision before changing readiness.

from collections.abc import Callable
from copy import deepcopy
from typing import Any

from ib_async import Fill, Trade


Record = dict[str, Any]


class ReconciliationBlocked(RuntimeError):
pass


class ReconciliationGate:
def __init__(
self,
authorized_accounts: frozenset[str],
expected_subscription_keys: frozenset[str],
persist: Callable[[Record], None],
) -> None:
if not authorized_accounts:
raise ValueError("authorized_accounts is required")
self.authorized_accounts = authorized_accounts
self.expected_subscription_keys = expected_subscription_keys
self.persist = persist
self.ready = False
self.poisoned = False
self.recovery_required = True

@staticmethod
def _broker_order(trade: Trade, surface: str) -> Record:
order = trade.order
return {
"surface": surface,
"account": order.account,
"orderRef": order.orderRef,
"permId": order.permId,
"clientId": order.clientId,
"orderId": order.orderId,
"apiIdentityAssigned": True,
"status": trade.orderStatus.status,
}

@staticmethod
def _validated_identity(order: Record) -> tuple[Record | None, str | None]:
account = order.get("account")
if not isinstance(account, str) or not account:
return None, "missing-account"
perm_id = order.get("permId", 0)
if not isinstance(perm_id, int) or isinstance(perm_id, bool):
return None, "invalid-perm-id"
order_ref = order.get("orderRef", "")
if not isinstance(order_ref, str):
return None, "invalid-order-ref"
api_assigned = order.get("apiIdentityAssigned") is True
client_id = order.get("clientId")
order_id = order.get("orderId")
if api_assigned and (
not isinstance(client_id, int)
or isinstance(client_id, bool)
or not isinstance(order_id, int)
or isinstance(order_id, bool)
):
return None, "invalid-api-identity"
if perm_id <= 0 and not order_ref and not api_assigned:
return None, "missing-stable-identity"
return {
"account": account,
"permId": perm_id,
"orderRef": order_ref,
"apiIdentityAssigned": api_assigned,
"clientId": client_id,
"orderId": order_id,
}, None

@staticmethod
def _order_candidates(identity: Record, broker_orders: list[Record]) -> tuple[set[int], bool]:
account = identity["account"]
same_account = {
index
for index, broker in enumerate(broker_orders)
if broker["account"] == account
}
perm_id = identity["permId"]
order_ref = identity["orderRef"]
api_assigned = identity["apiIdentityAssigned"]
by_ref = {
index
for index in same_account
if order_ref and broker_orders[index]["orderRef"] == order_ref
}
by_api = {
index
for index in same_account
if api_assigned
and broker_orders[index]["clientId"] == identity["clientId"]
and broker_orders[index]["orderId"] == identity["orderId"]
}
if perm_id > 0:
primary = {
index
for index in same_account
if broker_orders[index]["permId"] == perm_id
}
secondary = by_ref | by_api
elif order_ref:
primary = by_ref
secondary = by_api
else:
primary = by_api
secondary = set()

conflict = bool(secondary - primary)
for index in primary:
broker = broker_orders[index]
if order_ref and broker["orderRef"] != order_ref:
conflict = True
if api_assigned and (
broker["clientId"] != identity["clientId"]
or broker["orderId"] != identity["orderId"]
):
conflict = True
return primary, conflict

@staticmethod
def _execution_records(fills: list[Fill]) -> tuple[list[Record], list[str]]:
by_id: dict[str, Record] = {}
conflicts: list[str] = []
for fill in fills:
execution = fill.execution
record: Record = {
"execId": execution.execId,
"account": execution.acctNumber,
"permId": execution.permId,
"clientId": execution.clientId,
"orderId": execution.orderId,
"orderRef": execution.orderRef,
"shares": float(execution.shares),
"price": float(execution.price),
}
previous = by_id.get(execution.execId)
if not execution.execId:
conflicts.append("missing-exec-id")
elif previous is not None and previous != record:
conflicts.append(f"conflicting-exec-id:{execution.execId}")
else:
by_id[execution.execId] = record
return list(by_id.values()), conflicts

def reconcile(
self,
*,
connection_epoch: str,
reconciliation_run_id: str,
client_id: int,
accessible_accounts: frozenset[str],
durable_orders: list[Record],
open_trades: list[Trade],
completed_trades: list[Trade],
fills: list[Fill],
commission_complete_exec_ids: frozenset[str],
subscription_outcomes: list[Record],
) -> Record:
if self.poisoned:
raise RuntimeError("reconciliation gate is poisoned; replace it")
self.ready = False
self.recovery_required = True
if not connection_epoch or not reconciliation_run_id or client_id < 0:
raise ValueError("epoch, run ID, and non-negative client ID are required")

blockers: list[str] = []
if accessible_accounts != self.authorized_accounts:
blockers.append("account-scope-mismatch")

broker_orders = [
self._broker_order(trade, "open") for trade in open_trades
] + [
self._broker_order(trade, "completed") for trade in completed_trades
]
if any(
order["account"] not in self.authorized_accounts
for order in broker_orders
):
blockers.append("broker-order-account-out-of-scope")

matched_broker: dict[int, int] = {}
seen_intent_ids: set[str] = set()
order_outcomes: list[Record] = []
for durable_index, durable in enumerate(durable_orders):
intent_id = durable.get("intentId")
if not isinstance(intent_id, str) or not intent_id:
blockers.append(f"invalid-durable-intent-id:{durable_index}")
continue
if intent_id in seen_intent_ids:
blockers.append(f"duplicate-durable-intent-id:{durable_index}")
continue
seen_intent_ids.add(intent_id)
identity, identity_error = self._validated_identity(durable)
if identity_error is not None or identity is None:
blockers.append(
f"invalid-durable-order:{durable_index}:{identity_error}"
)
continue
if identity["account"] not in self.authorized_accounts:
blockers.append(f"durable-order-account-out-of-scope:{durable_index}")
continue
matches, identity_conflict = self._order_candidates(
identity, broker_orders
)
if identity_conflict:
blockers.append(f"conflicting-durable-order:{durable_index}")
continue
if len(matches) == 0:
blockers.append(f"unknown-durable-order:{durable_index}")
continue
if len(matches) != 1:
blockers.append(f"ambiguous-durable-order:{durable_index}")
continue
broker_index = next(iter(matches))
if broker_index in matched_broker:
blockers.append(f"duplicate-durable-match:{durable_index}")
continue
matched_broker[broker_index] = durable_index
order_outcomes.append(
{
"intentId": intent_id,
"outcome": broker_orders[broker_index]["surface"],
"broker": broker_orders[broker_index],
}
)

for broker_index in range(len(broker_orders)):
if broker_index not in matched_broker:
blockers.append(f"unclassified-broker-order:{broker_index}")

execution_records, execution_conflicts = self._execution_records(fills)
blockers.extend(execution_conflicts)
for record in execution_records:
account = record["account"]
perm_id = record["permId"]
by_api = {
index
for index, broker in enumerate(broker_orders)
if broker["account"] == account
and broker["clientId"] == record["clientId"]
and broker["orderId"] == record["orderId"]
}
order_ref = record["orderRef"]
by_ref = {
index
for index, broker in enumerate(broker_orders)
if order_ref
and broker["account"] == account
and broker["orderRef"] == order_ref
}
if perm_id > 0:
matches = {
index
for index, broker in enumerate(broker_orders)
if broker["account"] == account
and broker["permId"] == perm_id
}
conflict = by_api != matches or (
bool(order_ref) and by_ref != matches
)
elif order_ref:
matches = by_ref
conflict = by_api != matches
else:
matches = by_api
conflict = False
exec_id = record["execId"] or "missing"
if conflict:
blockers.append(f"conflicting-execution-identity:{exec_id}")
continue
if len(matches) != 1:
classification = "unattached" if not matches else "ambiguous"
blockers.append(f"{classification}-execution:{exec_id}")
continue
broker_index = next(iter(matches))
durable_index = matched_broker.get(broker_index)
if durable_index is None:
blockers.append(f"unattached-execution:{exec_id}")
continue
record["attachedIntentId"] = durable_orders[durable_index].get(
"intentId"
)
record["attachedBrokerSurface"] = broker_orders[broker_index][
"surface"
]
execution_ids = {record["execId"] for record in execution_records}
missing_commissions = execution_ids - commission_complete_exec_ids
blockers.extend(
f"commission-incomplete:{exec_id}" for exec_id in sorted(missing_commissions)
)
if any(
record["account"] not in self.authorized_accounts
for record in execution_records
):
blockers.append("execution-account-out-of-scope")

subscriptions = {
str(outcome.get("ownerKey", "")): outcome
for outcome in subscription_outcomes
}
if (
"" in subscriptions
or len(subscriptions) != len(subscription_outcomes)
or set(subscriptions) != set(self.expected_subscription_keys)
or any(
outcome.get("state") != "ready"
for outcome in subscriptions.values()
)
):
blockers.append("subscription-replay-incomplete")

record: Record = {
"kind": "reconciliation-blocked" if blockers else "reconciliation-ready",
"connectionEpoch": connection_epoch,
"reconciliationRunId": reconciliation_run_id,
"clientId": client_id,
"accessibleAccounts": sorted(accessible_accounts),
"authorizedAccounts": sorted(self.authorized_accounts),
"durableOrders": deepcopy(durable_orders),
"orderOutcomes": order_outcomes,
"brokerOrders": broker_orders,
"executions": execution_records,
"commissionCompleteExecIds": sorted(commission_complete_exec_ids),
"subscriptionOutcomes": deepcopy(subscription_outcomes),
"blockers": sorted(set(blockers)),
}
try:
self.persist(record)
except BaseException:
self.poisoned = True
raise
if blockers:
raise ReconciliationBlocked("; ".join(sorted(set(blockers))))
self.ready = True
self.recovery_required = False
return record

This strict gate assumes every broker order visible to this connection is represented in the durable control plane. A system intentionally sharing a master-client view must replace that rule with an explicit, persisted ownership classification; it must not silently ignore unmatched records.

1101, 1102, and new local epochs

  • On 1101, IBKR says market-data requests were lost and must be resubmitted. Reconcile critical order state as well.
  • On 1102, IBKR says market-data requests were recovered; on the same local socket, do not blindly duplicate them. Still verify application order invariants.
  • After any local disconnect/reconnect, pinned wrapper subscription maps are new regardless of the last upstream code. Replay every application-owned subscription from its durable logical specification and wait for its domain-specific readiness signal.

Do not make a farm-restored message, a successful TCP connect, or a nonempty order snapshot the global ready signal. Readiness is the conjunction of exact account scope, accepted order/execution reconciliation, complete required commissions, subscription replay, and a durable ready decision for the current epoch.

Source provenance