\section{Introduction}

Next-token prediction and sequential decision making share a prefix structure. A current state summarizes a history, and each new symbol extends it. Recent work makes this connection exact: in function space, an ARM can represent an EBM, and the required future correction is a soft Bellman recursion \citep{blondel2025}. This is a representational statement, not a license to treat every fitted sequence model as an optimal policy.

The distinction becomes structural when trajectories alternate between choices and events. A person chooses an action, after which market conditions, health shocks, institutional changes, or other circumstances occur without being chosen. A sequence model may accurately fit the joint law of actions and events while answering the wrong question when its generation probabilities are interpreted as action quality. In particular, the standard EBM-to-ARM factorization exponentially tilts every next token. Doing so at an exogenous chance node acts as if favorable random transitions could be selected. Related optimism in probabilistic control is known \citep{levine2018}; what is missing is an exact account of how this mismatch appears and propagates inside the ARM--EBM construction.

We formalize an alternating action--event process and distinguish three objects:
\begin{enumerate}
    \item the observational law $\rho_\beh=\pi_\beh\times\mu$, learned by teacher forcing on supported prefixes;
    \item a reward-tilted \emph{generative} law, whose exact autoregressive factorization uses log-moments at both action and event positions; and
    \item a KL-regularized \emph{control} solution, which soft-maximizes controllable actions but averages exogenous events under the environment kernel $\mu$.
\end{enumerate}

The paper makes four principal technical contributions. First, we preserve exact ARM--EBM factorization under a reference measure that contains both an action reference $\pi_{\refp}$ and the event kernel $\mu$. Second, we derive a local-to-global recursion for the generative--control value gap. The recursion isolates an entropic premium created at a current chance node and a term propagated from future nodes. It also shows why transition entropy alone does not determine the gap: dispersion of continuation return is the relevant object. Third, we derive two audits from the same formalism. A relative-energy identity separates event typicality from action typicality, while a pathwise identity decomposes realized return into controllable soft advantages and mean-zero event innovations. Fourth, we give a finite-horizon uniform simulation bound and a support-gated regret corollary that separates estimation error from the opportunity cost of excluding unsupported actions.

The experiments are deliberately aligned with the claims. LifeGrid is a finite, fully enumerable world with $K=4$ rounds, four actions, three events, and $20{,}736$ complete trajectories. Exhaustive tests verify each equality to floating-point tolerance. A pre-registered $2\times2$ design changes transition entropy without changing reward dispersion, and changes event-dependent continuation dispersion without changing transition logits. Tabular and Transformer estimators then test the distinction between learning an observational kernel and choosing a readout. Synthetic evidence is not presented as evidence about human lives; it tests implementation, identification under stated assumptions, and qualitative predictions of the mathematics.

\paragraph{Scope of novelty.} We do not claim that stochastic control-as-inference optimism, entropic certainty equivalents, general ARM--EBM equivalence, or martingale reward decompositions are new. The narrower contribution is their joint localization in typed action--event sequences: an exact gap-propagation law, a reference-measure interpretation audit, and a support-aware route from a learned sequence law to a control readout.

\paragraph{Road map.} \Cref{sec:setup} defines the process and observational semantics. \Cref{sec:readouts} develops generative and control recursions. \Cref{sec:audit} gives the audit and attribution identities, and \Cref{sec:error} gives error and regret bounds. \Cref{sec:experiments} reports exact, tabular, and neural experiments.

