\section{Experimental and implementation details}\label{app:details}

\subsection{LifeGrid construction}

For each environment seed, we draw fixed base tables for action reward, event reward, transition logits, and behavior logits. The previous event and current depth index the tables; the represented state remains the complete history. Event effects are centered across events before scaling. The four cells are then formed only by
\[
T_\mu\in\{0.35,2.50\},\qquad d_r\in\{0.15,1.10\}.
\]
The event law is $\softmax(\ell/T_\mu)$, and $d_r$ scales only event-dependent reward and continuation features. This construction guarantees that dispersion does not change the observational sequence distribution, while entropy does not change reward tables.

\paragraph{Chance-corrected readout algorithm.}
Given $\hat\mu,\hat r,\pi_{\refp},\tau$, set $\hat V=0$ at terminal states and sweep backward from $K$ to 1. At each $s,a$, compute
\[
\hat W(s\oplus a)=\Exp_{e\sim\hat\mu}[\hat r(s\oplus a,e)+\hat V(s\oplus a\oplus e)],\quad
\hat Q(s,a)=\hat r(s,a)+\hat W(s\oplus a).
\]
Then set $\hat V(s)=\tau\log\Exp_{a\sim\pi_{\refp}}\exp\{\hat Q(s,a)/\tau\}$ and
$\hat\pi(a\mid s)=\pi_{\refp}(a\mid s)\exp\{[\hat Q(s,a)-\hat V(s)]/\tau\}$.
The naive generative readout changes only the first update, replacing the expectation with $\tau\log\Exp_{\hat\mu}\exp\{(\hat r+\hat V)/\tau\}$.

\subsection{Tabular estimator}

The behavior and event conditionals use symmetric Dirichlet smoothing $\alpha=0.5$. Reward estimates are conditional sample means with a zero prior. Independent zero-mean Gaussian noise with standard deviation $0.10$ is added to each observed reward. A chance node is declared supported at threshold $c$ if its state--action visit count is at least $c$, and the corresponding action is admitted at that state. If no action passes, the implementation retains the most visited action to keep the recursion defined. The reported raw supported mass counts only nodes that actually pass the threshold; the true value loss of the resulting fallback-completed policy class is reported as the coverage-restriction cost.

\subsection{Transformer estimator}

The vocabulary contains BOS, four typed action tokens, three typed event tokens, and PAD. Sequences contain eight predicted tokens. A parity mask prevents event tokens at action positions and action tokens at event positions. The network has two pre-norm Transformer encoder layers, width 32, four attention heads, GELU feed-forward width 128, learned positional embeddings, and no dropout. AdamW uses learning rate $3\times10^{-3}$, weight decay $10^{-4}$, gradient clipping at 1, batch size 512, and 1,200 updates. All prefix conditionals are extracted exactly after training.

\subsection{Metrics and uncertainty}

Trajectory KL is computed by summing over all $20{,}736$ paths. Conditional TV is reported both as observational-occupancy weighted error and as a maximum on declared supported nodes. Policy regret always evaluates the extracted policy in the true environment under \eqref{eq:objective}. Figure intervals use 4,000 nonparametric bootstrap resamples with a fixed reporting seed. Reproducibility seeds are functions only of the environment, data, training replicate, and pre-declared sample size.

\subsection{Software checks}

The exact and tabular suites use NumPy float64. Neural training uses PyTorch on CPU; exact post-training evaluation returns to NumPy float64. Unit tests independently check each identity, the support-gated bound, and estimator normalization. The complete command is \texttt{bash scripts/run\_full.sh}.
