Tsukada Lab. Computer Network Group

Causal Scene Narration: Inference-Time Text Restructuring for Vision-Language-Action Driving

Yun Li, Yidu Zhang, Simon Thompson, Ehsan Javanmardi, Manabu Tsukada

Transactions on Machine Learning Research, Sep 20, 2026

Summary in three lines

  • Causal Scene Narration restructures text input to a driving model at inference time.
  • It explicitly connects the intended maneuver to constraints such as nearby hazards and expresses quantitative scene facts.
  • Closed-loop CARLA tests found higher driving scores, with ablations attributing part of the gain to causal structure.

Figures from the paper

Figure 2: Causal Scene Narration at a left turn
Figure 2. Causal Scene Narration at a left turnSource: Figure 2 from Li et al., Transactions on Machine Learning Research (2026) (authors’ figure from the manuscript)The example shows how scene facts and maneuver constraints become causally linked instructions for the driving model.

Abstract

Vision-Language-Action (VLA) models for autonomous driving receive navigation commands and hazard notices as disconnected text fragments, forcing the model to discover on its own which environmental constraints are relevant to the current maneuver. We introduce Causal Scene Narration (CSN), which restructures VLA text inputs through intent-constraint alignment, quantitative grounding, and structured separation, at inference time with zero GPU cost. We additionally evaluate runtime supervision and training-time preference alignment as experimental conditions rather than as contributions. A multi-town CARLA evaluation (16 routes, 8 towns

How to cite

Y. Li, Y. Zhang, S. Thompson, E. Javanmardi, and M. Tsukada, “Causal Scene Narration: Inference-Time Text Restructuring for Vision-Language-Action Driving,” Transactions on Machine Learning Research, 2026.

BibTeX

@article{Li2026g,
  title         = {Causal Scene Narration: Inference-Time Text Restructuring for Vision-Language-Action Driving},
  author        = {Yun Li and Yidu Zhang and Simon Thompson and Ehsan Javanmardi and Manabu Tsukada},
  url           = {https://arxiv.org/abs/2604.01723},
  year          = {2026},
  date          = {2026-09-20},
  urldate       = {2026-09-20},
  journal       = {Transactions on Machine Learning Research},
  abstract      = {Vision-Language-Action (VLA) models for autonomous driving receive navigation commands and hazard notices as disconnected text fragments, forcing the model to discover on its own which environmental constraints are relevant to the current maneuver. We introduce Causal Scene Narration (CSN), which restructures VLA text inputs through intent-constraint alignment, quantitative grounding, and structured separation, at inference time with zero GPU cost. We additionally evaluate runtime supervision and training-time preference alignment as experimental conditions rather than as contributions. A multi-town CARLA evaluation (16 routes, 8 towns},
  keywords      = {Other},
  pubstate      = {published},
  tppubtype     = {article}
}