Tsukada Lab. Computer Network Group

NL2TL-Tree: Natural Language to Nested Temporal-Logic Trees for Robustness-Guided Flow-Matching Planning

Jiachen Gong, Wencan Mao, Ehsan Javanmardi, Yun Li, Quanxi Zhou, Manabu Tsukada

Companion of the 2026 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2026 ACM International Symposium on Wearable Computers (UbiComp/ISWC '26), Poster, Shanghai, China, 2026年10月11日

3行要約

  • NL2TL-Treeは自然言語のロボット指示を、追加学習なしで入れ子の信号時相論理の木に変換する。
  • 木構造で作用域を確認でき、微分可能な頑健性スコアで学習済みフローマッチングプランナを誘導する。
  • TurtleBot3のシミュレーションでは6課題すべてに成功し、A*による補修なしでは4課題だった。

アブストラクト(原文)

Letting non-expert users command robots in plain language is a long-standing goal of ubiquitous computing, but free-form instructions are ambiguous and give no guarantee that the resulting behavior is correct. We present NL2TL-Tree, an early-stage prototype exploring whether a nested Signal Temporal Logic (STL) tree can serve as an interpretable, checkable interface between natural-language robot instructions and motion planning. The pipeline is training-free (no model is trained or fine-tuned): a frozen LLM is few-shot-prompted to parse a smart-device task into the tree (human-readable and error-localizable to a single sub-node), and a frozen graph-encoded flow-matching planner (TeLoGraF), steered by a differentiable STL-robustness term, turns the tree into motion. Test-time mechanisms (operator normalization, nested-tree decomposition, a feasibility self-check that defers to a classical backend) extend the frozen planner, and an online robustness monitor can trigger a stop/re-plan; a planning-latency-aware deadline further debits the planner's own wall-clock from timed goals. We report preliminary evidence (a small parsing pilot and a TurtleBot3 Gazebo simulation) and outline an evaluation plan for parsing accuracy, failure localization, and user-facing interpretability. Code, prompts, and configs are released.

引用方法

J. Gong, W. Mao, E. Javanmardi, Y. Li, Q. Zhou, and M. Tsukada, “NL2TL-Tree: Natural Language to Nested Temporal-Logic Trees for Robustness-Guided Flow-Matching Planning,” in Companion of the 2026 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2026 ACM International Symposium on Wearable Computers (UbiComp/ISWC '26), Poster, Shanghai, China, 2026.

BibTeX

@misc{Gong2026,
  title         = {NL2TL-Tree: Natural Language to Nested Temporal-Logic Trees for Robustness-Guided Flow-Matching Planning},
  author        = {Jiachen Gong and Wencan Mao and Ehsan Javanmardi and Yun Li and Quanxi Zhou and Manabu Tsukada},
  year          = {2026},
  date          = {2026-10-11},
  urldate       = {2026-10-11},
  booktitle     = {Companion of the 2026 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2026 ACM International Symposium on Wearable Computers (UbiComp/ISWC '26), Poster},
  address       = {Shanghai, China},
  abstract      = {Letting non-expert users command robots in plain language is a long-standing goal of ubiquitous computing, but free-form instructions are ambiguous and give no guarantee that the resulting behavior is correct. We present NL2TL-Tree, an early-stage prototype exploring whether a nested Signal Temporal Logic (STL) tree can serve as an interpretable, checkable interface between natural-language robot instructions and motion planning. The pipeline is training-free (no model is trained or fine-tuned): a frozen LLM is few-shot-prompted to parse a smart-device task into the tree (human-readable and error-localizable to a single sub-node), and a frozen graph-encoded flow-matching planner (TeLoGraF), steered by a differentiable STL-robustness term, turns the tree into motion. Test-time mechanisms (operator normalization, nested-tree decomposition, a feasibility self-check that defers to a classical backend) extend the frozen planner, and an online robustness monitor can trigger a stop/re-plan; a planning-latency-aware deadline further debits the planner's own wall-clock from timed goals. We report preliminary evidence (a small parsing pilot and a TurtleBot3 Gazebo simulation) and outline an evaluation plan for parsing accuracy, failure localization, and user-facing interpretability. Code, prompts, and configs are released.},
  howpublished  = {Companion of the 2026 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2026 ACM International Symposium on Wearable Computers (UbiComp/ISWC '26), Poster},
  keywords      = {Other},
  pubstate      = {published},
  tppubtype     = {misc}
}