BitTP: The Lightweight Trajectory Prediction Model with BitLLM for Edge-Devices

Abstract

Trajectory prediction is a fundamental task for autonomous systems, requiring complex reasoning about multi-agent interactions and intents. Large language models (LLMs) have recently been adopted for this task, as they provide strong contextual reasoning and interpretable, language-based trajectory representations. However, these LLM-based predictors are extremely memory- and compute-intensive, making them difficult to deploy on resource-constrained edge devices such as on-board computers in autonomous robots. To bridge this gap, we propose BitTP, which converts an LLM-based trajectory predictor into a lightweight bitlinear architecture. We demonstrate that weight-only quantization to 1.58-bit (BitTP-Weight) is optimal. Crucially, activations must remain in full precision, as quantizing them leads to severe degradation and instability in spatio-temporal reasoning. Empirically, BitTP-Weight not only preserves but improves prediction quality over the full-precision (BF16) LLM baseline, reducing ADE by 14.29% and FDE by 20.97% on average, while simultaneously reducing memory usage and inference latency relative to other quantization methods. These results demonstrate that carefully designed quantization acts as an effective regularizer, enabling the practical deployment of sophisticated LLM-based reasoning on edge devices.

Method

Figure 2: BitTP architecture and the three selective quantization strategies. All nn.Linear layers in the T5-small encoder–decoder are dynamically replaced by BitLinear modules: (a) Both quantizes 8-bit activations and 1.58-bit weights, (b) Activ quantizes activations only, and (c) Weight quantizes weights only.

BitTP turns a language-based trajectory predictor into a lightweight bitlinear model. Built on a T5-small sequence-to-sequence backbone, every nn.Linear layer (97 logical, 73 unique) is swapped for a custom BitLinear module. We study three selective strategies and find that quantizing weights only to ternary 1.58-bit {-1, 0, 1}, while keeping activations in full precision (BitTP-Weight), gives the best accuracy–efficiency trade-off. Quantizing activations instead severely destabilizes spatio-temporal reasoning, confirming that activation precision is critical in encoder–decoder models.

Key Results

On the ETH/UCY benchmark, BitTP-Weight not only matches but improves over the full-precision (BF16) baseline — reducing ADE by 14.29% and FDE by 20.97% — while cutting memory and inference cost. Quantizing activations (Both / Activ) instead collapses accuracy, showing that weight-only quantization acts as an effective regularizer.

Average ADE/FDE (meters) over the five ETH/UCY scenes. Complexity reported relative to BitTP-Both (100%).

CPU Demo

BitTP-Weight runs in real time on CPU-only edge devices — no GPU acceleration required.

BibTeX

@InProceedings{Kang_2026_CVPR,
    author    = {Kang, Mincheol and Lim, HyunJin and Kang, Bomin and Park, Daehee},
    title     = {BitTP: The Lightweight Trajectory Prediction Model with BitLLM for Edge-Devices},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings},
    month     = {June},
    year      = {2026},
    pages     = {3470-3480}
}