OnlineTAS: An Online Baseline for Temporal Action Segmentation

Authors

Qing Zhong^1,2, Guodong Ding², and Angela Yao²
¹University of Adelaide, ² National University of Singapore

Abstract

Temporal context plays a significant role in temporal action segmentation. In an offline setting, the context is typically captured by the segmentation network after observing the entire sequence. However, capturing and using such context information in an online setting remains an under-explored problem. This work presents the first online framework for temporal action segmentation. At the core of the framework is an adaptive memory designed to accommodate dynamic changes in context over time, alongside a feature augmentation module that enhances the frames with the memory. In addition, we propose a post-processing approach to mitigate the severe over-segmentation in the online setting. On three common segmentation benchmarks, our approach achieves state-of-the-art performance.

Resources

Files: [pdf]

Citation:

@inproceedings{zhong2024online,
bsp title = {OnlineTAS: An Online Baseline for Temporal Action Segmentation},
author = {Zhong, Qing and Ding, Guodong and Yao, Angela},
booktitle = {Advances in Neural Information Processing Systems},
year = {2024}
}