Archived

This content is available here for research, reference, and/or recordkeeping.

Author ORCID Identifier

https://orcid.org/0000-0003-0024-6943

Date Available

6-9-2026

Year of Publication

2026

Document Type

Doctoral Dissertation

Degree Name

Doctor of Philosophy (PhD)

College

Engineering

Department/School/Program

Electrical and Computer Engineering

Faculty

Sen-ching Samson Cheung

Faculty

Daniel Lau

Abstract

Fine-grained Temporal Action Segmentation (TAS) has become a cornerstone of video understanding, offering dense frame-level predictions essential for clinical assessment, surgical skill evaluation, and human-computer interaction. While TAS methods have delivered strong results on coarse-grained benchmarks, two fundamental challenges persist: (1) global attention mechanisms dilute boundary information critical for subsecond precision, a phenomenon we term the temporal granularity bottleneck, and (2) dense frame-level annotation remains prohibitively expensive, with most datasets requiring exhaustive labeling of lengthy untrimmed videos. These challenges are particularly pronounced in medical domains, where sub-second primitives define clinical outcomes while expert annotation remains scarce. In this dissertation, we address both limitations through principled windowed approaches to attention design and annotation strategy. Central to our approach is the observation that temporal windowed design, rather than architectural depth, governs sub-second precision. This insight emerges from a research trajectory spanning clinical behavior analysis in autism spectrum disorder screening, transformer-based behavior detection in infant videos, and fine-grained action segmentation for stroke rehabilitation. To address the granularity bottleneck, we introduce Multi Membership Temporal Attention (MMTA), a multi-membership attention operator applying bounded SoftMax normalization within overlapping local windows. Unlike prior windowed approaches, MMTA allows each frame to belong to multiple windows simultaneously, producing distinct locally normalized updates resolved through explicit aggregation, enabling emergent global context while preserving sub-second precision. On the annotation side, we propose Boundary-Centric Active Learning (B-ACT) for TAS, which concentrates labeling effort on action transitions using uncertainty estimation and a composite boundary scoring function capturing local confusion, classification ambiguity, and temporal dynamics. Our contributions aim to clarify how temporal receptive fields and annotation allocation shape both segmentation precision and labeling efficiency. Extensive experiments across clinical domains, including ASD behavioral analysis and stroke rehabilitation, and public activity benchmarks demonstrate that matching window parameters to action-specific durations, along with strategic boundary-focused annotation provide principled foundations for efficient fine-grained temporal action understanding.

Digital Object Identifier (DOI)

https://doi.org/10.13023/etd.2026.320

Archival?

Archival

Funding Information

Research reported in this thesis was supported by the National Institute of Mental Health of the National Institutes of Health under Award Number R01MH121344-01

Research reported in this publication was also supported by Igniting Research Collaborations (IRC) at the University of Kentucky.

Download

Share

COinS