基于时空注意力机制的群组行为识别方法研究

全文下载： 202303015.pdf

文章编号： 1672-6987（2023）03-0118-09； DOI： 10.16351/j.1672-6987.2023.03.015

王传旭， 于潇媛（青岛科技大学 信息科学技术学院， 山东 青岛 266061）

摘要： 视频中的群组行为识别是一项具有挑战性的任务，主要面临两个难点，一是如何构建群组成员间的交互关系；二是如何从时序上优选区别性强的时空特征构建简约的行为描述符。本工作提出的基于时空注意力机制建模框架，旨在解决这两个问题。对于前者，将群组成员间交互关系演变由图注意力网络（graph attention networks，GAT）中节点连接图的更新迭代来描述；每个节点成员由外观特征、位置信息以及轨迹特征来描述；GAT内置注意力机制在迭代中分离出权重系数不等的节点，注意力系数越大的节点聚合的信息多，称为关键节点，仅由关键节点成员构建的交互关系即为简约交互关系。对于后者，提出将当前帧预分类出成员行为属性和群组行为类别的交合相似度系数（intersection similarity coefficient，ISC）系数作为当前帧的时序权重，来进一步约简上述交互关系旨在构建强区分性时空特征描述符，最后由softmax分类器，实现个人行为识别和群组行为识别。该算法在CAD（collective activity dataset）和Volleyball数据集上分别取得936%和938%平均识别率，并与其它算法比较验证了其有效性。

关键词： 群组行为识别； 关键人物建模； 关键时空特征描述； 图注意力网络；简约交互

关系建模； 交合相似度系数

中图分类号： TP 301.6文献标志码： A

引用格式： 王传旭， 于潇媛. 基于时空注意力机制的群组行为识别方法研究［J］. 青岛科技大学学报(自然科学版), 2023, 44(3): 118-126.

WANG Chuanxu, YU Xiaoyuan. Research on group behavior recognition based on spatio-temporal attention mechanism［J］. Journal of Qingdao University of Science and Technology（Natural Science Edition), 2023， 44（3）： 118-126.

Research on Group Behavior Recognition Based on

Spatio-temporal Attention Mechanism

WANG Chuanxu, YU Xiaoyuan

(College of Information Science and Technology, Qingdao University of Science and Technology, Qingdao 266061,China)

Abstract: Group behavior recognition is a challenging task, which mainly confronts two difficulties: the first is how to construct the interaction relationship between group members; the second is how to optimize the discriminative spatio-temporal features from time sequence to construct simple behavior descriptors. In this paper, a framework is proposed to solve these two problems. For the former, the evolution of the interaction relationship between group members is described by the iteration computation of the nodes in graph attention networks (GAT), where each node member is described with appearance characteristics, position information and trajectory characteristics; the built-in attention mechanism in GAT separates nodes with unequal weight coefficients via attention iterations, and the nodes with larger attention coefficients aggregate more information and are called key nodes. The interaction relationship constructed only by the key nodes is becoming concise. For the latter, this paper proposes to use the intersection similarity coefficient(ISC), which is calculated between individual behavior attributes and group behavior categories pre-classified at the current frame, the ISC is used as the temporal weight of the current frame to further simplifie spatio-temporal feature in the whole video. Finally, the condensed video descriptor driven by key members and key frames is input into softmax to recognize group/individual behavior. This algorithm achieves average recognition rates of 936% and 938% on CAD (collective activity dataset) and volleyball datasets respectively, and its effectiveness is verified by comparison with other algorithms.

Key words: group behavior recognition; key person modeling; description of key spatio-temporal features; graph attention network; concise interaction relationship modeling; intersection similarity coefficient

收稿日期： 2022-05-23

基金项目： 国家自然科学基金项目(61672305).

作者简介： 王传旭（1968—），男，教授.