钱政华¹　林剑峰¹　潘星¹　陈辉¹　陈静霖¹　林宜轩¹　王玉琛²

（1.华电新能源集团股份有限公司福建分公司，福建  福州 350000

2.南京南自信息技术有限公司，江苏 南京 210032）

摘要：针对基于视频的人-物交互检测存在的鲁棒性不足问题，提出基于注视引导的复杂环境视频人-物交互检测模型。该模型利用基于特征相似度的头身匹配算法，提高了头身匹配准确率；设计特征融合模块，将图像、文本和注视热力图特征进行有效融合。在VidHOI数据集上，本文模型全部类的mAP比性能最好的对比模型HOIGF提高了0.84；在Action Genome数据集上，本文模型全部类的Rec@10比性能最好的对比模型HOIGF提高了0.7。该模型提升了复杂环境下视频人-物交互检测的准确性，具有较好的鲁棒性。

关键词：人-物交互检测；注视引导；特征融合；特征相似度匹配

中图分类号：TP391.41               文献标志码：A            文章编号：1674-2605(2026)01-0007-08

DOI：10.12475/aie.20260107                                   开放获取

Video Human-object Interaction Detection Model with Gaze Guidance for Complex Environments

QIAN Zhenghua¹ LIN Jianfeng¹ PAN Xing¹CHEN Hui¹

CHEN Jinglin¹LIN Yixuan¹ WANG Yuchen²

(1.Huadian New Energy Group Co., Ltd.,Fujian Branch, Fuzhou 350000, China

2.Nanjing Nanzi Information Technology Co., Ltd., Nanjing 210032, China)

Abstract: To address the robustness issues in video-based human-object interaction detection, a gaze-guided video human-object interaction detection model for complex environments is proposed. This model utilizes a head-body matching algorithm based on feature similarity to improve the accuracy of head-body matching. A feature fusion module is designed to effectively integrate image, text, and gaze heatmap features. On the VidHOI dataset, the proposed model achieved an improvement of 0.84 in mAP across all classes compared to the best-performing baseline model, HOIGF. On the Action Genome dataset, the proposed model achieved an improvement of 0.7 in Rec@10 across all classes compared to the best-performing baseline model, HOIGF. The model enhances the accuracy of video HOI detection in complex environments and demonstrates good robustness.

Keywords: human-object interaction recognition; gaze guidance; feature fusion; feature similarity matching