Forged facial video detection framework based on multi-region temporal relationship feature
文献类型：期刊论文
| 作者 | Fang, Xing3; Hao, YanNi1; Luo, Yin1,2 ; Xu, Nan1,2; Cao, Jia1,2
|
| 刊名 | AIP ADVANCES
 |
| 出版日期 | 2023-08-01 |
| 卷号 | 13期号:8页码:8 |
| DOI | 10.1063/5.0125032 |
| 通讯作者 | Fang, Xing(fangxing@tju.edu.cn) |
| 英文摘要 | Face generation and manipulation techniques based on deep learning have enabled the creation of sophisticated forged facial videos, which are indistinguishable by human eyes. However, the illegal use of deep fake technology will have a serious impact on social stability, personal reputation, and even national security. Therefore, the detection technology of fake facial videos is of great significance to protect national security and maintain social order. Although the existing video-based fake face video detection technology has achieved good detection performance on the public fake face video database, there are still the following problems: (1) the existing technology uses a 2D attention mechanism to obtain local region features from face images and lacks a 3D attention mechanism to obtain local area features from face videos; (2) after obtaining local area features, the existing technology is directly used to classify or only model the inter-regional relationship of images without modeling the temporal relationship between regions of the video. This paper proposes a fake facial video detection framework based on multi-region temporal relationship features, including designing a three-dimensional attention mechanism to extract local features of multiple regions of the face from the video. In order to model the time series relationship between different face areas, a time series graph convolution neural network is also introduced to extract the time series relationship features between multiple areas. In order to model the time sequence relationship between different face regions, the convolution neural network of a time sequence diagram is also introduced to extract the characteristics of the time sequence relationship between multiple regions. Through the change characteristics of the time sequence relationship between face regions, the timing inconsistency of the face video is detected so as to determine whether the face has been deeply forged. Through experiments on multiple datasets, the experimental results of the model accuracy test show that the method proposed in the present invention achieves the highest detection accuracy, and the accuracy in the FaceForensics++ (low definition) dataset is 18.19% higher than that of the benchmark method. The experimental results of the generalization ability test show that the method proposed in the present invention achieves the highest generalization performance, and the detection accuracy in the Celeb-DF dataset is 11.92% higher than that of the benchmark method. |
| 资助项目 | National Key Research and Development Plan Science and Technology Winter Olympics Key Project: Research and Application Demonstration of the Global Communication Service Platform for the Winter Olympics[2020YFF0305300] |
| WOS研究方向 | Science & Technology - Other Topics ; Materials Science ; Physics |
| 语种 | 英语 |
| 出版者 | AIP Publishing |
| WOS记录号 | WOS:001052966400001 |
| 资助机构 | National Key Research and Development Plan Science and Technology Winter Olympics Key Project: Research and Application Demonstration of the Global Communication Service Platform for the Winter Olympics |
| 源URL | [http://ir.ia.ac.cn/handle/173211/54089]  |
| 专题 | 舆论大数据科学与技术应用联合实验室 |
| 通讯作者 | Fang, Xing |
| 作者单位 | 1.Beijing Wenge Technol Co Ltd, Beijing 100000, Peoples R China 2.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 3.Tianjin Univ, Sch Management, Tianjin 300072, Peoples R China |
| 推荐引用方式 GB/T 7714 | Fang, Xing,Hao, YanNi,Luo, Yin,et al. Forged facial video detection framework based on multi-region temporal relationship feature[J]. AIP ADVANCES,2023,13(8):8. |
| APA | Fang, Xing,Hao, YanNi,Luo, Yin,Xu, Nan,&Cao, Jia.(2023).Forged facial video detection framework based on multi-region temporal relationship feature.AIP ADVANCES,13(8),8. |
| MLA | Fang, Xing,et al."Forged facial video detection framework based on multi-region temporal relationship feature".AIP ADVANCES 13.8(2023):8. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。