Authors:
Xinqi Su、Yawen Cui、Ajian Liu、Xun Lin、Yuhao Wang、Haochen Liang、Wenhui Li、Zitong Yu
Paper:
https://arxiv.org/abs/2408.10883
Introduction
In today’s digital age, the rapid spread of fake news across online social networks (OSNs) like Twitter and Weibo poses significant threats to society. The proliferation of fake news not only increases the information burden but can also incite panic and lead to severe societal consequences. To combat this, automated fake news detection (FND) has become a critical area of research. This paper introduces a novel approach called Dynamic Analysis and Adaptive Discriminator (DAAD) for detecting fake news, particularly focusing on multimodal fake news detection (MFND), which involves both text and images.
Related Work
Knowledge-based Fake News Detection
Knowledge-based methods leverage external knowledge sources such as knowledge graphs (KGs) and knowledge bases (KBs) to enrich news content and verify its authenticity. These methods capture consistency with external knowledge to distinguish between true and fake news. However, as fake news becomes more sophisticated, static KBs struggle to provide reliable factual signals. Recent approaches have started incorporating large language models (LLMs) to enhance fake news detection systems by providing supplementary information and reasoning capabilities.
Content-based Fake News Detection
Content-based methods focus on the semantic features of news content, including style, image manipulation, and image-text semantic consistency. These methods analyze elements such as sentiment, logic, and image manipulation to detect fake news. Despite significant performance improvements, these methods often rely on manually crafted detection models and apply the same model to all samples, limiting their effectiveness.
Research Methodology
The proposed DAAD approach aims to address the limitations of existing methods by introducing a dynamic and adaptive framework for fake news detection. The key components of DAAD include:
- Prompt Optimization: Utilizing the Monte Carlo Tree Search (MCTS) algorithm to optimize prompts for LLMs, enhancing their ability to generate insightful comments on news content.
- MemoryBank: Storing historical errors during the optimization process and providing global guidance to prevent the optimization from getting trapped in local minima.
- BatchPrompt: Expanding the prompt and sample spaces to reduce the risk of local minima.
- Adaptive Discriminator Modeling: Designing four discriminators to target different deceit patterns and using a soft-routing mechanism to explore the optimal detection model.
Experimental Design
Datasets
Experiments were conducted on three widely used datasets: Weibo, Weibo-21, and GossipCop. These datasets contain a mix of fake and real news samples, with varying degrees of balance between the two classes.
Implementation Details
The DAAD model was implemented using PyTorch and trained on an NVIDIA Tesla A100 GPU. Text and comment features were extracted using pre-trained BERT models, while image features were extracted using VGG-19. The MCTS algorithm was used for prompt optimization, and the adaptive discriminators were trained using AdaBelief optimizer.
Results and Analysis
Performance Comparison
The DAAD approach outperformed mainstream methods on all three datasets, achieving higher accuracy, precision, recall, and F1 scores. The improvements were particularly notable in the Weibo dataset, where DAAD achieved a 4.3% increase in precision and a 3.0% increase in F1 score for real news detection.
Ablation Study
An ablation study was conducted to evaluate the effectiveness of the proposed modules. The results demonstrated the importance of prompt optimization, the MemoryBank component, and the adaptive discriminators. Removing any of these components resulted in a significant drop in performance.
Qualitative Results
The routing paths for selected fake news cases were visualized to illustrate the role of each discriminator. For blatantly fabricated images, the model primarily activated the ReLU and frequency domain discriminators, while more complex news required a combination of different discriminators.
Overall Conclusion
The DAAD approach addresses the limitations of existing fake news detection methods by introducing a dynamic and adaptive framework. By leveraging optimized prompts, historical error feedback, and adaptive discriminators, DAAD achieves superior performance in detecting fake news. Future work could focus on automating the discovery of deception patterns and extending the methods to other areas such as sarcasm and harmful meme detection.
In summary, DAAD represents a significant advancement in the field of fake news detection, providing a flexible and robust solution to a pressing societal issue.
Code:
https://github.com/suxinqi/daad
