A Robustness-Assured White-Box Watermark in Neural Networks

• Published in: IEEE transactions on dependable and secure computing Vol. 20; no. 6; pp. 5214 - 5229
• Main Authors: Lv, Peizhuo, Li, Pan, Zhang, Shengzhi, Chen, Kai, Liang, Ruigang, Ma, Hualong, Zhao, Yue, Li, Yingjiu
• Format: Journal Article
• Language: English
• Published: Washington IEEE 01.11.2023; IEEE Computer Society
• Subjects: Artificial neural networks; Biological system modeling; Commercialization; Computational modeling; Deep learning models; Embedded systems; Equivalence; Glass box; Intellectual property; IP networks; Machine learning; Neural networks; Ownership; Pruning; Recurrent neural networks; Robustness; Training; watermark; Watermarking
• ISSN: 1545-5971; 1941-0018
• DOI: 10.1109/TDSC.2023.3242737

Recently, stealing highly-valuable and large-scale deep neural network (DNN) models becomes pervasive. The stolen models may be re-commercialized, e.g., deployed in embedded devices, released in model markets, utilized in competitions, etc, which infringes the Intellectual Property (IP) of the original owner. Detecting IP infringement of the stolen models is quite challenging, even with the white-box access to them in the above scenarios, since they may have experienced fine-tuning, pruning, functionality-equivalent adjustment to destruct any embedded watermark. Furthermore, the adversaries may also attempt to extract the embedded watermark or forge a similar watermark to falsely claim ownership. In this article, we propose a novel DNN watermarking solution, named <inline-formula><tex-math notation="LaTeX">HufuNet</tex-math> <mml:math><mml:mrow><mml:mi>H</mml:mi><mml:mi>u</mml:mi><mml:mi>f</mml:mi><mml:mi>u</mml:mi><mml:mi>N</mml:mi><mml:mi>e</mml:mi><mml:mi>t</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="peizhuo-ieq1-3242737.gif"/> </inline-formula>, to detect IP infringement of DNN models against the above mentioned attacks. Furthermore, HufuNet is the first one theoretically proved to guarantee robustness against fine-tuning attacks. We evaluate HufuNet rigorously on four benchmark datasets with five popular DNN models, including convolutional neural network (CNN) and recurrent neural network (RNN). The experiments and analysis demonstrate that HufuNet is highly robust against model fine-tuning/pruning, transfer learning, kernels cutoff/supplement, functionality-equivalent attacks and fraudulent ownership claims, thus highly promising to protect large-scale DNN models in the real world.