ImageNet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever et al.
2.1k
Citations
0
Influential Citations
IEEE Transactions on Information Theory
Venue
2001
Year
We consider the problem of embedding one signal (e.g., a digital watermark), within another "host" signal to form a third, "composite" signal. The embedding is designed to achieve efficient tradeoffs among the three conflicting goals of maximizing the information-embedding rate, minimizing the distortion between the host signal and composite signal, and maximizing the robustness of the embedding. We introduce new classes of embedding methods, termed quantization index modulation (QIM) and distortion-compensated QIM (DC-QIM), and develop convenient realizations in the form of what we refer to as dither modulation. Using deterministic models to evaluate digital watermarking methods, we show that QIM is "provably good" against arbitrary bounded and fully informed attacks, which arise in several copyright applications, and in particular it achieves provably better rate distortion-robustness tradeoffs than currently popular spread-spectrum and low-bit(s) modulation methods. Furthermore, we show that for some important classes of probabilistic models, DC-QIM is optimal (capacity-achieving) and regular QIM is near-optimal. These include both additive white Gaussian noise (AWGN) channels, which may be good models for hybrid transmission applications such as digital audio broadcasting, and mean-square-error-constrained attack channels that model private-key watermarking applications.
This paper addresses a fundamental challenge in digital watermarking: simultaneously maximizing embedding rate, minimizing distortion, and maximizing robustness. Prior methods like spread-spectrum and low-bit modulation suffered from suboptimal tradeoffs. By introducing quantization index modulation (QIM) and its distortion-compensated variant (DC-QIM), the authors provided a provably good framework that outperforms existing techniques. The work is highly cited (2092 citations) and remains a cornerstone in information embedding, influencing both theoretical research and practical watermarking systems.
The paper's significance extends beyond watermarking to broader signal processing and communications. The concept of using structured quantization to embed information is elegant and generalizable, with applications in data hiding, steganography, and even some forms of machine learning (e.g., adversarial perturbations). The provable optimality under certain channel models (AWGN, MSE-constrained attacks) gives practitioners confidence in deploying these methods.
The paper provides theoretical comparisons rather than experimental results on specific datasets. Key findings include:
This paper fundamentally changed the landscape of digital watermarking by introducing a principled, provably good approach. It provided a theoretical foundation that later work built upon, including extensions to dirty-paper coding and Costa's scheme. The ideas have been applied in audio, image, and video watermarking, as well as in steganography and data hiding. For AI practitioners, the concept of embedding information via structured quantization has parallels in adversarial robustness and model watermarking. The paper remains essential reading for anyone working on information embedding or secure communications.
Alex Krizhevsky, Ilya Sutskever et al.
Ashish Vaswani, Noam Shazeer et al.
Douglas M. Bates, Martin Mächler et al.
Diederik P. Kingma, Jimmy Ba