The growing interest in haptic applications suggests that haptic digital media will soon become widely available, and the need will arise to protect digital haptic data from misuse. In this article, we present our study and findings on psychophysical experiments regarding human abilities to perceive a digital watermark, or hidden signal, through a haptic interface. H aptic interfaces allow physical interactions with virtual 3D objects through the sense of touch. Possible applications include

more »

... ining for minimally invasive or microscopic surgical procedures, interacting with sculptures such as Michelangelo's David that we can't directly touch, perceptualizing multidimensional data sets such as earthquake simulations that we can't easily comprehend through visual displays alone, and assistance to sensory-impaired individuals by displaying visual and/or audio information through the haptic sensory channel. Because of the expected growing importance that digital haptic data will have in the near future, it's easy to predict that the need will soon arise to protect such data from misuse, like unauthorized copying and distribution, or false ownership claims. Among the available technologies to protect digital data, digital watermarking is receiving increased attention, thanks to its unique capability of persistently hiding a piece of information within the to-be-protected data. 1 We can use this hidden information to prove ownership, deny permission of copying the data, or detect tampering. In this article, we present the results of two psychophysical experiments that investigated the perceptibility and detectability of a hidden signal in the macro-and microgeometry of the virtual object's surface. In the first experiment, we embedded the watermark into a virtual surface's macrogeometry by modifying the underlying 3D model's wireframe. To begin with, we chose a flat surface so that signals related to the object's shape wouldn't inadvertently mask the watermark's detection. Nevertheless, we represented the surface with a 3D mesh so that we could readily extend this initial work to objects with arbitrary surface shapes. We modeled the watermark as an additive white noise superimposed on the host surface. The goal of the experiment was to estimate the noise intensity threshold as a function of the underlying mesh's resolution. Our second experiment focused on the microgeometry of object surfaces by embedding the watermark in the texture data. We used a simple one-dimensional sinusoidal model for both the watermark and the host signal. The goal of this experiment was to investigate whether existing detection threshold data 2,3 could successfully predict the perceptibility of the watermark. Despite the texture model's simplicity, this experiment provided the first evidence of the possibility of embedding a haptically imperceptible watermark that can later be detected via spectral analysis. Before going further into these experiments, we first provide a little background information regarding the basic issues and requirements for successful 3D watermarking techniques. (For further information, also see the "Overview of Watermarking Techniques" sidebar.) Basic issues and requirements In the past, a great deal of research has focused on digitally watermarking audio, images, and video-meanwhile, haptic interfaces are inherently related to 3D surfaces. Despite the fact that 3D models are widely used in several applications such as virtual prototyping, cultural heritage, and entertainment, watermarking 3D objects is still in its infancy. One of the reasons for this gap lies in the difficulty of extending common signal processing algorithms to 3D data. The first requirement that any watermarking technique must satisfy is watermark imperceptibility. In the case of still images and video sequences, the imperceptibility requirement has triggered a great deal of research about human visual systems, resulting in a number of possible algorithms that exploit the properties of human vision to improve watermark invisibility while keeping the watermark energy constant. 4 Recently, we've also seen some progress in 3D watermarking. 5 In this case, the watermark is hosted by the macrogeometry of the considered 84 1070-986X/07/$25.00 Feature Article virtual object's surface, which we can assume is represented by a 3D mesh. Accordingly, we can judge the watermark's intrusiveness in terms of its visibility in the mesh's rendered version. More generally, in applications where the virtual object is sensed through a haptic interface, guaranteeing the imperceptibility of the watermark requires characterizing the haptic channel's sensitivity. 85 Generally speaking, we can consider any watermarking system a communication system consisting of two major components: a watermark embedder and a watermark detector. The watermark usually consists of a pseudorandom sequence with uniform, binary, or Gaussian distribution. It's transmitted through the watermark embedder over the original, to-be-marked object (in our case, a 3D surface). The watermark detector extracts the watermark from the marked data. Intentional and unintentional attacks and distortions applied to the mesh hosting the watermark further characterize and complicate the transmission channel. We can divide watermarking techniques into two categories: ❚ spatial/temporal domain techniques that directly add the watermark to pixel values and ❚ transformed domain techniques that add the watermark in the frequency domain. Once we've chosen the host features, we need to specify the embedding rule. The most common approach to watermark embedding is the additive rule according to which y i ͌ x i ʳ ␥w i , where x i is the ith component of the original feature vector, w i the ith sample of the watermark, ␥ a parameter controlling the watermark strength, and y i the ith component of the watermarked feature vector. Recently, a new approach to watermark embedding has been proposed. This approach, commonly referred to as informed watermarking or Quantization Index Modulation (QIM) watermarking, 1 can greatly improve the system's performance as a whole. However, for the sake of simplicity, our analysis focused on additive watermarking, leaving the analysis of QIM schemes for future work. The way the watermark is extracted from data plays a crucial role. In blind decoding, the decoder doesn't need the original data (mesh) or any information derived from it to recover the watermark. Conversely, nonblind decoding refers to a situation where extraction is accomplished with the aid of the original, nonmarked data. We can also make an important distinction between algorithms embedding a mark that can be read and those inserting a code that can only be detected. In the former case, we can read the bits contained in the watermark without knowing them in advance. In the latter case, we can only verify the bits if a given code is present in the document. Though our perceptibility analysis was very general, we specifically focused on the case of blind watermark detection. As mentioned in the introduction, an important aspect of any watermarking system is the imperceptibility of the hidden information. For this reason it's of primary importance that we carefully study the properties of the sensory modality through which subjects detect the marked data. In audio watermarking, researchers have exploited existing data from studies on the human auditory system to better hide the watermarking signal within the host audio. More relevant to the 3D scenario is the case of still image watermarking. Several models of the human visual system have been modified and exploited to ensure the hidden signal's invisibility. In most cases, Watson's simple model of vision has been adopted, 2 leading to watermarking systems working in the discrete cosine transform (DCT) or discrete Fourier transform (DFT) domain. Watson's model is able to predict the visibility of a sinusoidal grating (watermarking signal) superimposed on another sinusoidal grating (host signal). One problem with visual watermarking in the frequency domain is the lack of spatial localization-hence, alternative models operating in the wavelet domain have been proposed that have led to improved watermark invisibility. As far as 3D meshes are concerned, few studies have been published so far. Of those studies, two different approaches have been taken: judging the watermark's visibility in selected views of the rendered mesh, and allowing the observer to freely play with the mesh by zooming and rotation. 3 Researchers still have much work to do to fully understand and successfully implement watermark visibility in 3D objects. To the best of our knowledge, no previous work on haptic watermark perceptibility has been presented, with the exception of the studies carried out by ourselves. For a more detailed discussion of watermarking issues, we refer readers elsewhere. 2,4 Despite an exponential increase in haptics research activities in the last decade, our understanding of how people sense and manipulate objects with their hands is still limited. 6 The most popular haptic interfaces-such as SensAble Technologies' Phantom and Force Dimension's Delta-let us interact with the virtual environment through one contact point only. Interfaces with a higher number of degrees of freedom (DOF) and with multiple interaction points are available, but are less common or reliable than those with three DOF and one interaction point. With the term haptic rendering, we refer to a branch of haptics research that deals with the calculation of interaction forces between a virtual representation of the user and a virtual object. In most cases, haptic rendering is a two-step process consisting of shape-and texture-based force rendering. In this context, shape refers to the macrogeometry of an object's surface, as opposed to texture that describes the surface's fine structure, or microgeometry. To render an object's shape, we can use typical single-point contact rendering algorithms such as the god-object algorithm. 7 To render the texture of a virtual surface, we can perturb the shape-based force using a texture model. 8 Experiment 1: Macrogeometry watermarking With our first experiment, we hoped to estimate the perceptibility threshold of a watermark modeled as white noise with uniform distribution embedded in the surface's macrogeometry description. As we previously noted, we began by using the simplest case of a flat surface implicitly described by a 3D mesh. We used the same representation for both the host plane and the watermark. The 3D meshes were encoded in data structures representing the spatial coordinates of all the vertices as well as their interconnections. We haptically displayed a virtual mesh using a force-feedback device that allowed single-point contact mediated by a stylus, as Figure 1 depicts. We conveyed information about the surface shape via the direction of the reaction forces that corresponded to the normal vectors to the mesh. The force interaction model did not include friction. We embedded the digital watermark in the surface's macrogeometry by modifying the data matrices according to the additive rule. In this case, we added the watermark signal to the height of the corresponding mesh's vertex. The watermark's strength was represented by the noise spectral power of the equivalent noise model. Human sensitivity to the noise was estimated as the minimum noise level required for the watermark to become detectable. Since the mesh's resolution-that is, the dimension of triangle elements-could vary with application specifications and surface shapes, we conducted the experiment using several 3D meshes with different resolutions. In this way, we were able to establish the relationship between the sensitivity to the watermark's strength and the size of the triangular mesh elements. Methods The host surface was a horizontal square plane of size 15 ´15 cm 2 represented by a 3D triangular mesh and placed in front of the subject. Let v(i) be the 3D vector of the ith triangle vertex and n(i) the surface normal defined at this point. As we mentioned earlier, the embedded watermark altered the mesh vertices according to the following rule: where v w (i) was the ith watermarked vertex and w(i) the watermark noise model. Specifically, we assumed a uniform distribution for w(i) in the range {ٞ⌬, ʳ⌬}. The corresponding frequency domain representation of the watermark noise consisted of a constant spectral power over all frequencies, P w () ͌ ⌬ 2 /12. We asked five human subjects to explore the virtual surfaces using a desktop model of the Phantom force-feedback device. They held the Phantom's stylus with their right hand and Figure 1. Subject touching a virtual surface through a stylus-like device.