Compressed Sensing of Memoryless Sources:A Deterministic Hadamard Construction

Saeid Haghighatshoar
2014
vii brother, I have always had the great privilege to be loved and to be supported by them all the time. I am also deeply grateful to my three little nephews who have always cheered me up as an uncle and have been giving me the happiest moments in my life. Special thanks and recognitions go to my beloved wife for her deep love and heart-filling affection. I truly thank her for sticking by my side and believing in me even in those moments that I did not believe in myself. There are no words that
more » ... can truly express my gratitude and love for her. Abstract Compressed sensing is a new trend in signal processing for efficient data sampling and signal acquisition. The idea is that most real-world signals have a sparse representation in an appropriate basis and this can be exploited to capture the sparse signal by taking a few linear projections. The recovery is possible by running appropriate low-complexity algorithms that exploit the sparsity (prior information) to reconstruct the signal from the linear projections (posterior information). The main benefit is that the required number of measurements is much smaller than the dimension of the signal. This results in a huge gain in sensor cost (in measurement devices) or a dramatic saving in data acquisition time. The drawback is that one needs complicated devices in order to implement the linear projections. Some difficulties naturally arise in applying the compressed sensing to realworld applications such as robustness issues in taking the linear projections and computational complexity of the recovery algorithm. Robustness issue arises because even if the devices are precisely calibrated, there is still a mismatch with the intended linear projection. Obtaining stable and low-complexity recovery algorithms has also been a challenge in compressed sensing. Although there are numerically stable convex optimization algorithms for recovery, their complexity usually scales like O(n 3 ) in signal dimension n, which prohibits their use in high dimensional applications that are encountered more and more nowadays. Consequently, there have been different attempts to reduce this complexity as much as possible. In this thesis, we design structured matrices for compressed sensing. In particular, we claim that some of the practical difficulties can be reasonably solved by imposing some structure on the measurement matrices. Almost all the thesis evolves around the Hadamard matrices, which are {+1, −1}-valued matrices with many applications in signal processing, coding theory, optics and theoretical mathematics. As the title of the thesis implies, there are two main ingredients to this thesis. First, we use a memoryless assumption for the source, i.e., we assume the nonzero components of the sparse signal are independently generated by a given probability distribution and their position is completely random. This is not a major restriction because most of the results obtained are not sensitive to the shape of the distribution. The advantage is that the probabilistic model of the signal allows us to use tools from probability, information theory and coding theory to rigorously assess the achievable performance. Second, using the mathematical properties of the Hadamard matrices, we design deterministic matrices for compressed sensing of memoryless sources by selecting specific rows of a Hadamard matrix according to a deterministic criterion. We call the resulting matrices partial Hadamard matrices. ix x Abstract We design partial Hadamard matrices for three distinct signal models: memoryless discrete sources and sparse signals with linear or sub-linear sparsity. A signal has linear sparsity if the number of its nonzero components k is proportional to n, the dimension of signal, whereas it has a sub-linear sparsity if the k scales like O(n α ) for some α ∈ (0, 1). We develop tools to rigorously analyze the performance of the proposed Hadamard constructions by borrowing ideas from information theory and coding theory. In the last part of the thesis, we extend our construction to distributed (multiterminal) signals. Distributed compressed sensing is a very interesting and ubiquitous problem in distributed data acquisition systems such as ad-hoc sensor networks. From both a theoretical and an engineering point of view, it is important to know how many measurements per dimension are necessary from different terminals in order to have a reliable estimate of the distributed data. We analyze this problem for a very simple setup, where the components of the distributed signal are generated by a joint probability distribution which, in some sense, captures the spatial correlation among different terminals. We give an information-theoretic characterization of the measurement-rate region that results in a negligible recovery distortion. We also propose a low-complexity distributed message passing algorithm to achieve the theoretical limits.
doi:10.5075/epfl-thesis-6356 fatcat:rmrtepg5uzb35bmhotbhehwbn4