The basic assumption in CS approach is that most of the signals in real applications have a concise representation in a certain transform domain where only few of them are significant, while the rest are zero or negligible [ 5 — 7 ].
This requirement is defined as signal sparsity. Another important requirement is the incoherent nature of measurements observations in the signal acquisition domain. Therefore, the main objective of CS is to provide an estimate of the original signal from a small number of linear incoherent measurements by exploiting the sparsity property [ 3 , 4 ].
The CS theory covers not only the signal acquisition strategy, but also the signal reconstruction possibilities and different algorithms [ 8 — 17 ]. Several approaches for CS signal reconstruction have been developed and most of them belong to one of three main approaches: convex optimizations [ 8 — 11 ] such as basis pursuit, Dantzig selector, and gradient-based algorithms; greedy algorithms like matching pursuit [ 14 ] and orthogonal matching pursuit [ 15 ]; and hybrid methods such as compressive sampling matching pursuit [ 16 ] and stage-wise OMP [ 17 ].
When comparing these algorithms, convex programming provides the best reconstruction accuracy, but at the cost of high computational complexity. The greedy algorithms bring about low computation complexity, while the hybrid methods try to provide a compromise between these two requirements [ 18 ]. The proposed work provides a survey of the general compressive sensing concept supplemented with the several existing approaches and methods for signal reconstruction, which are briefly explained and summarized in the form of algorithms with the aim of providing the readers with an easier and practical insight into the state of the art in this field.
Apart from the standard CS algorithms, a few recent solutions have been included as well. Furthermore, the paper provides an overview of different sparsity domains and the possibilities of employing them in the CS problem formulation. Additional contribution is provided through the examples showing the efficiency of the presented methods in practical applications.
The paper is organized as follows. In Section 2 , a brief review of the general compressive sensing idea is provided together with the conditions for successful signal reconstruction from reduced set of measurements and the signal recovery formulations using minimization approaches.
In Section 3 , the commonly used CS algorithms are reviewed. The commonly used domains for CS strategy implementation are given in Section 4 , while some of the examples in real applications are provided in Section 5. The concluding remarks are given in Section 6. Reducing the sampling rate using CS is possible for the case of sparse signals that can be represented by a small number of significant coefficients in an appropriate transform basis.
A signal having K nonzero coefficients is called -sparse. Assume that signal exhibits sparsity in certain orthonormal basis defined by the basis vectors. The signal can be represented using its sparse transform domain vector as follows: In matrix notation, the previous relation can be written as Commonly, the sparsity is measured using the -norm, which represents the cardinality of the support of : In real applications, the signals are usually not strictly sparse but only approximately sparse.
Therefore, instead of being sparse, these signals are often called compressible, meaning that the amplitudes of coefficients decrease rapidly when arranged in descending order. For instance, if we consider coefficients , then the magnitude decays with a power law if there exist constants and satisfying [ 19 ] where larger means faster decay and consequently more compressible signal.
The signal compressibility can be quantified using the minimal error between the original and sparsified signal obtained by keeping only largest coefficients :. Instead of acquiring a full set of signal samples of length , in CS scenario, we deal with a quite reduced set of measurements of length , where. The measurement procedure can be modeled by projections of the signal onto vectors constituting the measurement matrix : Using the sparse transform domain representation of vector s given by 2 , we have where will be referred to as CS matrix.
In order to define some requirements for the CS matrix , which are important for successful signal reconstruction, let us introduce the null space of matrix. The null space of CS matrix contains all vectors that are mapped to 0: In order to provide a unique solution, it is necessary to provide the notion that two K -sparse vectors and do not result in the same measurement vector.
In other words, their difference should not be part of the null space of CS matrix : Since the difference between two K -sparse vectors is at most 2 K -sparse, then a K -sparse vector is uniquely defined if null space of contains no 2 K -sparse vectors.
This corresponds to the condition that any 2 K columns of are linearly independent; that is, and since , we obtain a lower bound on the number of measurements: In the case of strictly sparse signals, the spark can provide reliable information about the exact reconstruction possibility. However, in the case of approximately sparse signals, this condition is not sufficient and does not guarantee stable recovery. Hence, there is another property called null space property that measures the concentration of the null space of matrix.
The null space property is satisfied if there is a constant such that [ 19 ] for all sets with cardinality K and their complements. If the null space property is satisfied, then a strictly -sparse signal can be perfectly reconstructed by using -minimization. For approximately -sparse signals, an upper bound of the -minimization error can be defined as follows [ 20 ]: where is defined in 5 for as the minimal error induced by the best -sparse approximation.
The null space property is necessary and sufficient for establishing guarantees for recovery. A stronger condition is required in the presence of noise and approximately sparse signals.
This property shows how well the distances are preserved by a certain linear transformation. We might now say that if the RIP is satisfied for 2 K with , then there are no two K -sparse vectors that can correspond to the same measurement vector. Finally, the incoherence condition mentioned before, which is also related to the RIP of matrix , refers to the incoherence of the projection basis and the sparsifying basis.
The mutual coherence can be simply defined by using the combined CS matrix as follows [ 21 ]: The mutual coherence is related to the restricted isometry constant using the following bound [ 22 ]:. The signal recovery problem is defined as the reconstruction of vector from the measurements. This problem can be generally seen as a problem of solving an underdetermined set of linear equations. However, in the circumstances when is sparse, the problem can be reduced to the following minimization: The -minimization requires an exhaustive search over all possible sparse combinations, which is computationally intractable.
Hence, the -minimization is replaced by convex -minimization, which will provide the sparse result with high probability if the measurement matrix satisfies the previous conditions. The -minimization problem is defined as follows: and it has been known as the basis pursuit.
In the situation when the measurements are corrupted by the noise of level and , the reconstruction problem can be defined in a form: called basis pursuit denoising.
The error bound for the solution of 19 , where A satisfies the RIP of order with and , is given by where the constants and are defined as [ 19 ] For a particular regularization parameter , the minimization problem 19 can be defined using the unconstrained version as follows: which is known as the Lagrangian form of the basis pursuit denoising.
These algorithms are commonly solved using primal-dual interior-point methods [ 22 ]. Another form of basis pursuit denoising is solved using the least absolute shrinkage and selection operator LASSO , and it is defined as follows: where is a nonnegative real parameter. The convex optimization methods usually require high computational complexity and high numerical precision. When the noise is unbounded, one may apply the convex program based on Dantzig selector it is assumed that the noise variance is per measurement, i.
Besides the -norm minimization, there exist some approaches using the -norm minimization, with : or using -norm minimization, in which case the solution is not rigorously sparse enough [ 23 ]:.
The -minimization problems in CS signal reconstruction are usually solved using the convex optimization methods. In addition, there exist greedy methods for sparse signal recovery which allow faster computation compared to -minimization. Greedy algorithms can be divided into two major groups: greedy pursuit methods and thresholding-based methods. In practical applications, the widely used ones are the orthogonal matching pursuit OMP and compressive sampling matching pursuit CoSaMP from the group of greedy pursuit methods, while from the thresholding group the iterative hard thresholding IHT is commonly used due to its simplicity, although it may not be always efficient in providing an exact solution.
Some of these algorithms are discussed in detail in this section. The matching pursuit algorithm has been known for its simplicity and was first introduced in [ 14 ]. This is the first algorithm from the class of iterative greedy methods that decomposes a signal into a linear set of basis functions. Through the iterations, this algorithm chooses in a greedy manner the basis functions that best match the signal. Also, in each iteration, the algorithm removes the signal component having the form of the selected basis function and obtains the residual.
This procedure is repeated until the norm of the residual becomes lower than a certain predefined threshold value halting criterion Algorithm 1.
The matching pursuit algorithm however has a slow convergence property and generally does not provide efficiently sparse results. The orthogonal matching pursuit OMP has been introduced [ 15 ] as an improved version to overcome the limitations of the matching pursuit algorithm.
OMP is based on principle of orthogonalization. It computes the inner product of the residue and the measurement matrix and then selects the index of the maximum correlation column and extracts this column in each iteration. The extracted columns are included into the selected set of atoms. Then, the OMP performs orthogonal projection over the subspace of previously selected atoms, providing a new approximation vector used to update the residual.
Here, the residual is always orthogonal to the columns of the CS matrix, so there will be no columns selected twice and the set of selected columns is increased through the iterations. OMP provides better asymptotic convergence compared to the previous matching pursuit version Algorithm 2. Then, the algorithm selects 2 columns of corresponding to the 2 largest absolute values of , where defines the signal sparsity number of nonzero components.
Namely, all but the largest 2 elements of are set to zero and the 2 support is obtained by finding the positions of nonzero elements. The indices of the selected columns 2 in total are then added to the current estimate of the support of the unknown vector. After solving the least squares, a 3 -sparse estimate of the unknown vector is obtained. Then, the 3 -sparse vector is pruned to obtain the -sparse estimate of the unknown vector by setting all but the largest elements to zero.
Thus, the estimate of the unknown vector remains -sparse and all columns that do not correspond to the true signal components are removed, which is an improvement over the OMP. Namely, if OMP selects in some iteration a column that does not correspond to the true signal component, the index of this column will remain in the final signal support and cannot be removed Algorithm 3. It finds the largest 2s components of the signal proxy.
It merges the support of the signal proxy with the support of the solution from the previous iteration. It estimates a solution via least squares where the solution lies within a support T. It takes the solution estimate and compresses it to the required support.
Another group of algorithms for signal reconstruction from a small set of measurements is based on iterative thresholding. Generally, these algorithms are composed of two main steps. The first one is the optimization of the least squares term, which is done by solving the optimization problem without -minimization. The other one is the decreasing of the -norm, which is done by applying the thresholding operator to the magnitude of entries in.
In each iteration, the sparse vector is estimated by the previous version using negative gradient of the objective function defined as while the negative gradient is then Generally, the obtained estimate is sparse, and we need to set all but the K largest components to zero using the thresholding operator.
Here, we distinguish two types of thresholding. Hard thresholding [ 24 , 25 ] sets all but the K largest magnitude values to zero, where the thresholding operator can be written as where is the K largest component of x.
The algorithm is summarized in Algorithm 4. The stopping criterion for IHT can be a fixed number of iterations or the algorithm terminates when the sparse vector does not change much between consecutive iterations. The soft-thresholding function can be defined as and it is applied to each element of. One of the algorithms that has been used for solving 32 is the iterated soft-thresholding algorithm ISTA , also called the thresholded Landweber algorithm.
In order to provide an iterative procedure for solving the considered minimization problem, we use the minimization-maximization [ 26 ] approach to minimize.
Hence, we should create a function that is equal to at ; otherwise, it upper-bounds. Minimizing a majorizer is easier and avoids solving a system of equations. This algorithm see [ 12 , 13 ] starts from the assumption that the missing samples, modeled by zero values in the signal domain, produce noise in the transform domain. Namely, the zero values at the positions of missing samples can be observed as a result of adding noise to these samples, where the noise at the unavailable positions has values of original signal samples with opposite sign.
For instance, let us observe the signals that are sparse in the Fourier transform domain. The variance of the mentioned noise when observed in the discrete Fourier transform DFT domain can be calculated as where M is the number of available and N is the total number of samples and is a measurement vector.
Consequently, using the variance of noise, it is possible to define a threshold to separate signal components from the nonsignal components in the DFT domain.
For a desired probability , the threshold is derived as The automated threshold based algorithm, in each iteration, detects a certain number of DFT components above the threshold. This will reveal the remaining components that are below the noise level. Further, it is necessary to update the noise variance and threshold value for the new iteration. Since the algorithm detects a set of components in each iteration, it usually needs just a couple of iterations to recover the entire signal.
In the case that all signal components are above the noise level in DFT, the component detection and reconstruction are achieved in single iteration.
This algorithm belongs to the group of convex minimization approaches [ 11 ]. Unlike other convex minimization approaches, this method starts from some initial values of unavailable samples initial state which are changed through the iterations in a way to constantly improve the concentration in sparsity domain.
In general, it does not require the signal to be strictly sparse in certain transform domain, which is an advantage over other methods. Particularly, the missing samples in the signal domain can be considered as zero values.
In each iteration, the missing samples are changed for and for. Then, for both changes, the concentration is measured as -norm of the transform domain vectors for change and for change , while the gradient is determined using their difference lines 8, 9, and Finally, the gradient is used to update the values of missing samples.
Here, it is important to note that each sample is observed separately in this algorithm and one iteration is finished when all samples are processed. NLP programming automates the translation process between computers and humans by manipulating unstructured data words in the context of a specific task conversation. View Full Term.
By clicking sign up, you agree to receive emails from Techopedia and agree to our Terms of Use and Privacy Policy. This makes signal processing and reconstruction much simpler and has a wide variety of applications in the real world, including photography, holography and facial recognition. Compressed sensing is also known as compressive sensing, compressive sampling and sparse sampling.
The Nyquist-Shannon sampling theorem states that a signal can be reconstructed perfectly if the highest frequency is less than half the sampling rate. The lower sampling rate makes storing and processing this data much more efficient. Some of the applications of this insight include mobile phone cameras, holography, facial recognition, medical imaging, network tomography and radio astronomy. By: Justin Stoltzfus Contributor, Reviewer.
By: Satish Balakrishnan. Dictionary Dictionary Term of the Day. Natural Language Processing. Techopedia Terms. Connect with us.
0コメント