Second-generation sparse modeling: Structured and collaborative signal analysis

In this chapter the authors go beyond traditional sparse modeling, and address collaborative structured sparsity to add stability and prior information to the representation. In structured sparse modeling, instead of considering the dictionary atoms as singletons, the atoms are partitioned in groups, and a few groups are selected at a time for the signal encoding. A complementary way of adding structure, stability, and prior information to a model is via collaboration. Here, multiple signals, which are known to follow the same model, are allowed to collaborate in the coding. The first studied framework connects sparse modeling with Gaussian Mixture Models and leads to state-of-the-art image restoration. The second framework derives a hierarchical structure on top of the collaboration and is well fitted for source separation. Both models enjoy very important theoretical virtues as well. Introduction In traditional sparse modeling, it is assumed that a signal can be accurately represented by a sparse linear combination of atoms from a (learned) dictionary. A large class of signals, including most natural images and sounds, is well described by this model, as demonstrated by numerous state-of-the-art results in various signal processing applications. From a data modeling point of view, sparsity can be seen as a form of regularization, that is, as a device to restrict or control the set of coefficient values which are allowed in the model to produce an estimate of the data.