Tools + Demos
|Zurück / back|
Robust and Fast Learning of Sparse Codes with Stochastic Gradient Descent
Particular classes of signals, as for example natural images, can be
encoded sparsely if appropriate dictionaries are used. Finding such
dictionaries based on data samples, however, is a difficult optimization
task. We have shown that simple stochastic gradient descent, besides
being much faster, leads to superior dictionaries compared to other
state-of-the-art methods ,. Moreover, we introduced the so-called
Bag of Pursuits methodas an extension of Orthogonal Matching
Pursuit,  and have shown that it provides an improved
approximation of the optimal sparse coefficients and, therefore,
significantly improves the performance of dictionary learning
approaches that employ it in order to determine the coefficients within
the learning process . Finally, we have shown how the Bag of
Pursuits and a generalized version of the Neural Gas algorithm can be
used to derive an even more powerful method for soft-competitive sparse
We provide a an OCTAVE/MATLAB package that contains an optimized implementation of Neural Gas for Dictionary Learning and the Bag of Pursuits method for sparse approximation:
(1) Download the Neural Gas for Dictionary Learning (NGDL) package (Warning! This file is about 150MB since it also contains all the training data that is required in order to perform the included example experiments).
(2) Extract the tar archive: tar -xzf NGDL.tar.gz
The archive contains precompiled MATLAB binaries for the linux x86-64 architecture. There are also OCTAVE binaries for linux x86-64 architecture included. For other platforms/operating systems, you have to compile the sources:
(3) Add the -msse2 compiler option to the CFLAGS variable of the MATLAB MEX compiler (mexopt.sh)
(4) Enter the NGDL directory
(5) Compile the C sources by typing make. Note that the MATLAB MEX compiler has to be in the path.
Note 1: If you don't want to use SSE2 you can disable it by removing the __USE_SSE__ compiler option in the make file.
Note 2: It is possible to compile the package for octave usage. In order to obtain octave compatible mex files modify the CC variable in the make file (replace mex by mkoctfile, see make file for more information).
Note 3: On x86-64 architecture use the __USE_SSE64__ compiler option (see make file).
Please contact firstname.lastname@example.org if you have further questions.