Fast Kernel ICA using an Approximate Newton Methodby Hao Shen, Stefanie Jegelka, and Arthur Gretton |
---|
This package contains a Matlab implementation of the Fast Kernel ICA algorithm, as described in SheJegGre09, SheJegGre07.
The goal of ICA is to separate linearly mixed sources to minimise the statistical dependence between the estimated unmixed sources. Kernel ICA algorithms use kernel measures of statistical independence as their optimisation criteria (BacJor02, GreHerSmoBouSch05). Fast Kernlel ICA (FastKICA) employs an approximate Newton method to perfom the optimization efficiently for larger-scale problems.
The kernel independence criterion we use here is the Hilbert-Schmidt norm of the covariance operator in feature space (see GreBouSmoSch05). Another interpretation of this criterion is as a characteristic function based independence measure (Feu93), as used previously in ICA by CheBic05 and EriKoi03.
The functions 'chol_gauss' and 'amariD' are based on code from Francis Bach. The derivative is computed as described in JegGre07 (for incomplete Cholesky decomposition).
Unzipping the downloaded archive will create a directory called 'fastKICA', which should contain the following files:
fastkica.m | main routine |
README.txt | Instructions on use |
utils\chol_gauss.c | Incomplete Cholesky decomposition (from Francis Bach). |
utils\dChol2.c | Mex code for derivative (Windows). |
utils\dChol2Lin.c | Mex code for derivative (Linux). |
utils\Kmn.c, dKmnKin.c | Mex code, also for derivative (Win/Linux). |
utils\getKern.c | Mex code to compute Gaussian kernel. |
utils\compDerivChol.m | Matlab interface for the gradient. |
utils\dChol.m, dCholLin.m | Needed for the gradient (Win/Linux). |
utils\hsicChol.m | Computes HSIC. |
utils\hessChol.m | Computes the Hessian. |
demo.m | Code for demo of kernel ICA. |
amariD.m | Amari divergence (based on the code by Francis Bach). |
source2.wav | Data file |
source3.wav | Data file |
source4.wav | Data file |
[SheJegGre09] | Shen, H., S. Jegelka and A. Gretton: Fast Kernel-Based Independent Component Analysis. IEEE Transactions on Signal Processing 2009. download |
[SheJegGre07] | Shen, H., S. Jegelka and A. Gretton: Fast Kernel ICA using an Approximate Newton Method. AISTATS 2007. download |
[JegGre07] | Jegelka, S. and A. Gretton: Brisk Kernel ICA. In Large Scale Kernel Machines, MIT Press, Cambridge, Massachusetts (2007) download |
[BacJor02] | Bach, F. and M. I. Jordan: Kernel Independent Component Analysis, Journal of Machine Learning Research 3, 1-48, 2002. |
[GreHerSmoBouSch05] | Gretton, A., R. Herbrich, A. Smola, O. Bousquet and B. Schoelkopf: Kernel Methods for Measuring Independence. Journal of Machine Learning Research 6, 2075-2129 (2005). download |
[GreBouSmoSch05] | Gretton, A., O. Bousquet, A. Smola and B. Schoelkopf: Measuring Statistical Dependence with Hilbert-Schmidt Norms. ALT 2005, 63-78. download |
[Feu93] | Feuerverger, A.: A Consistent Test for Bivariate Dependence, International Statistical Review 61(3), 419-433, 1993. |
[CheBic05] | Chen, A. and P. Bickel: Consistent independent component analysis and prewhitening, IEEE Trans. Signal Process. 53(10), 3625-3632, 2005. |
[EriKoi03] | Eriksson, J. and V. Koivunen: Characteristic-function-based independent component analysis, Signal Processing 83(10), 2195-2208, 2003 |