Department of Mathematics
Superlinear convergence of a general algorithm for the generalized Foley-Sammon discriminant analysis
Linear Discriminant Analysis (LDA) is one of the most efficient statistical approaches for feature extraction and dimension reduction. The generalized Foley-Sammon transform and the trace ratio model are very important in LDA and have received increasing interest. An efficient iterative method has been proposed for the resulting trace ratio optimization problem, which, under a mild assumption, is proved to enjoy both the local quadratic convergence and the global convergence to the global optimal solution (Zhang, L.-H., Liao, L.-Z., Ng, M. K.: SIAM J. Matrix Anal. Appl. 31:1584, 2010). The present paper further investigates the convergence behavior of this iterative method under no assumption. In particular, we prove that the iteration converges superlinearly when the mild assumption is removed. All possible limit points are characterized as a special subset of the global optimal solutions. An illustrative numerical example is also presented. © 2011 Springer Science+Business Media, LLC.
Dimensionality reduction, Generalized Foley-Sammon transform, Linear discriminant analysis, Superlinear convergence, The trace ratio optimization problem
Source Publication Title
Journal of Optimization Theory and Applications
Link to Publisher's Edition
Zhang, Lei-Hong, Li-Zhi Liao, and Michael K. Ng. "Superlinear convergence of a general algorithm for the generalized Foley-Sammon discriminant analysis." Journal of Optimization Theory and Applications 157.3 (2013): 853-865.