The Utilization of Partial Least Squares for Simultaneous Feature Selection and Extraction

Abstract

Several works concerning the utilization of Partial Least Squares as a supervised dimension reduction technique have been developed over the years in the field of chemometrics, among others, for regression purposes. However, Partial Least Squares can be a challenging procedure especially in the case of multivariate multiple regression due to data characteristics and complexity. Thus, in this work we propose the use of Partial Least Squares method as a variable selection technique in linear regression tasks that involve high dimensional spectral data sets. More precisely, we suggest the exploitation of the regression coefficients that Partial Least Squares estimates in order to identify and eject the insignificant predictor variables from the analysis. In such manner we are able to remove the uninformative variables and obtain in most cases better results than the classical Partial Least Squares regression but with simpler structure. We compare our proposed technique with the classical Partial Least Squares and Principal Component Analysis in both univariate and multivariate regression.