A feature selection technique for support vector machine (SVM) classification makes use of fast Newton method that suppresses input space features for a linear programming formulation of a linear SVM classifier, or suppresses kernel functions for a linear programming formulation of a nonlinear SVM classifier. The techniques may be implemented with a linear equation solver, without the need for specialized linear programming packages. The feature selection technique may be applicable to linear or nonlinear SVM classifiers. The technique may involve defining a linear programming formulation of a SVM classifier, solving an exterior penalty function of a dual of the linear programming formulation to produce a solution to the SVM classifier using a Newton method, and selecting an input set for the SVM classifier based on the solution.