Methods and systems are provided for feature selection in machine learning, in which the features selected for inclusion in a prediction rule are selected based on statistical metric(s) of feature contribution and/or model fitness.