A system and method for the discovery and selection of an optimal probability model. Probability model selection can be formulated as an optimization problem with linear order constraints and a non-linear objective function. In one aspect, an algorithm that employs an information-theoretic based approach is used in which a scalable combinatoric search is utilized as defined by an initial solution and null vectors. The theoretical development of the algorithm has led to a property that can be interpreted semantically as the weight of evidence in information theory.