A method of training neural systems and estimating regression coefficients of regression models with respect to an error criterion is disclosed. If the error criterion is a risk-averting error criterion, the invented method performs the training/estimation by starting with a small value of the risk-sensitivity index of the risk-averting error criterion and gradually increasing it to ensure numerical feasibility. If the error criterion is a risk-neutral error criterion such as a standard sum-of-squares error criterion, the invented method performs the training/estimation first with respect to a risk-averting error criterion associated with the risk-neutral error criterion. If the result is not satisfactory for the risk-neutral error criterion, further training/estimation is performed either by continuing risk-averting training/estimation with decreasing values of the associated risk-averting error criterion or by training/estimation with respect to the given risk-neutral error criterion or by both.