A first surface mirror includes a reflective layer and one or more dielectric layers. A chromium (Cr) inclusive layer is provided under the reflective layer in order to improve durability and/or optical characteristics of the first surface mirror. In certain example embodiments, a silicon nitride inclusive layer may be located under the Cr inclusive layer to provide for still further improvements.