There is provided a coated article (e.g., solar cell) that includes an improved anti-reflection (AR) coating. This AR coating functions to reduce reflection of light from a glass substrate, thereby allowing more light within the solar spectrum to pass through the incident glass substrate. In certain example embodiments, the AR coating includes a graded base layer having a varying refractive index value, and an overcoat layer which may be provided for destructive interference purposes.