The invention relates to a method for optimizing a mask layout pattern comprising at least one structural feature. First a desired layout pattern is provided. Based on the desired layout pattern, an optimized reference diffraction coefficient is provided. After selecting an initial mask geometry having polygon-shaped structures, initial diffraction coefficients are calculated. A difference based on the reference diffraction coefficient and initial diffraction coefficients is used to optimize the initial geometry in order to provide a mask layout pattern.