One embodiment of the present invention provides a system that improves lithography performance by correcting for 3D mask effects. During operation the system receives a mask layout that contains etched regions, called shifters, which can have a phase shift relative to other regions. Next, the system chooses a shifter in the mask layout. The system then corrects for 3D mask effects by, iteratively, (a) selecting a region within the shifter, (b) adjusting the phase shift of the selected region in a simulation model to account for 3D mask effects, and (c) modifying the shape of the shifter based on the difference between a desired pattern and a simulated pattern generated using the simulation model.