For obtaining p-Si by irradiating a laser beam to an a-Si layer to polycrystallize, an energy level in a region to be irradiated by the laser beam is set such that a level at the rear area of the region along a scan direction of the laser beam is lower than that at the front area or the center area of the region. The energy level at the front area or the center area of the region is set such that it is substantially equal to or more than the upper limit energy level which maximizes a grain size of the p-Si obtained. Since an energy profile is set as described above, when the laser beam is scanned on the a-Si layer, an irradiated energy of the laser on the region is gradually lowered from the upper limit as the laser beam passes through, which allows the semiconductor layer to be annealed within an optimal energy level during the latter half of the annealing process.