A semiconductor memory device comprises: a memory cell array having a standard memory cell array part in which dynamic memory cells are arranged in a matrix pattern, and a redundant memory cell array having a redundant memory cell set up to replace a defective memory cell in the standard memory cell array part; an access control part controlling external access operation and refresh access operation regarding the memory cell array; and a redundancy judgment circuit executing redundancy judgment to determine whether the memory cell which is a subject to the external access operation or the refresh access operation is the redundant memory cell or not, controlling so as to access the redundant memory cell, if the subjected memory cell is the redundant memory cell, and controlling so as to access the memory cell in the standard memory cell array, if the subjected memory cell is not the redundant memory cell. In case of executing the redundancy judgment for the refresh access operation by the redundancy judgment circuit and the refresh access operation according to generation of a refresh access request indicating the start of the refresh access operation, if the external access request indicating the start of the external access operation generates during the time from start of redundancy judgment for the refresh access operation until completion of the refresh access operation, the access control part makes the redundancy judgment circuit execute the redundancy judgment for the external access operation in parallel to the refresh access operation, and execute the external access operation after completion of the refresh access operation.