One embodiment of the present invention provides a system with a mechanism
that facilitates determining whether a kernel stack will overflow during
development of an operating system. During execution of the operating
system, the operating system executes a procedure. Upon entering the
procedure, after the current stack pointer has been automatically
adjusted to provide stack space for the procedure, the system retrieves
the current stack pointer for the process, and determines the stack limit
for the process. The system then compares the current stack pointer to
the stack limit to determine if the process will cause, or is likely to
cause a stack overflow. If a stack overflow will occur, or is likely to
occur, the system indicates the existence of a stack overflow condition.