A method includes stalling a call to a critical operating system (OS) function and determining whether branch trace records of the call include a return instruction. Upon a determination that the branch trace records of the call do include a return instruction, the method further includes taking protective action to protect a computer system.