A motion controlled handheld device includes a user interface comprising a display having a viewable surface and operable to generate a current image and a motion detection module operable to detect motion of the device within three dimensions and to identify components of the motion in relation to the viewable surface. The device includes a device state tracking module operable to analyze the components to determine an environmental state of the device. The environmental state comprises a motion state and an orientation of the device with respect to gravity. The device also includes a controller operable to execute an application and to perform an operation of the application based on the environmental state.