An image brightness and contrast management module for automatically adjusting brightness and contrast of an input video signal includes a histogram generation unit that generates a pixel luminance histogram based on pixel luminance values in a frame, whereby the histogram shows a distribution of luminance values in the frame and indicates the dynamic range and the dominant luminance value(s) for the frame. The module includes a histogram characterization unit that uses the pixel luminance histogram to identify which of a plurality of brightness/contrast properties is exhibited in the frame. The module also includes a brightness and contrast adjustment unit that nonlinearly adjusts, in real-time, the pixel luminance values in a next frame based on the identified brightness/contrast properties exhibited in the preceding frame. In this manner, bright or dark dominant areas of an image are stretched to enhance the contrast of the dominant luminance values without blackening the darker portions or saturating the brighter portions of the image.