Provided are an image brightness controlling apparatus and method, and an adaptive brightness controlling apparatus and method based on the brightness degree and/or brightness range of an image. The image brightness controlling apparatus includes a brightness increment arithmetic unit for outputting a brightness increment for a pixel, and an individual component brightness increment arithmetic unit for outputting the brightness increments of individual components constituting the pixel in response to the brightness increment for a pixel. The individual component brightness increment arithmetic unit multiplies the brightness increment of the pixel by each of the unit vectors of the components constituting the pixel to obtain the brightness increments of the individual components. The image brightness controlling apparatus further includes an adder for adding the components constituting the pixel to the brightness increments of the components. The image brightness controlling apparatus and the adaptive image brightness controlling apparatus can maintain the sense of color of an image (e.g., the hue or chroma of an image) while increasing the brightness degree and brightness range of an image.