Method and apparatus for face alignment by building a hierarchical classifier network. The hierarchical classifier network connects the tasks of face detection and face alignment into a smooth coarse-to-fine procedure. Texture classifiers are trained to recognize feature texture at different scales for different resolution layers. A multi-layer structure is employed to organize the texture classifiers, which begins with one classifier at the first layer and gradually refines the localization of feature points using additional texture classifiers in subsequent layers.