A signal processing device utilizes a stochastic approximation of a gradient descent algorithm for updating a transform. The signal processing device is configured to implement the transform for producing a desired transformed output signal, and the transform is updated using the stochastic approximation of the gradient algorithm based on received data associated with the signal being processed. The transform is represented in a reduced-parameter form, such as a Givens parameterized form or a Householder form, such that the reduced-parameter form for an NN transform comprises fewer than N² parameters. The updating process is implemented using computations involving the reduced-parameter form, and an adaptation of the transform is represented directly as one or more changes in the reduced-parameter form. The gradient algorithm may be configured to minimize a negative gradient of a pairwise energy compaction property of the transform. Advantageously, the gradient algorithm may be made locally convergent in mean for a specified step size. The invention can also be implemented in a backward adaptive form in which the updating process is driven by quantized data.