A method and apparatus for providing a processor based nested form polynomial engine are disclosed. A concise instruction format is provided to significantly decrease memory required and allow for instruction pipelining without branch penalty using a nested form polynomial engine. The instruction causing a processor to set coefficient and data address pointers for evaluating a polynomial, to load loading a coefficient and data operand into a coefficient register and a data register, respectively, to multiply the contents of the coefficient register and data register to produce a product, to add a next coefficient operand to the product to produce a sum, to provide the sum to an accumulator and to repeat the loading, multiplying, adding and providing until evaluation of the polynomial is complete.