Drywall mud pump for pumping drywall joint compound, for instance, from a bucket into a drywall tool. The pump includes a cylinder, a piston, a rod, and a means, or a connection structure between the piston and the rod, for allowing the end of the rod to move laterally relative to the piston. Some embodiments have an elongated hole in the piston that receives a flattened or reduced diameter end of the rod, for example. Different embodiments include a pump head through which the rod passes, a handle, a linkage, a means for allowing the rod to pivot, a shortened guide, a means for preventing the piston from rotating about the rod, or a combination thereof. Pistons may include, for example, passageways with straight and curved sides, a flapper, an elongated washer to block the elongated hole, and a means for controlling the rotational position of the washer.