A riveting yoke assembly (11) comprises a yoke (30), a force applying mechanism (22) and a rivet forming device (34,36). The yoke has a first end (38), a second end (40), and a middle section (42) coupled between the first and second ends (38,40). An opening (44) is formed through the yoke between the first and second ends. The force applying mechanism (22) is coupled to the first end (38) of the yoke (30). The lower rivet forming device (36) is removably coupled to the second end (40) of the yoke. The lower rivet forming device (36) has a base end (46) attached to the second end (40) of the yoke (30) and a forming end (48) with a recess (50) to form rivets (17). The recess (50) has a concave, interior surface (52) having an annular step (54) positioned between a top edge (56) of the interior surface (52) and a bottom-most point (58) of the interior surface (52) in order to properly align the rivet (17).