Bumping a substrate having a metal layer thereon may include forming a barrier layer on the substrate including the metal layer and forming a conductive bump on the barrier layer. Moreover, the barrier layer may be between the conductive bump and the substrate, and the conductive bump may be laterally offset from the metal layer. After forming the conductive bump, the barrier layer may be removed from the metal layer thereby exposing the metal layer while maintaining a portion of the barrier layer between the conductive bump and the substrate. Related structures are also discussed.