The Virgin Digital Sessions was an internet-only EP release from The Hold Steady in early 2005. Alternate mixes of the first three tracks subsequently appeared on Separation Sunday, with the notable exception of “212 Margarita”.

A Positive Rage is a live album and documentary double-disc set by the Brooklyn-based rock band The Hold Steady, released on April 7, 2009 by Vagrant Records. The CD features a live set performed on Halloween 2007 at the Metro Chicago in Chicago, Illinois, made up of songs from the band’s albums Almost Killed Me, Separation Sunday, Boys and Girls in America, and the then-forthcoming Stay Positive, as well as three tracks previously only released as bonus tracks on these albums.

Live at Fingerprints is an EP of acoustic music performed live by The Hold Steady on October 15, 2006 at the Fingerprints record shop in Long Beach, CA. The release was limited to 5000 copies distributed by Junketboy to independent record shops throughout the United States. The album features acoustic versions of three songs featured in The Hold Steady’s 2006 Boys and Girls in America album, including “Chips Ahoy”, “You Can Make Him Like You” and “Citrus”.