And eventually answers to the call of Autumn

from Kanye West (Ft. Kid Cudi & Raekwon) – Gorgeous Lyrics on Rap Genius

Meaning

Autumn is commonly used in poetry as a metaphor to refer to the decline of something. i.e. the cooling after a summer.

In this line Kanye refers to the decline, or fall, of those folks marked (branded) by race or class taking risks with jail time (penitentiary chances) in the hopes of shinin' (for the love of ballin') as their continued risks eventually fail to pay off and they end up incarcerated.

Each lines in these first eight bars are very intertwined.
Also, he could be referencing Andre 3000, from the song 13th Floor/Growing Old off the album ATLiens.

To help improve the meaning of these lyrics, visit “Gorgeous” by Kanye West (Ft. Kid Cudi & Raekwon) Lyrics and leave a comment on the lyrics box