The first album freedoms and them fourth album pressures
A big difference between 'em but I get why you said it
Your first album is the one you have the most freedom with because it’s your debut, and you are creating the style into which you are expected to fit. As time goes on, people begin to create perceptions of you as an artist and as a person; they will expect certain things and will press you for that type of music. Behind the scenes, corporate executives may demand a hit album because it’s your fourth one and they want to milk you for the money cow you are (in their eyes).
Furthermore, in the original song Cole says:
The song “You Owe Me” is from Nas’s fourth album Nastradamus. Nas says that, since it was his fourth album, he was under more pressure from his label execs to produce a hit and so he “sacrificed his art” to make a radio single much like Cole would later do with “Work Out”.
The point is both artists were under pressure to make a hits (and money for their labels) and that the material they released at certain points may not have been their best work. Nas understands the pressures and applauds J. Cole on his return to form on Born Sinner.
Additionally, while Cole World wasn’t Cole’s fourth album, it was his fourth LP release, with The Come Up, The Warm Up, and Friday Night Lights all preceding it. Both Cole’s and Nas' fourth releases were viewed as a commercial sellout, further underlining the similarity between the two and thus strengthening Nas’s statement that he can relate.
Alternatively, one could realise that Cole made a sell-out track, so to speak, on his first album while Nas did it on his fourth, and you can’t really relate the two because “first album freedoms and fourth album pressures” are so different. In this way, Nas is sort of refuting Cole’s line in the original song “i thought that you could relate”.. which he can but he also notes the difference.
To help improve the meaning of these lyrics, visit "Let Nas Down (Remix)" by J. Cole (Ft. Nas) Lyrics and leave a comment on the lyrics box