Felt you was out of my league
Its like if you was silk and linen then I was fatigue
He thought she was above him and that she was unattainable. He references the fabric of silk and linen (expensive and high-end) and fatigues (cheap and low-grade quality) as metaphors for their social status.
To help improve the meaning of these lyrics, visit "Hold U" by Masta Ace (Ft. Jean Grae) Lyrics and leave a comment on the lyrics box