Numbers is hard and real and they never have feelings
But you push too hard, even numbers got limits
Since the 3/5 compromise, blacks have been viewed as “numbers” by the U.S. government, ciphers and statistics to be manipulated (voter registration) or feared (crime statistics), but fundamentally not worthy of engagement as real, three-dimensional individuals with “feelings.” Mos is warning that this dehumanizing treatment can only continue for so long before eliciting a reaction.
Note, also, that “limits” are a fundamental concept in Calculus, so Mos is actually making a pretty nice math pun here.
To help improve the meaning of these lyrics, visit "Mathematics" by Yasiin Bey Lyrics and leave a comment on the lyrics box