Just because someone doesn't want the same QB as you doesn't make them racist. If anything, you're using his skin color as tokenism.
We all just want the best QB possible on the field at the end of the day. Who cares if they're gay, straight, black, white, christian, muslim, etc?