The private property owner makes that distinction, and anyone in violation does not have the right to be on that property, including law enforcement. Of course, if they have a search warrant or an exception, then the property owner's wishes don't matter. But short of that, they are trespassing if they enter the "gun free zone" areas while carrying, just like you or I would be.
Unless, of course, the property owner makes an exception for law enforcement, which is the case for most businesses. And I am POSITIVE that no law enforcement officer would leave his service weapon behind just to enter a certain business. So in effect, you're right, but it's because of selective enforcement of the restriction. I'm sure someone will be along shortly to file a lawsuit about that, too...