I blame Locke for this.
It is not self-evident that people have certain rights, otherwise there wouldn't have been so much dispute over the centuries as to what rights--if any--men have. Anchoring rights to God sounds good, as it makes them sound secure, but you really aren't securing them to anything. If someone disagrees that such rights exist or that such rights were granted by god you need to use some other form of argumentation, e.g., Rawlsian or Utilitarian.