Someone might be able to deduce what circles you have them in based on the membership, but beyond that, I don't see the problem there. If anyone comments, they're revealed to everyone in the circle anyway, like -- fantasy scenario -- if you have a fuckbuddy circle, it's not exactly a secret when someone responds to your internet booty-call.