That used to be the job of liberals :'(
But the big pushes for civil rights/liberties came when you had large groups of people being affected, who served as the core activist constituency. Blacks, women, gays, etc. And the victories happened as the rest of the public became sympathetic to their goals.
The increasing leeway for how the government deals with those it deems terrorists is like the increasing restrictions on sex offenders after they leave prison: lousy policy that's very possibly unconstitutional, but targeted at a small group that most people feel deserve punishment. There's no organized Molester Lobby to speak of, and if there were no politician would be caught dead dealing with them. So the ACLU keeps plugging away hoping the courts do the right thing, but we've seen enough recently to know you can't count on that.
It doesn't help that the people most vocal about this are clueless about the political process or deliberately alienating potential allies, but it's not like it matters. We'd need an inadvertent drone strike on a blonde 19 year old college girl before people would give a shit.
