Back when I was in film school we called shaky cam "cinéma vérité," a style named after a French Film movement which sought "truth" in film. It was designed to emulate the hand-held news or documentary cameras that people saw either on screen in news-reels or on the evening news. That's why it is deployed to make something feel more "real."
With the prevalence of home video cameras and people capturing life all around them at all times, we're becoming more attuned to hand-held camera work as being the "true" view of the world around us. And most amateur videographers not only don't use tripods or other stabilizers, but refuse to use them because it slows them down. Hence, "real" video becomes more and more shaky in our collective minds.
I feel that judicious use of "shaky cam" can be effective, and that it was used well in BSG and D9, for the most part. But if used badly, it can distract from the story enormously. As our collective tastes change, and as our cultural tolerance for amateur videography increases, I fear we'll see more and more of it in major cinema releases, not because it is the right directorial choice, but because it's what people expect.
At least until the trendiness pendulum swings back to more traditional filmmaking.
