Exactly. Not only do chemicals like water, carbon or nitrogen seem optimal for supporting the chemical reactions in life, but they're very common in the universe compared to other proposed chemicals.
Thus it is important to search for planets within the temperature ranges in which water is liquid, and familiar chemical reactions can occur. A planet may be at 60 degrees, and be suitable for life.
Though I think there is a possibility of complex organisms existing in an atmosphere with little to no free oxygen. I believe there was a story recently about animals living without oxygen in cold seeps on the mediterranean seafloor, or something to that effect, though unfortunately I can't provide a link.
But just because life is built of the same basic building blocks does not mean there is potential for wildly different biochemistries. It will probably be the norm, rather than the exception, for alien life to be composed of many weird and wonderful never seen before molecules.