Hi Aravind,
As mentioned here already, are you pushing the filtering operation to the lowest level. PFB a nice document from Ravindra Channe which shows the effect of pushing down filtering operation to lowest possible level.
http://scn.sap.com/docs/DOC-41562
Hope it helps.
Regards,
Nikhil Joy