For one, we live in a small society and with so many related to each other, the tendency is to let close friends and relations off the hook. Also, we have an imbibed culture which has taught us that we don't have to worry too much about doing wrong, since it is done by those who enjoy the comforts and respect of the society. Another reason could be that if we do not punish others, then when our turn comes for retribution, we will get a free pass. Lastly, our concept of wrong and right is way off base and lacking in moral fibre.