Don't only look at the crime, look at punishment too: PM