Saying no is not a sin, but there are times we must say it because otherwise we are affected later.