When we say something is tolerated, there is often the sub-text that one needs to put up with something negative.

Hence, religious harmony that is based on tolerance implies that people of one religion will have to submit themselves and suffer in silence when encountering differences in other religions, which they may perceive as the "bad stuff".