toxic homefront : decisions to make?

I contribute my time to a volunteer organization. As with any organization, there are people who have personal agendas and, in attempting to carry out those agendas, are generally toxic and unhappy. I am a key person in this organization (as a volunteer) and I get nothing from it, other than a sense of mission and community involvement. (In fact, I am volunteering at the behest of a friend who is also deeply involved in the organization.)

I'm particularly sensitive to the toxicity and have sometimes been a target of it. I keep telling myself not to take it personally, but I can't help it. I wonder why I should devote my time to a place where some people (with lots of time on their hands, way more than I do, as I work two jobs) are generally just hateful. I can objectively stand back and assess what I have done for the organization and assert that it has been good. But in the eyes of the toxic people, I can never do anything right.

I want to leave the organization, but my friend is pressuring me to stay. I have about another year in my commitment and leaving means that the organization would have to seek a replacement (which isn't really a big deal). Sometimes I feel that I really need to honour my original commitment, but.....

I am torn. Help!