What you say is accurate, any one can choose to support a majority consensus. In fact, this group could be democratic and anyone who didn't want to do something could, at worst, leave the group.

What I am recommending is just to go one step further (mostly so I don't have to leave the group) and only have democracy within voluntary subgroups. I have also suggested in the discord that we use ranked choice voting, which is a way to avoid 'winner-take-all'.