Was it really wrong of me to get myself out of lose-lose situation?

I don't blame you. If there is a history of fighting, etc. during past family trips, I wouldn't want to subject myself and especially my children to it. If your husband wants to go so badly, then let him go and leave you and the kids at home. Tell the in-laws that the kids have some event that they have to go to so they couldn't come. My experience with in-laws is that less time together is better.