Me and fiance...holiday woes!?

My fiance's dad and stepmom drove 14 hours to visit us over Thanksgiving. Their visit was last-minute and I have never even met these people before. My fiance also has close family who live not even 5 minutes away who he wanted to visit on Thanksgiving, then drive 45 minutes to see his mom and her family. He expected me to go with him to all of this. Meanwhile, my parents live 5 minutes away from us, and my grandparents are in town, and I want to spend time with them. My dad is very sick from chemo, and it's not certain that he will even be around for the holidays next year. Holidays stress me enough as it is, and now my fiance is upset that I am staying at my parents' house all day, and that he wasn't even invited over (my dad can't handle a lot of company right now, and I didn't want to add to his anxiety). I think he's being unfair and too demanding. His dad and stepmom will be staying with us for 3-4 days, so I feel I will have plenty of time to spend with them. My fiance said, "Well I guess we'll spend Thanksgiving together next year. Have a good time." Of course I want to spend Thanksgiving w/him, but I can't invite him to my parents' house and not invite his parents, and my dad just can't handle that many people. I am so beyond irritated....but am I in the wrong? I didn't even find out until today what his exact plans were for Thanksgiving.