Then it could go back up again. I know it's only cold for a few days at a time but again, it's like a roller coaster. If it would get cold and stay cold, maybe it wouldn't be so bad.
I see people go to work in short sleeves on those cold days then wonder why they come down with a cough/cold/flu. I bundle up in layers and "peal" as need be to match the temp.