My mother had an expression that I loved, and still say today though only quietly in my head. If we left the chair pulled away from the table she said "You left the devil sitting in your chair".
To see chairs left out is one of my pet peeves. It really is rude, left out people either have to maneuver around the chair, and someone else eventually has to push it back in.