Isn't That Special?

I walk a very fine line in my professional world. I like (read insist) on getting my own way, but I don't want be seen as difficult. I speak my mind but don't want to be seen as a complainer or overly negative. These conflicts cause quite a bit of anxiety on my part. Case in point: I was offered the chance to meet with one of my lesser favorite colleagues. He is one of those people that I refer to as an idiot, and meetings with him as a massive waste of time (I often can hear my brain cells crashing and burning out of exposure to this level of banality). So, I gently inquire of this person's supervisor something to the effect of "Why am I wasting my time meeting with this person?" So the next day, he does not show up to the meeting. At first I am delighted. I feel like I have won some sort of prize. Then I get a little paranoid. I wonder - "What, so I am not good enough to meet with? not worthy of your time, you f%*ing idiot?!" Perhaps he did not want to meet with me. (Unthinkable, I know - but I have been feeling a bit insecure lately). As he was (I think) actually called into a meeting with his supervisor, I am merely grateful. So, I am trying to work on my game face and at least appear a bit more professional even when dealing with idiots.