You could have the character mutter complaints under their breath or give sharp, short responses to show they're annoyed. Maybe have them sigh loudly or roll their eyes. All these combined can make the annoyance come across clearly in a comic.
One way to do it is by having the character stomp their feet or throw things around in frustration. Also, using thought bubbles with angry words can convey annoyance. For example, 'This is so annoying!'