I can't stand Pete Holmes. Whenever he's on Doug Loves Movies, I just skip the episode. I even skipped last week's episode of Comedy Bing Bong.
Him on DLM is the worst way to experience him. If you haven't, try some of his actual stand up, or him on his own show (go for one with someone you really like as the guest).
I haven't listened to the CBB ep with him yet, but I doubt he is as disruptive there.