While Warburton may not be that old, doing a superhero show is pretty taxing. On top of the physical issues of being in that suit all the time (he was literally molded into it in the morning and couldn't get out, even to pee, all day). I don't blame him for not wanting to do that again.
I'd still love to see Warburton be a superhero, though. He's my first choice for Shazam (Captain Marvel).