I used to be on the receiving end of these comments quite a lot on 'Daddy days' when the boys were younger. Less so these days, but I think there's still a general sense if I am alone with the boys for a weekend that it's a miracle if they come through it unscathed.
I maintain that many of the Dads I know are fantastic, and that many parents - Dads and Mums alike - inevitably have the odd moment they're less proud of. Incidentally, on that note, it's well worth listening to Adam Buxton's interview with Doc Brown where he talks about children's books, and in particular 'Not now, Bernard'. If you haven't read it, do: it's a splendidly dark exploration of parental neglect.