No, the “bugger” in “full of bugger all” is not swearing. Nor would it have been swearing for Aziraphale to have publicly discussed the Bugger All This Bible.
People have looked on this post askance and maintain that it is swearing. However, they are wrong. I know they are wrong, because when Terry Pratchett and I turned up late one night in late 1990 to do the Hour 25 show with Joe Straczynski on KPFK, the programming director came over and told Terry and me that we were NOT ALLOWED TO SWEAR. She explained that English people often swore and that no swearing was allowed to occur on live radio or they would lose their broadcasting license. So Terry and I naturally and immediately asked if we could say bugger, because there was the Bugger All This bible in the book, and we might mention it, and she went away to ask. She was gone quite a long time, but when she came back she said she had taken it all the way to the top, and it wasn’t swearing, and we could say Bugger on the radio.