I have checked for routines that will remove underlays of all sorts that are unreferenced (not actually inserted into the drawing) or that are unloaded (inserted, but have been unloaded for various reasons). I had found routines that claimed they would do this, just to have them remove all PDFs for example, not just unreferenced/unloaded ones or some would just remove unreferenced, but leave unloaded. In all fairness, many of these routines were from 2013 or earlier, so the way AutoCAD works may have changed since then.
So I have taken to writing my own, which is indeed functional. I would post it in the Show Your Code section; however, there is a bug in it that I cannot figure out. It works the first time just fine, but in many cases if you try to run it a second time on the same drawing, it will return the following error:
Automation Error. Object was erased
I believe that aside from this error, it should work on any version of AutoCAD that can handle PDFs.
Due to the size of the code, it is uploaded as an attachment here.
I tried to give credit where credit is due; however, some of the code was so old that even the person posting didn't know where it came from. So, if you see any code that is yours and it is not credited, please speak up.
ETA:
Additional bugs have been removed
One key detail to note is that changes are not reflected in the XREF palette until you save and close the drawing and reopen it. The changes should be reflected in the classicxref dialog box right away, but sometimes they aren't. When in doubt, save the drawing, close it and reopen it.