My take is if you're cleaning up files for internal use ( background ) .. Blast the smithereens out of them. . if you're sharing use caution.
I agree 82.5% mainly because it really depends upon why you are sharing the drawings.
Again, everything I do is 2D plan view ... I remove everything that isn't absolutely necessary for the user to actually see the layout of the plans.