Welcome to the swamp.
Can you attach a sample drawing?
I stack my houses as well. Foundation, floor 1, floor 2, Framing 1, Framing2, elect1, elect2, etc.
They share some layers, walls, windows, doors etc.
Works for me.
Does it make sense to Freeze all layers in the VP, then thaw the ones you want visible?