From my perspective I'd rather 'see' delays than not SEE data that should be there when XREF'ed; but that is just me.
Everything loads just fine with no indexing. Turning it off doesn't cause things to 'not appear'; when set it causes them to load faster (in theory), particularly under partial loading conditions.