Yea, even though the file says 522kb, the file size of the xref's is not counted. All that tallied up could make something pretty big, and depending on what is in those xref's, your computer could be having to work hard to display it all. I'd check there first, after trying Deeg's suggestion to tame Autocad's behavior.