That file size is not unheard of. I know of civil background files are are in excess of 100mb. I've worked with steel models that were 55mb. Since you are working with others, ask them if they are having problems and what kind of machines they are having. One thing to remember is that for each xref you bring in, Autocad needs memory requirements and such for EACH file brought in. Basically, think of Autocad running the same number of times you bring in an xref. When you have large files such as this, you are going to need goobs of Ram. Loading the file in seems to be your biggest problem right now. Once you achieve them to load in, you should be ok.
One trick that you can use, when you open your file do a cancel to cancel the regen of all the xrefs. I would remove any xref's that are not pertinent to what you need to show. Then you can also hit the layer dialogue box to freeze layers that are off. Freezing them will remove them from the database, thus from the regen. Don't just turn them off. I would also ask all the other vendors to purge their drawings and see if they can do anything to condense their file size. I cautiously urge you to be careful if you try to condense their files as there could be repercussions if some of their data does not show up properly on your drawing.
pieter