Topic: Get the most out of your Ram if you have 3gig or more in boot.ini file

[sourdough]

Did you know that if you have more than 2gig of ram your xp or vista can't access it unless
you do some changes to your boot.ini file.... Read on

Warning, be careful, you have administrator rights and Blessings of IT where it applies.
This is fully documented below from Autodesk and Microsoft. I just didn't know I had to do this
to get use of the ram I already have.

If this helps you and I'm glad to help get more out of your computer. If not,
don't make these changes to your machine. I'm sure you know what I mean here.
I have a backup Ghost 12 image to fall back on. So, I was 100% sure of what I was doing.

Mike



I contacted my dealer for my Civil3d/LDC 2008 and was told something from there Cad
Camp in Texas over the last few days where they are learning Civil3D 2009.... this worked on my machine I use at home/business
and thought it would be beneficial to share. As always, this isn't something to play with
unless your sure. This is my bio.ini file

[boot loader]
timeout=5
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional " /fastdetect
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional 4GB" /4GB /fastdetect
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional 3GB" /3GB /fastdetect




this line was the original:
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional " /fastdetect

this allowed me if anything didn't work to use when it booted next.

I have 4 gig of ram and used this one with success:
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional 4GB" /4GB /fastdetect

Here are the instructions I got from my authorized autodesk dealer:

3GB switch and AutoCAD® Civil 3D®
   Publish date: May-31-2007
   ID:   TS1071001

  Applies to:

AutoCAD Civil 3D 2008
 
AutoCAD Civil 3D 2009
 

 

 

 
 


Issue
You want to know if there is a way to increase the memory usage of Civil 3D® to improve performance when working with large datasets.

Technical Solutions
By default, the Windows operating system limits the amount of physical memory available to applications to 2 GB. It reserves the remaining memory for the operating system itself. However, if you have more than 3 GB of physical memory, it is possible to increase the amount of memory available to applications, such as Civil 3D®, to 3 GB.

Autodesk has conducted some preliminary testing on this configuration and has found some improvement in performance when working with larger datasets.

To enable the 3GB switch on Windows XP

1.        Right-click My Computer. Click Properties.

2.        In the System Properties dialog box, click the Advanced tab.

3.        On the Advanced tab, under Startup and Recovery, click Settings.

4.        In the Startup and Recovery dialog box, under System startup, click Edit.
The Windows boot.ini file will be opened in Microsoft® Notepad. 

5.        Save a backup copy of the boot.ini file on your computer, in case you need to revert back to the original version of the file.
Note: The contents of the Boot.ini file may vary from computer to computer.

6.        Select the following line in the boot.ini file:

multi(0)disk(0)rdisk(0)partition(2)\WINDOWS="Microsoft Windows XP Professional" /fastdetect

7.        Copy (Press CTRL+C) this line and paste (Press CTRL+V) it immediately below the original.
Note: Your string may be different from the string shown. Be sure to copy the string from your boot.ini file, not the string shown here.

8.        Modify the copied line to include “ /3GB”, as shown in the following example.
Note: Do not overwrite any existing lines.

multi(0)disk(0)rdisk(0)partition(2)\WINDOWS="Microsoft Windows XP Professional 3GB" /3GB /fastdetect

9.        Save the boot.ini file and exit Notepad.

10.     Click OK to close each dialog box. 

11.     Restart your computer. 

12.     During startup, select the 3GB option. If you do not select the 3GB option, the system will default to the 2GB memory setting.

If there are problems during startup, you may need to update some of your drivers.

To enable the 3GB switch on Windows Vista™

1.        Right-click Command Prompt in the Accessories program group of the Start menu. Click Run as Administrator.

2.        On the command line, enter "bcdedit /set IncreaseUserVa 3072"

3.        Restart the computer.

To turn off the 3GB switch on Windows Vista

1.        Right-click Command Prompt in the Accessories program group of the Start menu. Click Run as Administrator.

2.        On the command line, enter "bcdedit /deletevalue IncreaseUserVa"

3.        Restart the computer.

For more information on the 3GB switch, refer to the following Microsoft MSDN article:

http://msdn2.microsoft.com/en-us/library/ms791558.aspx

If you experience any problems when using the 3GB configuration, you can switch back to the original configuration.


I hope this helps those who have the horsepower in ram to see what I saw, a faster xp install. I also
have the XP SP3 RC2 refresh installed too.

[Tuoni]

To take advantage of the 3 GB available to user-mode programs, the program must be linked with the /LARGEADDRESSAWARE option.

I think the speed improvements you saw were possibly a little to do with the fact that you expected to see speed improvements?

Seems to me for that link that all it's doing is halving the Kernel's available RAM to 1GB (intended for driver testing)...

[sourdough]

Thanks for the comment. I noticed a change in my loadup time mostly of my Civil3D and LDC 2009. It wasn't huge, but my dealer when he went to the recent cad camp for 2009 said that it was recommended for people who have more than 2 gig of ram and Civil 3D. My system is very fast to begin with. If this helps someone then great. I also have sp 3 on my wp pro so maybe the combination did some help as well. I'm using a intel q9550 @2.83GHz 1333 fsb, with 4 gig of 1066 ram... it all may be just a combination of things.. not sure.  I'm no expert on the inner workings of XP and won't touch Vista right now... again thanks for your thoughts.

Mike

[mjfarrell]

It would appear that one would need to add a line in there specifically for your application NOT the OS, and that the application must be able to address the higher memory, and I'm not sure if C3D is or does. Because this seems to setting aside memory for the OS and not for C3D.   Tuoni, is that your read on this as well? Not saying that giving XP more overhead is a bad thing, just trying to clarify my understanding of the matter.
/USERVA=
This switch is only supported on Windows XP and Windows Server 2003. Like the /3GB switch, this switch gives applications a larger address space. Specify the amount in MB between 2048 and 3072. This switch has the same application requirements as the /3GB switch and requires that the /3GB switch be present. Applies to 32-bit systems only.

The above from:http://technet.microsoft.com/en-us/sysinternals/bb963892.aspx

[sourdough]

I also found this in the system properties that wasn't there before so it did make this change. It may just be that the /3G just might only be specific to a program that uses the call to Civil 3D. Maybe... would have to ask my dealer if they can ask Autodesk about this feature and why they were told to use it under the circumstance of having more than 2 gig of memory. See the Physical Address Extension... in the attachment. below

[mjfarrell]

I might need to find a way to benchmark this. I think a user would still get better performance on a system running a second physical disk for the Windows Page files, instead of using this switch in the boot.ini file.

[sinc]

They aren't mutually exclusive.

And for the pagefile, if adding only one additional disc drive, I would actually expect performance to be increased more by using the second drive as a data disk, and leaving the page file on the primary disc where the OS and C3D is installed.  Of course, with three discs, the best would be OS/C3D on one, data on another, and pagefile on the third, but if only two drives are involved, I would expect to get a better increase in speed by splitting up pagefile and data, rather than by splitting pagefile and OS.

[mjfarrell]

That might be what one would expect however; if one searches the TechNet for Workstation performance tuning this is a primary task. And the OS and the swap file enjoy residing on separate physical discs. This being the case the OS, and the Application (C3D) performance are not impacted by being on the same disc. However it is the swapping of the virtual memory page file that impacts system performance the most if they are on the same disc.
For optimal performance it would indicate a 5 disc raid with the page file striped across all five disc. However most aren't going to put a 5 disc raid in their cad station when most balk at paying more than $400 (USD) for a video card. For an unscientific analysis, if your hard drive light stays on more than off, you should have another disk drive.

[Tuoni]

It would appear that one would need to add a line in there specifically for your application NOT the OS, and that the application must be able to address the higher memory, and I'm not sure if C3D is or does. Because this seems to setting aside memory for the OS and not for C3D.   Tuoni, is that your read on this as well? Not saying that giving XP more overhead is a bad thing, just trying to clarify my understanding of the matter.
/USERVA=
This switch is only supported on Windows XP and Windows Server 2003. Like the /3GB switch, this switch gives applications a larger address space. Specify the amount in MB between 2048 and 3072. This switch has the same application requirements as the /3GB switch and requires that the /3GB switch be present. Applies to 32-bit systems only.

The above from:http://technet.microsoft.com/en-us/sysinternals/bb963892.aspx

I'm sorry, for some reason this thread wasn't flagged in my email!

Yes, that is my read on it - the application doesn't know anything about it unless you tell it otherwise, according to the line I originally quoted from that tech doc. 

Also...

I also found this in the system properties that wasn't there before so it did make this change. It may just be that the /3G just might only be specific to a program that uses the call to Civil 3D. Maybe... would have to ask my dealer if they can ask Autodesk about this feature and why they were told to use it under the circumstance of having more than 2 gig of memory. See the Physical Address Extension... in the attachment. below

I think you're misunderstanding PAE.

In computing, Physical Address Extension (PAE) refers to a feature of x86 and x86-64 processors that allows more than 4 gigabytes (GB) of physical memory to be used in 32-bit systems, given appropriate operating system support. PAE is provided by Intel Pentium Pro and above CPUs (including all later Pentium-series processors except the 400 MHz bus versions of the Pentium M), as well as by some compatible processors such as Athlon and later models from AMD.

Also note my screenshot.

Like I say, to be perfectly honest I think this is another one of those "do x and it will make your machine run faster!" which I've been hearing since day 1 with XP - a lot of which are absolute nonsense, but very obviously show "improvement" to anyone who implements them.... except in terms of benchmarking, it really doesn't.

[sinc]

I think my favorite is people who disable the pagefile in order to improve performance.   

[sourdough]

Morning all
    I found this tid-bit about using the 3G switch... from the help inside of of 2009 C3D:

Performance can also be improved by adding memory to your system. This is especially true when working on larger models.

The system requirements for AutoCAD require at least 512 MB of physical memory (RAM) for working in 2D. For creating and working with 3D models, at least 2 GB of RAM is required.

The size and complexity of a model often defines how efficiently an application runs. If you notice increased hard drive activity, it means that physical memory has been exceeded and data is being passed to a swap file (virtual memory).

A swap file is an area on the hard drive that Windows uses as if it were physical memory (RAM). The swap file size is basically a limit which restricts the total virtual size of the AutoCAD process. A good rule of thumb for configuring your swap file is three times the amount of physical memory on your system. This usually sets the limit high enough that AutoCAD doesn't run out of swap space.

Typically, systems are limited to a maximum of 2 GB of virtual memory. On some systems, you can set the /3GB switch that allows applications to use up to 3 GB of virtual memory. Before attempting to set the switch, you need to find out the following:

If your system supports the switch.
If other device drivers (video card, plotter drivers, etc.) may be affected if the switch is set.
How to set the switch.
Refer to the Microsoft website for details regarding the /3GB switch.

this with some work now might yield some more results, if I find something after doing this suggestion of changing the swapfile size to 3times my physical memory of 4gig should be interesting, anybody else done this step yet?

Mike

[Tuoni]

<snip>

this with some work now might yield some more results, if I find something after doing this suggestion of changing the swapfile size to 3times my physical memory of 4gig should be interesting, anybody else done this step yet?

That is actually somewhat of a red herring in Windows XP,

any modern OS automatically uses virtual memory addressing (then turned into a physical RAM address by hardware), only the kernel will bypass this and address the RAM directly.  Also, AFAIK, windows will dynamically increase pagefile size if needed.

All processes (e.g. application executables) running under 32 bit Windows gets virtual memory addresses (a Virtual Address Space) going from 0 to 4,294,967,295 (2*32-1 = 4 GB), no matter how much RAM is actually installed on the computer.
 
In the default Windows OS configuration, 2 GB of this virtual address space are designated for each process’ private use and the other 2 GB are shared between all processes and the operating system.  Normally, applications (e.g. Notepad, Word, Excel, Acrobat Reader) use only a small fraction of the 2GB of private address space.  The operating system only assigns RAM page frames to virtual memory pages that are in use.
 
Physical Address Extension (PAE) is the feature of the Intel 32 bit architecture that expands the physical memory (RAM) address to 36 bits (see KB articles 268363 and 261988).  PAE does not change the size of the virtual address space, which remains at 4 GB, just the amount of actual RAM that can be addressed by the processor.
 
The translation between the 32 bit virtual memory address used by the code running in a process and the 36 bit RAM address is handled automatically and transparently by the computer hardware according to translation tables maintained by the operating system.  Any virtual memory page (32 bit address) can be associated with any physical RAM page (36 bit address).

I guess from this that the /3GB switch increases "2 GB of this virtual address space are designated for each process’ private use" to 3GB and decreases " the other 2 GB are shared between all processes and the operating system" to 1GB.

Can you do any benchmarking?  I'd be interested in seeing what sort of actual gain this gives, if it works.

[mjfarrell]

Yes this is a RED herring and I do not believe that it works as "advertised" nor as the intended (or believed) effects the user is suggesting it will provide.

[sinc]

So did anyone ever do any benchmark testing on this 3GB switch?

From my understanding, it MAY help, for very specific data sets.  For general usage, I would expect that, if it has any impact at all, that impact would be negative.

Setting the 3GB switch does more than cut the address space available to the Kernel in half.  It also cuts the size of many important system resources are cut by more-than-half, such as the Free Memory Page Table, which maps virtual memory to physical memory.  It also reduces key resources like the nonpaged pool, which can have dramatic effect on device drivers, among other things.

From reading Microsoft's site, it appears that this option was originally created with certain server applications in mind.  Some server-based applications like to use lots of memory, but have little need to access peripherals such as video cards.  So cutting back system resources, in favor of applications, makes sense, for those applications.  However, AutoCAD is not such an application - it actually hits various computer resources rather hard - and I would not expect it to benefit from the 3GB switch most of the time.  The exception might be certain operations involving very large data sets, where the increased address space given to C3D might improve the application performance enough to more-than-compensate for the general performance hit created by reducing the system's resources.

As another factor to consider, applications that are not "large-address aware" will ignore the extra 1GB available to them.  However, the system will still be restricted to 1GB, if that 3GB switch is set.  This can mean that, by setting the 3GB switch, you are effectively disabling (and wasting) a full GB of your RAM for almost everything you do on your computer, except for certain things in C3D.

But all this is just "theory".  Some significant benchmark testing would be required to see what is really happening, and I haven't tried doing any of that.  My instinct says the 3GB switch is not a good idea for most users, but I'm curious about what some real benchmark testing would show...

[numa]

32 bit apps can't address more than 2gb of memory.

XP can't see your 'full' 4gb of ram since it uses the top 512mb-1gb for device memory addressing.  IE, video ram.  So, if you have a machine with 512mb on the vid card, it gets carved off the top so to speak. 

64bit os can address more memory, but, since even c3d2009 is a 32bit app it is sorta wasted.  The slick thing is, is that vista will use more memory if you have it....  saw a good article on this somewhere, I'll see if I can find it, I think it's on ars-technica or anandtech or tomshardware. 

here's a good snippet... from http://www.anandtech.com/gadgets/showdoc.aspx?i=3034&p=3

"As it turns out, it's possible and actually quite easy to move the 2GB barrier by increasing the size of the user space, but at the cost of reducing the size of the kernel space. Under Windows XP, this is the fabled "/3gb" switch for boot.ini, and for Windows Vista it's the "IncreaseUserVa" option in BCDedit. By using these options applications can use more than 2GB of virtual address space (generally up to 3GB), and ideally this would be the end of the article.

Unfortunately this is not the case as there are problems on both the application and kernel side of things. On the application side, a common poor programming practice has been to always assume that an application will only be dealing with 2GB of user space; code that makes this assumption will likely error if more than 2GB of user space is actually available. This is avoidable by following proper programming practices, but as a safety precaution even with additional virtual address space allocated to user space Windows still defaults to limiting an application to 2GB. Only finally, if an application indicates to Windows that it is capable of handling more than 2GB, via the "/LARGEADDRESSAWARE" flag, may it have access to any space above 2GB.

As for the kernel, having had up to half of its space taken away must now find a way to live in a smaller space. The (in)ability of any specific system/Windows configuration to deal with this is why the 3gb switch is considered dangerous, seldom recommended, and just generally a bad idea. The biggest culprit here is drivers that run in kernel space. Like applications, they may assume that there's an entire 2GB of address space to work with, except unlike applications this space gets smaller instead of bigger."

[sinc]

Well, yeah...  That basically just repeated what's already been said in this thread.

Autodesk claims that Civil-3D is large-address-aware.  The exact quote:

Autodesk has conducted some preliminary testing on this configuration and has found some improvement in performance when working with larger datasets.

This seems to be causing lots of people to set the 3GB switch, because they are under the impression that C3D will run better.  I have serious doubts about this, but that's why I was asking if anyone has actually done any benchmark testing under various conditions.

One of the unknown factors is just what is the impact of limiting the Kernel to 1GB?  After all, C3D runs just fine for most things on a system with only 2GB RAM total.  So just how big a hit do we actually notice if we limit the system to 1GB, while we let the application use 3GB?

Because of the way system resources like the Free Memory Page Table get dramatically cut in size with the 3GB switch, I am leery of this option.  But I'm still curious about actual bench results.  I've been too busy with other stuff to try doing my own testing on this point, but I might get around to it eventually.

[mjfarrell]

I think this falls under the category of;

Just because you can

Does not mean you should.


This is a red herring. The real issue is product stability.

I'll type it real slow.....and loud for those in the back of the room...

S    T     A    B   I   L   I   T   Y.


3GB Switch or no, the random crashes have to go!

[numa]

yah, I was mainly trying to quote the portion of that article and the various semi-anecdotal evidence that the /3gb switch causes some problems in some of the random software that we use on a regular basis (winamp comes to mind) at any moment I have quite a few silly programs running, and it's sorta better if my machine doesn't flip out when I switch songs. 

[John Mayo]

This is a really good and informative post. Sourdog thanks for starting it, Michael, Sinc & the rest thanks for adding.

With all this said, noting that I agree that the 3GB switch does nothin' noticable for me, we will be loooking a new PC's.

Is the overall consensus to add a second HD for swap space? What about Don R's old trick of creating a partition on a drive & using that partition for swap space alone? With HD's being made with hundreds of GB, what would Windoze do with 250 GB for the swap file?

Forget the 3GB switch? What about all these boxes with 4 Gig & 8 Gig of ram? Is this just a waste? Why bother with more that 2GB?

Quad Cores?

John

[sinc]

Creating a partition on a hard drive for swap space is almost always a bad idea.  In fact, I can't think of a time when it would be a good idea.

You'll see far more benefit from setting up two drives in a RAID 0 array than you'll see by using one as the primary, and one for swap.  Use 10K Raptors, or 15K SCSIs, or some of the new vertical-write drives, configured in a RAID0 as your primary drive (with a MOBO that can handle that rate of data transfer), and you'll be amazed at the difference.

If you are running a 32-bit OS, then yes, those boxes with 8GB of RAM are wasting the bulk of it.  If you want to start using large amounts of memory, you should move toward a 64-bit OS.

If you are running 32-bit Windows XP, the difference between 3GB RAM and 4GB RAM tends to be rather insignificant for most operations.  It takes special circumstances to really notice any sort of difference between 3GB and 4GB, at least for a CAD workstation.

[Swift]

RAID 0 is the way to go

I just built a machine using an EVGA 780I nforce board and 4 10k WD Raptors in RAID 0

Loud but VERY FAST!

[John Mayo]

Thanks Sinc.

We will be buying at the end of the year so I have time. The machines we have were purchased 4 years ago for LDT.

We will more than likely get Vista 64 (perhaps a dual boot with XP Pro) unless I hear unbelievable horror stories. I have heard of some issues but have not heard any drop dead horror stories yet.

Does anyone here have any?

I agree that fast HD's are always a must but I still wonder about the swap file. I think Michael posted that Windoze prefers the swap on a separate drive but I don't think anyone was comparing the configurations with Raid in mind. Any other advice knowing that our users can fill the primary drive with email, music & other essential items pretty fast? Would you still go Raid 0? This fact has left me trying to send the swap file to another drive in the past. I was planning on two 250 GB 10K RPM HD's hopefully SCSI if Dell will put a SCSI card in the box.

"If you are running 32-bit Windows XP, the difference between 3GB RAM and 4GB RAM tends to be rather insignificant for most operations." Does the 3 GB help over the 2 GB if the switch is not used? What is windoze doing with the extra gig if the switch is off? I'm guessing it's just a waste then.

[mjfarrell]

As you will have the hardware either way; build one with the swap file on the second drive, and one set to raid 0. You can always go the other way if you don' like the other.
Put the 4GB of RAM in as most memory prefers to travel in pairs.
Although at this juncture I would most likely get a MB that will hold 6-8 in the future.
Run 32 bit for now, as even running this on 64bit XP pro has shown to be no better and without the lack of drivers issues.

In the nearterm autocad does not really know what to do with the extra headroom anyway.

[numa]

http://www.newegg.com/Product/Product.aspx?Item=N82E16822136260&Tpk=VelociRaptor

are a pretty good deal at this point.  (i know still pricey).  Raid-0 is kinda neat, I run it on some machines, and it can be quite fast.  I don't know that it is a super cure-all and extra headache of having a system that has such a failure point (no recovery, no redundancy) is necessarily worth it for all users.

SCSI is interesting though it gets quite pricey even now and if you are looking at a very serious raid install, I would recommend going for one of the new 3ware pci-express SATA hardware controller cards, maybe even one with some onboard memory.  figure spending 300-500 dollars for such a card, 3 hundred per drive.  Some of the 3ware controllers allow you to combine raid 0 with raid1, (requires a minimum of 4 hard drives) to give you  a performance and reliablity advantage.  Then there's Raid-5 though you need a lot of drives for it to start paying a performance dividend.  This stuff really can get very expensive, and I don't know that it is worth it.  really.  

Some SCSI drives are very specifically tailored to server use, some are tailored for desktop use, there is some researching that has to be done to figure it out (cache sizes, and seek method).

Another consideration is that the solid state drives are improving, both in speed (which they lack) reliability (which has been questioned), and capacity (which is still dismal) and at some point in the near future will overtake mechanical drives (2 years, 5 years who knows).  When these drives improve, they will become the 'next great thing' that everyone will want and you might kick yourself for wasting money on a system that isn't satisfying at the moment.  

I have found the best performance/value/hassle balance to date on a system with a stripped down xp install (gutted everything I didn't need).  I use nlite to tweak stuff at install, add the drivers I need for my systems, slipstream in SP3, and generally disable stuff that is obsolete, or not needed for our small office.  The nice thing about slipstreaming is you can make your installs unattended, so when a machine starts to have a problem, reloading windows at least is very fast.  Course there is still the installing your adesk products...

And as alluded to  above, most of the 'tweaks' that are done by these randomn xp tweaker pro super whatever programs are totally useless.  I sorta like slipstreaming because it lets me have an up to date install without a bunch 'system restore points' and what not.  slipstreaming also gives you a good baseline (once you get everything working) to start from at a diagnostic standpoint. 

[sinc]

OK, maybe some further explanation is in order.

Let's look at the way this is handled in WINDOWS XP 32-bit ONLY.  As we shall see, if you are running a 64-bit OS, this is all far less important.

But to start, I should point out that most people way over-think the pagefile.  If we could plot the number of hours spent on various topics, vs. the net gain in computer performance realized, I think the pagefile would rank at or near the top when it comes to wasted effort.

Now that I've gotten that out of the way, though, here's the important details (and remember, this is for 32-bit XP only):

All Windows applications run in something called a "virtual memory space".  This is ALWAYS 4GB, regardless of the amount of RAM you have in your system.  Each application has it's own 4GB "virtual memory space", regardless of how many other applications are running, and this virtual memory space is ALWAYS 4GB.  This is important point #1.

Now, some of this address space is "reserved" for the system.  Depending on whether or not you have the 3GB switch set, it is either split evenly between application and system, or 3GB go to the application, and 1GB goes to the system.

Now for important point #2:  the part allocated to the system is shared among every application that is running on your system.  The rest of the "virtual address space" (either 2GB or 3GB) is available to your application.

Important point #3: the system will automatically map this virtual address space to available RAM.  At any given time, only a portion of each application's virtual memory space will actually be taking up any RAM.  The rest of the virtual address space may be unused and unallocated.  Or, if it hasn't been used for a while, it can be sent from RAM into the pagefile, so the RAM can be used by another application.  Then, when that part of the virtual address space is needed again, it is swapped back from the pagefile into RAM.

Keep in mind that EACH APPLICATION has its own 4GB virtual address space.  So as you start more applications, Windows creates more 4GB virtual address spaces.  Each of these 4GB virtual address spaces gets some portion of it mapped to the physical RAM; the rest remains unallocated and unused, or gets swapped out to the pagefile.

Now let's look at how this impacts our system.  If we have 3GB of RAM, and we DO NOT use the 3GB switch, then each application can use a maximum of 2GB of RAM at any one time.  This means that we can theoretically have the entire allocated memory for AutoCAD in RAM all at once, and we are only using 2GB of it.  The remaining 1GB is shared between the OS and other applications.

If we raise the amount of RAM to 4GB, but leave the 3GB switch OFF, then Autocad can still only use 2GB of memory at once.  This is because it is limited to only 2GB of the virtual address space, so it doesn't matter that there is more RAM available.  The OS won't let the application address any more than 2GB.

Enter the 3GB switch.  This now allows the application to address up to 3GB of its VIRTUAL ADDRESS SPACE.  Remember, however, that some of this 3GB may be swapped out to the swap file at any one time, depending on how many other applications are demanding memory, and how much they are demanding.  So in theory, it would allow an application to run faster/better.  But the cost is to remove memory from the system.

With some applications, this is not an issue.  But the memory allocated to the system is also used by device drivers, and applications such as AutoCAD hit certain devices (the graphics card, in particular) very hard.  So limiting the resources of the system can have unintended side-effects, that keep the application from making full use of the extra RAM it now has access to.

Earlier, I mentioned that I had some concern of the paging tables, and other resources in the system.  This may not be an issue.  The PAE is designed to work with 32-bit Windows 2003 Server running up to 32GB of RAM, and turning on the 3GB switch means that it can now only access 16GB of RAM.  So if Windows 2003 Server can still address 16GB of RAM with the 3GB switch enabled, then I don't think it would be a problem for an XP machine with only 4GB of RAM.  The real concern would probably be the effect (if any) on the graphics drivers.

Now all of this probably answers another question you had - what would Windows do with 250GB of swap space?  The answer is: waste most of it.  Remember, each application is limited to only 2GB or 3GB total of usable address space.  So, as you run more and more applications, more and more virtual address spaces get created.  But in order to impact the pagefile, the application has to actually allocate the memory, then the total RAM requested by all applications has to exceed the amount of RAM available on the system, to trigger paging.  (Well, actually, a bit of paging will happen in any case, so this isn't strictly true, but it's true enough.)  So if you keep launching more and more applications, you would gradually eat into the pagefile.  But you would have to run an awful lot of memory-intensive applications to go through 250 GB of pagefile, and you would probably run out of other system resources long before you used a significant fraction of a 250GB pagefile.

[sinc]

Now for the configuration issues - multiple drives, where to put the swap, etc.

We can split disc access into three major chunks: loading/swapping application code, accessing application data, and general data swapping.

Application code undergoes swapping, just as data code does.  Your application can be loaded into memory, but if a chunk of the application code goes a long time without being run, it gets swapped out.  Application code is always swapped out to the place it came from on the drive, and takes up no space in the pagefile.  However, when it needs to be swapped back in, you see a performance hit similar to the first time you run the application, since the code must once again be loaded from disk.

Your application data is relatively self-explanatory - this would include your actual DWG file, and the temp files Autocad creates, etc.  This data is read from disk, or written to disk, in direct response to user operations.

Then there is data swapping.  This is the "behind-the-scenes" action, where Windows automatically moves data to/from RAM and the pagefile, in order to make sure that operations that are currently running have enough RAM.

To get absolute best performance, each of these three categories of action would hit a separate drive.  This is where we start to see Michael's recommendation of putting the swap on its own drive.  If it's on the primary drive, then application loading/swapping and data swapping will hit the same drive, possibly at the same time, causing the disk drive to rapidly jump back and forth between disparate locations on the disk.  This is what we want to avoid.  If we are also storing all our DWG files and temp files on this same drive, we see an even bigger hit in performance, as all these operations may end up in a fight for simultaneous access to the drive.

Now, in an office situation, many people have their DWG files on the server.  This off-loads one item, leading to a potential increase in performance (unless you have a slow network).  So now we're looking at the application loading/swapping, and the pagefile access.

Now, if we want to use only two drives, what is the best way to handle this?

Method one is to use one drive for the primary, the other for swap.  This means that the majority of the swap drive will be unused, because our swap file is probably less than 4GB in size.  But it can be used as a generic storage drive.  This now gets our swap space hitting one drive, while the other is reserved for application loading/swapping, and temp files.  This is an ideal situation for someone with not-very-much memory, such as 1GB of RAM, where swapping happens pretty much continuously.  But if you have 4GB of RAM, the benefits are much less, because your system doesn't need to swap any where near as much.

So now let's compare that to a RAID 0.  With the RAID 0, we are now hitting the same drive for application loading/swapping, temp file usage, and swap space.  But if we have a lot of RAM, we won't notice much of a difference having the swap space hit the same drive as everything else, because our computer won't swap much.  However, we'll notice a HUGE difference in application loading/swapping and temp file usage, because of the RAID 0, which typically results in about a 30% gain in overall disk performance over a single drive.

This is why I say you will get far more bang for the buck using two drives in a RAID 0, as opposed to dedicating one as a swap drive.  If you want to throw a third drive in there, then by all means, set up the first two drives in a RAID 0, then use the third drive for swap and overflow data storage or backup.  But in general, you will notice an overall increase in EVERYTHING if your primary drive is a RAID 0.

The big rule in all this is NEVER NEVER NEVER use a removable (i.e., external) drive for swap space.  If that connection gets interrupted, or you try to startup your system while the drive is powered down or disconnected, bad things will happen.

Now for the point Numa brought up:  which RAID?  First off, RAID arrays should NEVER be used for redundancy or recovery.  That's what backups are for.  A RAID 1 is useless, except for in very limited situations, typically only for servers that should not fail unexpectedly.  In other words, if that server MUST remain up as much as possible, you may want to use a RAID 1 or RAID 10 or RAID 5.  Those systems are designed so that the system can keep running (for a time) after a disk fails.  However, it is designed only for short-term redundancy, basically, only long enough until the IT person can get there, shut down the server in an "official" way, and replace the failed drive.  But rebuilding ANY sort of RAID array is more trouble than any sane person would want to deal with, so it should not be considered a path to data redundancy/recovery.  Use a good backup system instead.  And when you stop looking toward RAID as something that provides redundancy or backup, then a RAID 5 (or even a RAID 10) is obviously not worth the expense.

As far as a RAID 0 being more trouble than it's worth, I strongly disagree with that.  The argument against using a RAID 0 is basically that, with two drives involved, the chance of a drive failure is twice as great as when only one drive is involved.  However, these days, disks are good enough that drive failures happen very rarely.  I don't view this as a big concern.  Once upon a time, it was a valid concern, but not any longer (in my opinion).  If one of your drives fails, then you just deal with it the way you would deal with it if you had only the usual single-drive system, and your single drive fails.

Personally, I really like having 20-second startup times for Civil-3D.  Not to mention, software updates and installations go at blinding speed.  And that doesn't happen unless you have a fast primary drive, namely multiple fast drives in a RAID 0.

[sinc]

Oh, I forgot the final point - using a separate partition for the pagefile.

This idea came about in response to the way that Windows, by default, will auto-manage the size of the pagefile.  Typically, for most users and applications, this is the best way to go.  Most people overthink the pagefile, and create problems when they try to mess with it.  So for most people, simply leaving Windows setup in the default way is the best.

Where this causes an issue is that, as Windows resizes the pagefile, it can become "fragmented".  This is the same as what happens to normal files, as data gets appended to them - part of the file sits in one place on disk, another chunk in another place on disk, and the rest in a third place (as an example - in reality, some files may even be split up into as many as hundreds of fragments).  In order to read the file, the drive must jump around to many different places on the disk, resulting in lots of "seek time", and decaying performance.

When the pagefile gets fragmented, operations that hit the pagefile can take longer to perform, as the disk goes through lots of "seek time".

The theory behind creating a separate partition for the pagefile is that, since the pagefile is the only thing in the partition, the pagefile cannot become fragemented.  Unfortunately, creating a partition for the pagefile forces the pagefile way off to one edge of the disk.  This can result in a situation where, in order to access the pagefile, your drive must continuously bounce back and forth between the two partitions.  This can increase the amount of "seek time" by A LOT.  In effect, you are constraining the hardware, and telling the disk drive you know how it should be allocating its resources.  These days, disk drives are so smart that they are undoubtedly better at this task than a person, so best performance is typically achieved by letting the disk drive itself figure out where to put everything.

But when we do this, we still may want to make sure that the pagefile does not get fragmented.  Now we get to another of Michael's recommendations - don't let Windows automatically resize the pagefile, and set the minimum and maximum sizes of the pagefile to the same thing.  This guarantees that the pagefile will not become fragmented.

Of course, the pagefile may already be fragmented by the time you make this change to your pagefile settings.  In that case, you will want to defragment it.  Unfortunately, the pagefile cannot be defragged while it is in use, and it is always in use if Windows is running.  So in order to defrag your pagefile, you CANNOT use the Windows defragmenter.  (The pagefile is one of the things that shows up as a giant green "unmovable block" in the standard Windows defragmenter.)  There are various utilities (some of them free) available on the web that will defrag your pagefile.

This alternative is FAR preferable to the option of creating a separate partition for your pagefile.

[John Mayo]

Thank-you VERY much Sinc. I know it take time to post threads of this magnatude.

I appreciate your efforts greatly. I have a much better understanding.

John

[John Mayo]

One more thing. A Windoz user cannot turn off the page file to defrag & then restart it?

Not trying to be a wisenhiemer but there is a toggle to turn the pagefile off. Does Windoz still maintain a pagefiel even if it is turned off?

[Mark]

That's some great info Sinc but you know I have to disagree with you when it comes to raid 0.

Raid 0 is fast but you have to remember if one goes your SOL! Like Sinc said, don't rely on a raid 0 setup for data retention.
 

[John Mayo]

Raid 0 would worry me if it was setup on a server or a workstaion that stored data. For our use I am not too worried. I typically store a HD image that can be placed on any workstation if it fails. I have yet to use one & we have a pretty good backup system that can also restore the workstaions.

But you are correct Mark. If you are worried about the data on a particular machine Raid 0 may not be the answer. If two drives are set up with Raid0 and one drive fails all data is lost.

[mjfarrell]

One more thing. A Windoz user cannot turn off the page file to defrag & then restart it?

Not trying to be a wisenhiemer but there is a toggle to turn the pagefile off. Does Windoz still maintain a pagefiel even if it is turned off?

One could do a swap file shuffle, defragment the drive, then create a NEW contigous swap file space.
Remove, the old one. De-fragment the entire drive.
Then to get back to the original swap file location and size do the reverse.
If one really wanted to.

[sinc]

That's some great info Sinc but you know I have to disagree with you when it comes to raid 0.

Raid 0 is fast but you have to remember if one goes your SOL! Like Sinc said, don't rely on a raid 0 setup for data retention.
 

Yeah, but how often does one go?

We have a lot of machines with RAID 0 at this point, and have never had one fail.  And even if it does, all the important stuff is on the server with redundant backups.

[sinc]

One more thing. A Windoz user cannot turn off the page file to defrag & then restart it?

Not trying to be a wisenhiemer but there is a toggle to turn the pagefile off. Does Windoz still maintain a pagefiel even if it is turned off?

Yeah, I seem to remember people doing that, too - simply setting the pagefile to 0, then setting it to the size you want.  When Windows recreates the pagefile, it is all contiguous, assuming you have a large chunk of contiguous free space on your drive when you do it.  I think that's what Michael was just saying, about defragging the whole drive when doing this...?

As another note, though, turning the pagefile off and leaving it off can cause dramatic performance decreases.  This is one of the ways people get themselves into trouble.  Particularly with things like Digital Audio Workstations (e.g., home or professional audio recording, using products like Sonar), people get this idea that if they turn off the pagefile, it will keep Windows from hitting their drive for swapping.  Since the dreaded "dropouts" (i.e., gaps in recording audio) can be caused by Windows suddenly deciding to do a bunch of paging, a lot of people get this idea that turning off the pagefile will be a good thing.

This is invariably not true.  This is because memory allocation is an expensive operation for the OS.  It is faster and easier for the OS to swap pages between RAM and the pagefile than it is to allocate more memory for a process.

So what a lot of applications do is allocate the maximum amount of memory they intend to use.  They may not use a lot of this memory, so these "empty pages" just sit there taking up RAM.  However, since these pages don't get actively used, the OS will swap them out to the pagefile as soon as RAM starts running low.  Once these "empty pages" are in the pagefile, they do not take up any active system resources.  They merely take up space in the pagefile, and since the pagefile is usually such a small percentage of overall disk space, this is something that we usually do not care about at all.

This is why a lot of people get hung up on the amount of RAM that Task Manager says is being taken up by each process, and why they start looking for utilities that "free up unused memory".  However, all these utilities do is eat up extra CPU cycles, without doing anything effective.  Windows has already swapped all this unused space out the swapfile, so it's not actively using any RAM to start with.  The utility that claims to "free up unused memory" really serves no purpose at all, and hopefully you did not spend any money on it.

But now, let's take a look at what happens when you have the pagefile turned off.  All of a sudden, Windows can no longer swap out these "unused pages".  Now they REALLY DO start taking up RAM for no purpose.  Before long, you'll find yourself running out of memory, and your system will grind to a halt (or crash).  All because applications initially allocate the maximum amount of memory they think they might need, so they can avoid the expensive process of allocating more memory.  That's what happens in Windows when the pagefile is turned off.

And it might also be worth noting that it actually is not possible to "turn off" the pagefile.  You can set its size to zero, but that's not the same as turning it off.

There are actually a number of different states for memory in Windows.  There's memory that is actively in-use.  Then there's memory that is holding applications that were run recently, and are still resident, in case the user wants to run them again.  Then there's pages that are in-use, but have been swapped out to disk, and are ready to be freed up for something else to use.  Then there's memory that has started to be prepared for another process (Windows goes through, and "zeros out" the entire range, so the new process starts with a "blank slate").  Then there's memory that is all prepared, and ready to be used by another process - the only true "free" memory.

Windows goes through a rather elaborate dance, in order to try to balance all of this, and hit resources in a way so that the user gets the best "perceived performance".  Turning off the pagefile interrupts this delicate dance, and prevents Windows from using a key resource in this dance.  It doesn't make the dance go any faster - Windows still does the entire dance, even if the size of the pagefile is 0 - it just can't offload those unused pages of memory to the pagefile, to free up more of the all-important RAM.  So by turning off the pagefile, we severely cripple the memory management capabilities of Windows, without getting any real benefits anywhere.

[John Mayo]

Thanks again.

I have only turned off the pagefile for defragging. It always gets restarted after the process & nothing else runs on the PC during. I turn off all AV processes and anything else that may be running.

My general practice with Windows was to always start up bare bones. Windows & AV. All unessential apps are disable at startup with MSConfig. I max out the pagefile & let the users call me if they have an issue.

[John Mayo]

...and no I haven't spent any money on memory management apps.