I take it you want to manipulate the dialog box from some kind of reactor or at very least a VB(a) program that opens the window and does some things ...
You need to know several things, depending upon which you get, will depend upon how many you need ..Lets start with the basics:
To manipulate a windows control via VB(a) you need to use SendMessage or SetWindowLong ... each has positives and negatives ... I'll let you research those ..
To send a message or set the style of a window or control, you need to know the handle, to get that you need to be able enumerate all of the controls on a specific window, that window you will have to find by its class or its handle .. so you see it all revolves around getting that all important handle to the checkbox you want to manipulate.
Handles change from instance to instance, so if you open the window now, it will be one thing, but the next time you open it, it will be different .. so how do we find this with some degree of accuracy ...
By the window caption, but since it isn't a topmost window, how do we know with any degree of certainty what that caption might be ..
We don't unless the caption is static, bet then we don't know that to be true either .. so we have to find the caption of the main application AutoCAD .. this is done relatively easily using the Application.Caption property in VB(a).
Once you have the caption, you can use the FindWindow API to get the handle of the AutoCAD application window. Armed with that, you can use EnumerateChildWindows until you find the class and/or caption of the window holding the control you want to manipulate. Clearly a wildcard comparison would be in order to ensure you have the correct window.
Once you locate the dialog box by its caption and/or class, then enumerate all of the controls until you find the one you want to manipulate. Of course you will need to do a little snoop work using Spy++ to determine the class, key, text or other uniquely identifiable item to know that you have the correct checkbox. EnumChildWindowsProc will identify the handle to the control you wish to manipulate.
Finally armed with the handle, you can use SendMessage or SetWindowLong to change the checkbox to a checked state.
Do a bit of research on the following:
FindWindow
EnumChildWindows
SendMessage
SetWindowLong
All constants required by the above procedures will also need to be investigated to determine which ones you need.