Author Topic: Best Approaches  (Read 13720 times)

0 Members and 2 Guests are viewing this topic.

DBARANAS

  • Guest
Best Approaches
« on: July 08, 2006, 01:48:09 AM »
About 2 years ago when i was trying to develop an App in vba/vb6 I ran into a big memory leak problem. memory would leak and then when I could not get more memory it would go BANG

I kind of solved that problem by using UDT types and writing UDT types into binary files instead of using xData. This quieted that problem down and then the app got bigger and did more stuff. Using xdata was a safe place to be until it leaked and crashed acad.  Now the app is bigger and the memory leak problem reared it's swamp monster head again.

I am thinking about doing the xRecord thing again but want to know if real world programers do it this way.

Anyway this thread is about best practices. I you are writing a commercial app do you still use the autodesk db model or do you put data in a safer place.

MP

  • Seagull
  • Posts: 17750
  • Have thousands of dwgs to process? Contact me.
Re: Best Approaches
« Reply #1 on: July 08, 2006, 09:36:42 AM »
I don't meet the RWP criteria but I'll pipe in anyway. Generally speaking, for modest amounts of data I'll use internal data structures (xdata / xrecords / dictionaries / xdictionaries) but beyond that I'll go external and use database(s), normally via ADO.

Also, if the data is highly relational trying to replicate a database structure using the afore mentioned internal structures is do-able but you tend to write more code to manage "the system" rather than direct data storage / retrieval.

Shooting in the dark: Can you keep a window of data rather than all of it? e.g. A program that logs select user activities (primarilly for post mortem investigations) may only need to keep a window of say 1000 entries, discarding anything beyond that.

Any way we can get you to provide a little more info about the kind and magnitude of data you're storing?

PS: I don't use CAO if that's what you mean by "the Autodesk db model".

/ramble on
Engineering Technologist • CAD Automation Practitioner
Automation ▸ Design ▸ Drafting ▸ Document Control ▸ Client
cadanalyst@gmail.comhttp://cadanalyst.slack.comhttp://linkedin.com/in/cadanalyst

Jim Yadon

  • Guest
Re: Best Approaches
« Reply #2 on: July 08, 2006, 10:39:45 AM »
I am not a professional programmer. I do have mixed feelings on this issue though. What I consider to be average amounts of data for a user session consists of anywhere from 5000 to 40,000 individual pieces of data, usually Integers, Decimals or Strings, that I read into MDA of UDT. Similar to what you do I'd suspect from what I read in your post. My largest endeavor contained just under 3,000,000.  I've experimented with various forms of storage (I'm currently toying with the limits of using SQL databases in my spare time) and am still not set on any one method of storage other than I will not store it within the drawing itself. I have taken more beatings over the shear size of drawing files when I do this than any other complaint. I am partial to attaching XData to entities to use as keys for accessing my data from other locations. I also see the drawing files corrupt more often when I am storing my data within them. I think it's a wonderful ideal to want to have one stop shopping for our data. I just don't think it realistic with the dynamic and varied possibilities of software combinations that are present across multiple machines within even the same network. Not to mention the collander like memory management that ACAD already possesses without any help from folks like us.

DBARANAS

  • Guest
Re: Best Approaches
« Reply #3 on: July 09, 2006, 12:54:40 AM »
My app is kind of like ADT/Mitek/VisionRes. I parametrically model 3D framing and pull it apart for prefabrication and load pieces onto trucks. Then build back the model as assemblies get craned off so a site guy can look at his pocket PC and see where that assembly goes. I isolate assemblies for all the work stations in my factory....i.e cut station, sub assembly station. assembly station, swing crane, etc.

I have about ~20 structures and they average about 1,000 elements each for each object.
There usually is about 100 to 200 of every object type in a model. Each assembly has about 20 to 100 solid elements. Everything is positioned in modelspace.

I store all the handles as I need to bring objects back and move them around (make bundles for trucks,machinery,etc)

In com/interop life was simple but memory climbed most noticably at every xdata write. I went to UDT's and the leaks were 95% stopped. Using VB put and get methods were lean, quick and mean...not pretty but they did the job great.

Then add more features and it was back again.

.Net to the rescue!!! If I could rewrite the app I could break free and take control.

If I made an EXE things worked but it was WAY TOO slow. I decided to go in process .DLL evrything would be nice and fast like VBA....but you cannot stream structures to a binary file and get them back from inside the DLL. The read part blows up. OK then I can do it in XML....but now I have bloated XML files. The average read and write making a wall assembly reads and writes about ~20 times. The disk read and write times are now an issue.

...........That is why I was asking the "Best Approach" question. I am a coder/carpenter/labourer, and eveything else in between. I wish I could ask the guy in the cubicle next door and get a simple answer.

Going to Autocad development in .Net is off the beaten path. I am damn grateful a forum like this even exists!

.....enough ranting...I am now thinking about using some low down dirty database appproach. All I wish for Christmas is to be about to store data structures in binary(or something close) and get them back fast. This stuff should just be a couple functions where I can pass the arguments too.

Jay...

I took my beatings too...when I started to do things this way then Autocad could crash and I could fire it up again and not lose any work. The model could rebuild itself.

I learned the hard way...after getting a condominium project 80% finished and having Autocad die was too much to handle. I was saving things every 15 minutes in MY OWN FILE that I CHOSE and I had to go back 30 saves to find a file that was usable.

That was it, I went to external data storage.

I was in VBA and thought that I was going too far in an environment not suited for what I was trying to do.

My hope was that maybe in the .Net world all was OK by using xdata


MickD

  • King Gator
  • Posts: 3636
  • (x-in)->[process]->(y-out) ... simples!
Re: Best Approaches
« Reply #4 on: July 09, 2006, 01:37:38 AM »
I use 3d solids pretty much in the same way as you mention but I keep my 'data' in the drg to a minimum. We do steel detailing which is very similar to how you describe.
I'm currently using xdata to store just the bare minimum of info related to the object such as a db key or world points etc and all other database type work is done with queries or reports using access for the data grunt work. This allows us to free form model anything we like, it wouldn't be too hard to combine a bit of automation in there though for simple structures/assemblies. I don't even use acad for matlists in the drg, I make sure my main model db is up to date and run a query based on the assemblies in the detailed drg.

Is it possible that you are 'appending' to your xdata each time you access it and not deleting and replacing?? Maybe it's a COM thing??

Quote
I decided to go in process .DLL evrything would be nice and fast like VBA....but you cannot stream structures to a binary file and get them back from inside the DLL.

This shouldn't be a problem at all, in fact it would be faster than vba, if you are using the arx wrappers and no interop to read data from the autocad db. If your data structures inherit the ISerializable interface you can read/write to binary pretty much as fast a C from where ever you want.
"Programming is really just the mundane aspect of expressing a solution to a problem."
- John Carmack

"Short cuts make long delays,' argued Pippin.”
- J.R.R. Tolkien

DBARANAS

  • Guest
Re: Best Approaches
« Reply #5 on: July 09, 2006, 04:33:53 PM »
I could not see any reason why I could not do a binary read from within a dll either. This is a snippet of what I was trying to do. It could write OK (I checked it in a hex veiwer) but raised an exception when I tried to read it back. Oddly it works when the solution is an exe.

    <Serializable()> _
 Public Structure solidBeamPocket
        Dim angle As Double
        Dim assemblyOrder As Integer
        Dim blockDefHnd As String
        Dim blockRefHnd As String
        Dim built As Boolean
        Dim centroid As Object
        Dim centroidHnd As String
        Dim centerLineHnd As String
        Dim crippleHnd() As String
        Dim crippleLength As Double
        Dim crippleQty As Integer
        Dim desiredElevation As Double
        Dim exists As Boolean
        Dim exterior As Boolean
        Dim floor As Integer
        Dim height As Double
        Dim hiLimit As Double
        Dim offset As Double
        Dim offsetDirection As Boolean
        Dim offsetFromObjectPosition As Integer    '0 Left, 1 Center, 2 Right
        Dim openHnd As String
        Dim position As Integer'0-left 1-center 2-right
        Dim realOffset As Double
        Dim ref As Integer
        Dim refTextHnd As String
        Dim symbolHnd As String
        Dim standerdElevation As Double
        Dim studHnd() As String
        Dim studLength As Double
        Dim studQty As Integer
        Dim studWidth As Double
        Dim style As Integer
        Dim wall As Integer
        Dim wallOfs As Double
        Dim weight As Double
        Dim width As Double
        Dim widthAdjust As Double

        Sub New(ByVal a1 As Integer, ByVal b1 As Integer, ByVal b2 As Integer)
            crippleHnd = New String(11) {}
            studHnd = New String(1) {}
        End Sub

      End Structure

  Friend Sub initStructures()

   ............

        Dim i As Integer = 0

          For i = 0 To 200
            bp(i) = New solidBeamPocket(1, 1, 1)
          Next i

End Sub

Public Sub dataBeamPocket(ByVal store As Boolean)

Select Case store
Case True

Dim bf As New BinaryFormatter()
Dim fs As New IO.FileStream("t:\projects\frameWhore\data\beampocket.dat", IO.FileMode.Create)
bf.Serialize(fs, bp)
fs.Close()
fs = Nothing
bf = Nothing

Case False

Dim bf As New BinaryFormatter()
Dim fs As New IO.FileStream("t:\projects\frameWhore\data\beampocket.dat", IO.FileMode.Open)
bp = DirectCast(bf.Deserialize(fs), solidBeamPocket())
fs.Close()
fs = Nothing
bf = Nothing

End Select

End Sub


When I try to read the structure back from the binary file.........

System.Runtime.Serialization.SerializationException was unhandled by user code
Message="Unable to find assembly 'frmWhr, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'."
Source="mscorlib"
StackTrace:
at System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo.GetAssembly()
at System.Runtime.Serialization.Formatters.Binary.ObjectReader.GetType(BinaryAssemblyInfo assemblyInfo, String name)
at System.Runtime.Serialization.Formatters.Binary.BinaryConverter.TypeFromInfo(BinaryTypeEnum binaryTypeEnum, Object typeInformation, ObjectReader objectReader, BinaryAssemblyInfo assemblyInfo, InternalPrimitiveTypeE& primitiveTypeEnum, String& typeString, Type& type, Boolean& isVariant)
at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadArray(BinaryHeaderEnum binaryHeaderEnum)
at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run()
at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(HeaderHandler handler, __BinaryParser serParser, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream)
at frmWhr.Structures.dataBeamPocket(Boolean store)
at frmWhr.Structures.initStructures()
at frmWhr.frmWhr.Form1.cmdDirectionSouthWest_Click(Object sender, EventArgs e)
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

MickD

  • King Gator
  • Posts: 3636
  • (x-in)->[process]->(y-out) ... simples!
Re: Best Approaches
« Reply #6 on: July 09, 2006, 06:14:25 PM »
I'm just wondering if because it's a class library, you might need more assembly attributes so other assemblies can 'see' your methods and objects in your class library??
"Programming is really just the mundane aspect of expressing a solution to a problem."
- John Carmack

"Short cuts make long delays,' argued Pippin.”
- J.R.R. Tolkien

Glenn R

  • Guest
Re: Best Approaches
« Reply #7 on: July 09, 2006, 08:16:55 PM »
What is frmWhr?

DBARANAS

  • Guest
Re: Best Approaches
« Reply #8 on: July 10, 2006, 04:53:39 AM »
What is frmWhr?

Oops!

frmWhr is the nickname of this project. It is a name that we know constuction trades fondly identify with. It's a little crude but the carpenters in Canada we work with like it. Most of them do not want to even use a computer or be thought of as a geek.

The goal is to enable a carpenter to do 3D parametric modelling of building framing without being computer savvy. When I get this working on a pocket PC you might here things like...."Just a sec.... let me get my frameWhore to figure that out"

e.g. a crane operator just looks at is pocket PC and sees where the next assembly is going to be lifted to. No paper because it usually lands in a puddle or something. Hopefully the pocket PC's last at least a week:)

DBARANAS

  • Guest
Re: Best Approaches
« Reply #9 on: July 10, 2006, 05:10:09 AM »
No all of the structures are inside their own  module.

My best <uneducated> guess is that somehow because the dll is inside the Autocad process I need to better qualify the destination that the deserialization process belongs to.

That fact that it will work inside an .exe and not inside a .dll makes me think this way. Maybe there is something like   

<psuedo code> autocad.runningprocess.frmwhr.etc.etc

to get the binary file back home.


MickD

  • King Gator
  • Posts: 3636
  • (x-in)->[process]->(y-out) ... simples!
Re: Best Approaches
« Reply #10 on: July 10, 2006, 05:27:16 AM »
Does your app run purely as a dll or is it mixed exe with a .net dll running in autocad?
If you are running your dll in acad to get the binary data, passing it back to your exe could be a problem.
I ask because if your exe is driving the acad geometry (using COM, as this is the only way it can from an exe) it would be better to get your data in your exe as both your app and autocad are running in seperate processes and will require some 'massaging' to get things to work across processes.
"Programming is really just the mundane aspect of expressing a solution to a problem."
- John Carmack

"Short cuts make long delays,' argued Pippin.”
- J.R.R. Tolkien

DBARANAS

  • Guest
Re: Best Approaches
« Reply #11 on: July 10, 2006, 05:43:00 AM »
Does your app run purely as a dll or is it mixed exe with a .net dll running in autocad?
If you are running your dll in acad to get the binary data, passing it back to your exe could be a problem.
I ask because if your exe is driving the acad geometry (using COM, as this is the only way it can from an exe) it would be better to get your data in your exe as both your app and autocad are running in seperate processes and will require some 'massaging' to get things to work across processes.

I am running in a dll within acad entirely. I killed off all the interop interfaces and am trying to be completely .net. All the interop subs are commented out and as I figure things out in .net I bring them online. figuring out how to make solids in .net has kept me busy all weekend hooking things up again.

I netload my dll and run a CommandMethod that fires everything up. <windows form>


MickD

  • King Gator
  • Posts: 3636
  • (x-in)->[process]->(y-out) ... simples!
Re: Best Approaches
« Reply #12 on: July 10, 2006, 05:54:03 AM »
hmmm...
Are you using autocad's 'showmodal/modelessdialog' methods to show your forms? -

Autodesk.AutoCAD.ApplicationServices.Application.ShowModalDialog(dlg);

and you may also need to lock your document -

DocumentLock doclock = Autodesk.AutoCAD.ApplicationServices.Application.DocumentManager.MdiActiveDocument.LockDocument();
//some do it code:
doclock.Dispose();

hth, and if you need a hand with the solids and transformations etc just holler.
Mick.
"Programming is really just the mundane aspect of expressing a solution to a problem."
- John Carmack

"Short cuts make long delays,' argued Pippin.”
- J.R.R. Tolkien

DBARANAS

  • Guest
Re: Best Approaches
« Reply #13 on: July 10, 2006, 05:58:06 AM »
This is what I am doing...


     Private Shared ReadOnly Property ThisForm() As Form1

            Get
                If s_ThisForm Is Nothing OrElse s_ThisForm.IsDisposed Then
                    s_ThisForm = New Form1
                End If
                Return s_ThisForm
            End Get

        End Property

        <CommandMethod("FRMWHR")> _
        Public Shared Sub FormShowCommandMethod()

            With ThisForm
                .Top = 0
                .Left = 0
                .Width = 400
                .Height = 1024
            End With

            AcadApp.ShowModalDialog(ThisForm)
        End Sub

MickD

  • King Gator
  • Posts: 3636
  • (x-in)->[process]->(y-out) ... simples!
Re: Best Approaches
« Reply #14 on: July 10, 2006, 06:08:08 AM »
I think your answer lies in this discussion, you were close :)

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=128610&SiteID=1
"Programming is really just the mundane aspect of expressing a solution to a problem."
- John Carmack

"Short cuts make long delays,' argued Pippin.”
- J.R.R. Tolkien