Author Topic: Fenton Webb's advice re "To dispose or not to dispose"  (Read 74682 times)

0 Members and 1 Guest are viewing this topic.

BlackBox

  • King Gator
  • Posts: 3770
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #30 on: August 07, 2012, 09:35:58 AM »
As a Visual LISP guy, who's trying to teach myself .NET, I'd just like to thank everyone for posting (great?) questions, code examples, and the overall discussion... Especially being new to .NET concepts in general, not just AutoCAD's .NET API, but also Civil 3D's .NET API, I don't often know what I don't know.

Working for a company that doesn't exactly encourage development (I do most of it entirely in my personal time), I'm often relegated to using the developer documentation which is not easy to read. With the ActiveX documentation being removed from even the LISP developer documentation (at least as integrated in the main *dev.chm), Autodesk isn't exactly encouraging me to invest much at all in the products I use regularly.

FWIW - Reading this thread is the first time I've even seen StartOpenCloseTransaction() mentioned; granted I should read DevBlog more, but I just don't have time to read everything, as this is still very much just a hobby (or something I work on for my night job).

Having members that I've historically learned much from, discussing this openly with ADN member(s) like Fenton, is of great value to me, and I just wanted to say thanks to the lot!

*bows out to 'listen' and learn*
"How we think determines what we do, and what we do determines what we get."

Jeff H

  • Needs a day job
  • Posts: 6144
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #31 on: August 07, 2012, 07:00:41 PM »
I am screwing something up for anyone who wants to try to test with code and drawings and results below.
I just do not have the attention span, will or honestly do not care to investigate anything more than a quick slapping together, but tried to make them equal but looking quickly in Reflector looks like you need to use the OpenCloseTransaction object unlike the Transaction object where you can use TransactionManager for GetObject
 
Basically a test using
Open Close method
OpenCloseTransaction
Transaction
 
Here is the code
*******************************************EDIT*********************
Just noticed I open ModelSpace for write in half of them and read for the other
Code - C#: [Select]
  1.         Random ran = new Random();
  2.         Stopwatch sw = new Stopwatch();
  3.  
  4.         [CommandMethod("TurnAllEntitesRandomColor")]
  5.         public void TurnAllEntitesRandomColor()
  6.         {
  7.             int colorIndex = ran.Next(1, 256);
  8.            
  9.             Database db = HostApplicationServices.WorkingDatabase;
  10.             ObjectId msId = SymbolUtilityServices.GetBlockModelSpaceId(db);
  11.             Autodesk.AutoCAD.DatabaseServices.TransactionManager tm = db.TransactionManager;
  12.             sw.Reset();
  13.             sw.Start();
  14.             using (Transaction trx = tm.StartTransaction())
  15.             {
  16.                 BlockTableRecord btr = (BlockTableRecord)tm.GetObject(msId, OpenMode.ForRead, false, false) ;
  17.                
  18.                 foreach (ObjectId id in btr)
  19.                 {
  20.                     Entity ent = (Entity)tm.GetObject(id, OpenMode.ForWrite, false, false);
  21.                     ent.ColorIndex = colorIndex;
  22.                 }
  23.  
  24.                 trx.Commit();
  25.             }
  26.             sw.Stop();
  27.             Application.DocumentManager.MdiActiveDocument.Editor.WriteMessage("\nStartTransaction\n");
  28.             Application.DocumentManager.MdiActiveDocument.Editor.WriteMessage(sw.ElapsedTicks.ToString());
  29.         }
  30.  
  31.  
  32.         [CommandMethod("TurnAllEntitesRandomColorOpenClosetransaction")]
  33.         public void TurnAllEntitesRandomColorOpenClosetransaction()
  34.         {
  35.             int colorIndex = ran.Next(1, 256);
  36.             Database db = HostApplicationServices.WorkingDatabase;
  37.             ObjectId msId = SymbolUtilityServices.GetBlockModelSpaceId(db);
  38.             Autodesk.AutoCAD.DatabaseServices.TransactionManager tm = db.TransactionManager;
  39.             sw.Reset();
  40.             sw.Start();
  41.             using (OpenCloseTransaction trx = tm.StartOpenCloseTransaction())
  42.             {
  43.                 BlockTableRecord btr = (BlockTableRecord)trx.GetObject(msId, OpenMode.ForWrite, false, false);  
  44.                
  45.                 foreach (ObjectId id in btr)
  46.                 {
  47.                     Entity ent = (Entity)trx.GetObject(id, OpenMode.ForWrite, false, false);
  48.                     ent.ColorIndex = colorIndex;
  49.                 }
  50.  
  51.                 trx.Commit();
  52.             }
  53.             sw.Stop();
  54.             Application.DocumentManager.MdiActiveDocument.Editor.WriteMessage("\nOpenCloseTransaction\n");
  55.             Application.DocumentManager.MdiActiveDocument.Editor.WriteMessage(sw.ElapsedTicks.ToString());
  56.         }
  57.  
  58.  
  59.         [CommandMethod("TurnAllEntitesRandomColorOpenClose")]
  60.         public void TurnAllEntitesRandomColorOpenClose()
  61.         {
  62.             int colorIndex = ran.Next(1, 256);
  63.             Database db = HostApplicationServices.WorkingDatabase;
  64.             ObjectId msId = SymbolUtilityServices.GetBlockModelSpaceId(db);        
  65.             sw.Reset();
  66.             sw.Start();
  67.             BlockTableRecord btr = (BlockTableRecord)msId.Open(OpenMode.ForRead, false, false);
  68.             foreach (ObjectId id in btr)
  69.             {
  70.                 Entity ent = (Entity)id.Open(OpenMode.ForWrite, false, false);
  71.                 ent.ColorIndex = colorIndex;
  72.                 ent.Close();
  73.             }
  74.             btr.Close();
  75.             sw.Stop();
  76.             Application.DocumentManager.MdiActiveDocument.Editor.WriteMessage("\nOpenClose\n");
  77.             Application.DocumentManager.MdiActiveDocument.Editor.WriteMessage(sw.ElapsedTicks.ToString());
  78.         }
  79.  
  80.  
  81. /////////Added at end just to mix them
  82.  
  83.  [CommandMethod("TurnAllEntitesRandomColorOpenClosetransaction2")]
  84.         public void TurnAllEntitesRandomColorOpenClosetransaction2()
  85.         {
  86.             int colorIndex = ran.Next(1, 256);
  87.             Database db = HostApplicationServices.WorkingDatabase;
  88.             ObjectId msId = SymbolUtilityServices.GetBlockModelSpaceId(db);
  89.             Autodesk.AutoCAD.DatabaseServices.TransactionManager tm = db.TransactionManager;
  90.             sw.Reset();
  91.             sw.Start();
  92.             using (Transaction trx = tm.StartTransaction())
  93.             {
  94.                 BlockTableRecord btr = (BlockTableRecord)tm.GetObject(msId, OpenMode.ForWrite, false, false);
  95.                 foreach (ObjectId id in btr)
  96.                 {
  97.                     using (OpenCloseTransaction openCloseTrx = tm.StartOpenCloseTransaction())
  98.                     {
  99.                         Entity ent = (Entity)openCloseTrx.GetObject(id, OpenMode.ForWrite, false, false);
  100.                         ent.ColorIndex = colorIndex;
  101.                         openCloseTrx.Commit();
  102.                     }
  103.                 }
  104.  
  105.                 trx.Commit();
  106.             }
  107.             sw.Stop();
  108.             Application.DocumentManager.MdiActiveDocument.Editor.WriteMessage("\nStartTransAction&OpenCloseTransaction\n");
  109.             Application.DocumentManager.MdiActiveDocument.Editor.WriteMessage(sw.ElapsedTicks.ToString());
  110.         }
  111.  
  112.  
  113.  

Here were the results on drawing with
150,000
1,020,000
150,000
entities
 
For each test
-I started AutoCAD
-Opened drawing
-Netloaded test
-Ran one test
-Closed AutoCAD
 
Quote

150,000
 
OpenClose
5,369,146
 
OpenCloseTransaction
30,092,959
 
StartTransaction
10,525,513
 
StartTransAction&OpenCloseTransaction
11,047,617
 

1,020,000
 
OpenClose
37,983,450
 
OpenCloseTransaction
39,409,458
 
StartTransaction
72,692,233
 

150,000
OpenClose
5,688,127
 
OpenCloseTransaction
5,609,783
 
StartTransaction
10,439,014

Since the drawings are pretty large and not to take up Mark's space these are links to Dropbox public folder
Drawing with 15,000 Entities
Drawing with 60,000 Entities
Drawing with 150,000 Entities
Drawing with 1,020,000 Entities
Zip with all drawings above.

TheMaster

  • Guest
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #32 on: August 07, 2012, 08:58:08 PM »

Fenton, I've been back to re-read the couple of blog posts in question.

http://adndevblog.typepad.com/autocad/2012/07/the-right-tools-for-the-job-autocad-part-3.html

Here, Fenton Webb wrote about the poor speed LISP.
In fact, it's not so bad - it all depends on the skill of the programmer and his language skills.

Sorry, but I'm going to put my $0.02 in here, in defense of Fenton, because you've maligned him, intentionally or not.

The skill of a programmer cannot overcome inherent limitations of a programming language.

You cannot say to me that you can write LISP code that can perform complex coordinate geometry calculations faster, or even remotely close to the speed it can be done in .NET or C++. You cannot say to me that you can use an association list in LISP to lookup values associated with a key, faster than I can do that using a hashcode-based solution like the Dictionary in .NET, or the std:map in C++.  In fact, your LISP code will be 1000x slower at that even with a relatively small amount of data, compared to the equivalent .NET or C++ solution.

Quote
Three years ago, I tested the dictionary in autocad - added dictionaries for 100 mb. My program to emulate the internal dictionary as the SQL database. Speed, compared to an external database, located on the same computer that was ten times faster!

I do not think that the author is equally good command of all the above languages​...

Sorry again, but are you are talking about a kludge to achieve O(1) or hashcode-like lookup performance using AutoCAD dictionaries? Well, that solves one of the thousands of problems that are very easily solved when using the right tools for the job, doesn't it.

If on the other hand, you're comparing the performance of AutoCAD operations, rather than operations performed purely by LISP code, of course, the same AutoCAD APIs, whether they're being used from LISP, ActiveX, .NET or C++ will perform roughly the same, but Fenton was not talking about the performance of AutoCAD or its common APIs that are available to most or all programming interfaces, and he wasn't talking about kludges that use AutoCAD APIs to achieve reasonable performance either, what he was talking about is performance in the context of using APIs that are not even available to LISP, and of operations performed purely in user/client code (e.g., something that must be fully-implemented in the language being used) without kludges.

You can choose any common problem in programming that you wish, implemented in LISP, or any kludge you can find that mooches off of AutoCAD, and it will never come remotely close to the performance of native code.

How about Delaunay triangulation ?  Would you care to give that a shot ?
« Last Edit: August 08, 2012, 01:50:43 AM by TT »

fentonwebb

  • Guest
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #33 on: August 07, 2012, 09:00:04 PM »
Hi all

here are my replies - sorry if it seems too short, but I need to get home to the "wife to be"...

Keith tm - "I personally don't recommend leaving unfinished tasks to the GC, instead I like to dispose of items as soon as I am done with them (or as soon as practical). Doing it this way seems straightforward if you ask me."
Hey Keith, you definitely have the right approach!

TT - "And here's something those Autodesk people didn't tell you:
Avoiding calls to Dispose() when it is unnecessary can actually improve the performance of AutoCAD and your plug-ins. The reason for that is because by default, the GC runs on a high-priority background thread, which means that on a multiple-processor system (do you know anyone that still runs AutoCAD on a single-processor system?), the GC can execute on a different (and presumably under-utilized) processor, concurrently, while AutoCAD and your plug-in run on another processor (e.g., a form of parallel execution). And, since AutoCAD doesn't leverage multiple processors, work done by the GC can be offloaded to another CPU core with virtually no performance impact on AutoCAD and your plug-in.

So, if you do follow the advice of Fenton and Stephen, the work that must be done by Dispose() will be done synchronously, requiring AutoCAD and your plug-in to wait for it to complete, which is not the case if you just allow the GC to take care of it on another under-utilized CPU core."

Hey TT, What you say is totally true, but as I have mentioned before, AutoCAD is *not* thread safe, meaning that if you allow the GC to dispose of AutoCAD objects you run the risk of some AutoCAD code being executed on something otherthan the main thread, which it was never designed for - try it and see what happens - how many of you out there get random crashes happening that you cannot track down...? If you insist on allowing the GC to dispose your AutoCAD class objects, be sure to utilize my blog entry on gcConcurrent - that's my advice.

TT - "The reason for all of the confusion in the case of AutoCAD's managed API, is that many AutoCAD managed types implement IDisposable even when it is unnecessary, and largely out of convenience. Many managed types derive from DisposableWrapper (which is the class that implements IDisposable), and in some cases, do so because they must also derive from the RXObject base type (which is derived from DisposableWrapper), because the managed class represents a native object that is an instance of AcRxObject."

Hey TT, I'll do my best to clear up any confusion where I can, I'll do this on our blog soon. In the meantime, the IDisposable interface as per the Microsoft documentation "The primary use of this interface is to release unmanaged resources". The simple fact that our .NET API wraps an unmanaged ObjectARX API, well, I personally think that generally speaking we have done things correctly. That said, I'd like to definitely hear from you which classes you think implement this interface frivolously?

dgorsman - "Do you think that total re-write could be done without significantly breaking existing code?  And in a timeframe consistent with the rate at which tools, .NET frameworks, etc. are released and retired?"

If you are talking about changing our .NET API so that IDisposable is added/removed then no - It is a version-breaking change to add the IDisposable interface to an existing class, because it changes the semantics of the class.

ElpanovEvgeniy - "Here, Fenton Webb wrote about the poor speed LISP." 
Hey ElpanovEvgeniy, I believe I said I was surprised at how fast LISP was, actually.

gile - "Do I have to dispose the plineCollection (line 18) ?

2. Do I have to dispose the regionCollection (line 25) ? And if so, do I have to dispose each region too (line 40)

3. If I dispose each region (line 40), do I need to dispose the regionCollection too (line 25) ?

4. (subsidiary) May I have used StartOpenCloseTransaction() instead of StartTransaction() (line 14) ?"

Hey gile

1) Yes
2) DBObjectCollection should automatically dispose of all 'disposable' objects held within
3) Calling Dispose on the region is fine, but it should not be needed because of (2)
4) Yes

ElpanovEvgeniy

  • Water Moccasin
  • Posts: 1569
  • Moscow (Russia)
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #34 on: August 07, 2012, 11:56:28 PM »
How about Delaunay triangulation ?  Would you care to give that a shot ?

Are you kidding?
Perhaps you are trying to scare me, the complexity of the algorithm?

Here, I'm forced programmers to improve the ARX triangulation that is used in products Autodesk ...

For complex systems, is important not the language, and the internal algorithms and simplicity.

And you, ever, made its own algorithm for Delaunay triangulation or Voronoi polygons - Chart Tisson?

ElpanovEvgeniy

  • Water Moccasin
  • Posts: 1569
  • Moscow (Russia)
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #35 on: August 08, 2012, 01:37:15 AM »
After Autodesk University I am ready to compete with you, comparing the speed of programs. Choose an interesting problem for the competition.
I am sure, the language just a tool. Speed ​​is gained by algorithms. LISP makes it easier to develop algorithms!
 

it will be hot 'CHALLENGE'  8-)

TheMaster

  • Guest
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #36 on: August 08, 2012, 02:57:26 AM »
Quote
gile - "Do I have to dispose the plineCollection (line 18) ?

2. Do I have to dispose the regionCollection (line 25) ? And if so, do I have to dispose each region too (line 40)

3. If I dispose each region (line 40), do I need to dispose the regionCollection too (line 25) ?

4. (subsidiary) May I have used StartOpenCloseTransaction() instead of StartTransaction() (line 14) ?"

Hey gile
1) Yes
2) DBObjectCollection should automatically dispose of all 'disposable' objects held within
3) Calling Dispose on the region is fine, but it should not be needed because of (2)
4) Yes[/b]

Hi Fenton - Sorry, but WRT 2 & 3, that's not the case.

A DBObjectCollection returned by a call to an API that creates AcDbObjects initially contains only native pointers to those AcDbObjects, but contains no managed wrappers for them. The DBObjectCollection initially assumes ownership of those contained AcDbObjects, until a managed wrapper is created for one and is returned to the client code using the DBObjectCollection.

At the point when a managed wrapper is created for an AcDbObject in the collection, the DBObjectCollection relinquishes ownership of the native AcDbObject, and the client code that obtained the managed wrapper for it assumes ownership of the object, and must Dispose() it, or add it to a database. A DBObjectCollection never disposes managed wrappers.

When a DBObjectCollection is disposed or finalized, it will delete any AcDbObjects it contains, for which a managed wrapper was never created, and conversely, will not delete any AcDbObjects for which managed wrappers were created.

So, the correct answer to 3 is yes, each region managed wrapper obtained from the DBObjectCollection must be disposed or be added to a database, regardless of whether the DBObjectCollection is disposed or not.

« Last Edit: August 08, 2012, 04:25:29 AM by TT »

TheMaster

  • Guest
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #37 on: August 08, 2012, 03:14:11 AM »
After Autodesk University I am ready to compete with you, comparing the speed of programs. Choose an interesting problem for the competition.
I am sure, the language just a tool. Speed ​​is gained by algorithms. LISP makes it easier to develop algorithms!
 
it will be hot 'CHALLENGE'  8-)

I already suggested an interesting problem.


ElpanovEvgeniy

  • Water Moccasin
  • Posts: 1569
  • Moscow (Russia)
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #38 on: August 08, 2012, 03:26:06 AM »
I already suggested an interesting problem.

excellent!
You have chosen Delaunay triangulation?

I accept the challenge.
My conditions are:
1. test computer with 4 real cores or more.
2. AutoCAD 2013 x64 (without vertical product)

We play with a multi-threaded computing...

ps. I want to compare the performance of algorithms. It makes no sense to compare the performance of AutoCAD.
With impatience I will expect to December!

TheMaster

  • Guest
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #39 on: August 08, 2012, 03:29:19 AM »
How about Delaunay triangulation ?  Would you care to give that a shot ?

Are you kidding?
Perhaps you are trying to scare me, the complexity of the algorithm?

Here, I'm forced programmers to improve the ARX triangulation that is used in products Autodesk ...

For complex systems, is important not the language, and the internal algorithms and simplicity.

And you, ever, made its own algorithm for Delaunay triangulation or Voronoi polygons - Chart Tisson?

Yes, 15 years ago I wrote a Delaunay triangulation method for an ActiveX server that I used from LISP, Delphi, etc. But, that was long before your contest.

Now, I have parallel versions of many algorithims that run concurrently on as many CPU cores as there are in a system, and all of them run about 2-3x faster than the original (single-threaded version), on a quad-core CPU.

You see, that is where you fall down.

No algorithm will allow you to do parallel execution across multiple CPU cores (or on a GPU) using LISP, but having said that, I really don't need to resort to parallel execution to run circles around LISP code.


TheMaster

  • Guest
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #40 on: August 08, 2012, 03:33:11 AM »
I already suggested an interesting problem.

excellent!
You have chosen Delaunay triangulation?

I accept the challenge.
My conditions are:
1. test computer with 4 real cores or more.
2. AutoCAD 2013 x64 (without vertical product)

We play with a multi-threaded computing...

ps. I want to compare the performance of algorithms. It makes no sense to compare the performance of AutoCAD.
With impatience I will expect to December!

I see, and how are you going to do parallel execution with LISP?

Oh, and you must open the source code, or no deal.

ElpanovEvgeniy

  • Water Moccasin
  • Posts: 1569
  • Moscow (Russia)
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #41 on: August 08, 2012, 03:38:13 AM »

I see, and how are you going to do parallel execution with LISP?

Oh, and you must open the source code, or no deal.

At Autodesk University in Moscow, I will tell how to use multithreading in AutoLISP ...

Alexander Rivilis

  • Bull Frog
  • Posts: 214
  • Programmer from Kyiv (Ukraine)
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #42 on: August 08, 2012, 04:05:43 AM »
Tony and Evgeniy!
And who will be the judges?  :-)

ElpanovEvgeniy

  • Water Moccasin
  • Posts: 1569
  • Moscow (Russia)
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #43 on: August 08, 2012, 04:13:23 AM »
My friend Alexander Rivilis said your name behind the nickname TT. I hasten to say, I really respect you for your contribution to programming. Indeed, the competition will be hot and fun!

ps. Yes, I will publish here all the sources. I hope you will too...

TheMaster

  • Guest
Re: Fenton Webb's advice re "To dispose or not to dispose"
« Reply #44 on: August 08, 2012, 04:20:58 AM »

I see, and how are you going to do parallel execution with LISP?

Oh, and you must open the source code, or no deal.

At Autodesk University in Moscow, I will tell how to use multithreading in AutoLISP ...

You can only do that by calling native code written in another language.  It should go without mention, that any solution must be pure LISP, and may not depend on any external API calls or any undocumented APIs.