TheSwamp

Code Red => AutoLISP (Vanilla / Visual) => Topic started by: CADDOG on July 02, 2013, 01:53:30 PM

Title: Proxy & AEC Clean/Remove
Post by: CADDOG on July 02, 2013, 01:53:30 PM
Looking for a sure fire method to purge, remove proxies programatically (especially AEC remnants).
Acceptable methods: .NET(c#) or Vlisp, or both
I posted here because lisp if preferable, but some things can't be done in lisp, but I like to expand lisp and keep working in it.
If, the moderator feels this conversation moves to a .net only topic, then perhaps it can be moved?  Don't know if that's possible for the mod to do here (otherwise I will delete and repost in .net).  Sorry ahead of time.

Attached example file that will not bind-insert due to 'proxies'.

Goals:
I've been researching for three days, and have no idea which direction to go.  There are leads, but nothing solid.  And with that I'm forced to ask a question of how, in an overall sense, instead of asking for guidance on a very specific function.  So this post is going to sound a little emotional, because dangit, it's frustrating.

My problems are the things I don't know...

So here's my merge of Augusto and Nagy's proxy cleaner into a lisp function.  Not that it does anything against AEC, but does appear to work will with other proxy's. It cleaned the clock of CADWorx, lol, thus the filter.
Code - C#: [Select]
  1. //http://adndevblog.typepad.com/autocad/2013/04/search-and-erase-proxies.html
  2. //http://adndevblog.typepad.com/autocad/2012/07/remove-proxy-objects-from-database.html
  3. [LispFunction("cd:ProxyClean")]
  4. public Boolean ProxyClean(ResultBuffer rb)
  5. {
  6.         if (rb == null)
  7.         { return false; }
  8.         TypedValue[] args = rb.AsArray();
  9.         if (args.Length != 1 || (LispDataType)args[0].TypeCode != LispDataType.ObjectId)
  10.         { return false; }
  11.         //we accept any object from the drawing that is to be processed.  I trend in sending modelspace, but it doesn't matter
  12.         //just need an object so the owning database can be retrieved
  13.         ObjectId objId = (ObjectId)args[0].Value;
  14.         Database db = objId.Database;
  15.         using (Transaction tr = db.TransactionManager.StartOpenCloseTransaction())
  16.         {
  17.                 //first let's run through the blocks
  18.                 BlockTable bt = (BlockTable)tr.GetObject(db.BlockTableId, OpenMode.ForRead);
  19.                 foreach (ObjectId btrId in bt)
  20.                 {
  21.                         BlockTableRecord btr = (BlockTableRecord)tr.GetObject(btrId, OpenMode.ForRead);
  22.                         foreach (ObjectId entId in btr)
  23.                         {
  24.                                 Entity ent = (Entity)tr.GetObject(entId, OpenMode.ForRead);
  25.                                 if (ent.IsAProxy)
  26.                                 {
  27.                                         //Might should be a wilcard filter from function caller
  28.                                         ProxyEntity proxyEnt = ent as ProxyEntity;
  29.                                         if (proxyEnt.ApplicationDescription.IndexOf("cadworx", 0, StringComparison.OrdinalIgnoreCase) >= 0)
  30.                                                 continue;
  31.                                         proxyEnt.UpgradeOpen();
  32.                                         using (DBObject newEnt = new Line())
  33.                                         {
  34.                                                 ent.HandOverTo(newEnt, false, false);
  35.                                                 newEnt.Erase();
  36.                                         }
  37.                                 }
  38.                         }
  39.                 }
  40.                 //call recursive call to remove dict proxies
  41.                 RemoveProxiesFromNamedObjDict(db.NamedObjectsDictionaryId, tr);
  42.  
  43.                 tr.Commit();
  44.                 return true;
  45.         }
  46. }
  47.  
  48.  
  49. private void RemoveProxiesFromNamedObjDict(ObjectId dictId, Transaction tr)
  50. {
  51.         using (ObjectIdCollection proxyIds = new ObjectIdCollection())
  52.         {
  53.                 DBDictionary dict = (DBDictionary)tr.GetObject(dictId, OpenMode.ForRead);
  54.                 foreach (DBDictionaryEntry entry in dict)
  55.                 {
  56.                         DBObject dictObj = tr.GetObject(entry.Value, OpenMode.ForRead);
  57.                         if (dictObj.IsAProxy || entry.Value.ObjectClass.Name == "AcDbZombieObject")
  58.                                 proxyIds.Add(entry.Value); //collect them so we only have the namedobjdict open while removing
  59.                         else if (entry.Value.ObjectClass == RXClass.GetClass(typeof(DBDictionary)))
  60.                                 RemoveProxiesFromNamedObjDict(entry.Value, tr);
  61.                 }
  62.                 if (proxyIds.Count > 0)
  63.                 {
  64.                         dict.UpgradeOpen();
  65.                         foreach (ObjectId proxyId in proxyIds)
  66.                         {
  67.                                 ProxyObject proxyObj = (ProxyObject)tr.GetObject(proxyId, OpenMode.ForRead);
  68.                                 if (proxyObj.ApplicationDescription.IndexOf("cadworx", 0, StringComparison.OrdinalIgnoreCase) >= 0)
  69.                                         continue;
  70.                                 proxyObj.UpgradeOpen();
  71.                                 using (DBObject newRec = new Xrecord())
  72.                                 {
  73.                                         proxyObj.HandOverTo(newRec, false, false);
  74.                                         newRec.Erase();
  75.                                 }
  76.                         }
  77.                 }
  78.         }
  79. }
  80.  

Test Calling from command line (express .net debugging)
Code: [Select]
(setq doc (vla-get-activedocument (vlax-get-acad-object)))
(setq edoc (vlax-vla-object->ename (vla-get-modelspace doc)))
(cd:ProxyClean edoc)

Side Note/Tip:
I love the code type formatting, but copying to your own editor creates (for me at least) a single line of text with the row numbers in line of the code...
If using Notepad++ or equivilent Find/Replace w/ RegEx.
Find: [0-9]+\.
Replace: \r\n
Title: Re: Proxy & AEC Clean/Remove
Post by: irneb on July 03, 2013, 02:12:20 AM
Yep, these are irritating at best.

I think the reason some are not shown in MP's and my code is that they're "hidden" as some sort of dictionary only attached to entities (instead of listed as a DWG dictionary). It's probably possible to get at these even through lisp, only you'd need to step through each entity in the entire DWG database (including non-graphic stuff such as dimstyles, layers, etc.). It might even be possible that they use some custom stuff which is totally unaccessible from ActiveX or even raw DXF - in which case Lisp won't be able to get to it.

I'm sorry to say this, but even .Net is not necessarily going to enable absolutely all possibilities. Most (if not all) these AEC (and other) stuff are generated by the ARX addons and/or verticals, so it's possible that they might have used something which is not opened through the managed wrappers which is available in ACad's .Net API's. But obviously chances are better that .Net would be able to get to those which Lisp can't.

BTW, usually when something fails with proxies, it means you don't have the object enablers installed. These are ARX addons (usually freely available from ADesk) which are like "viewers" for those proxy elements. Acad "should" download and install the appropriate one when it finds such a proxy ... in theory ... but as we've encountered in many aspects of ACad, this is only a utopic vision which doesn't always work. What I've found is that sometimes I need to search for the enabler and manually download + install it ... only then do the proxies work properly, enabling you to explode them into their constituent objects and clearing their extraneous data.
Title: Re: Proxy & AEC Clean/Remove
Post by: CADDOG on July 03, 2013, 03:33:32 AM
ok...

heheheheheee..... Although I was hoping you'd be the one to take on this, I knew it could be a double-edged sword; In that you'd say what you said.

Ignoring that it could possibly be an enabler I don't have installed, although I have proxynotify enabled, and don't have any notifications.  The .net function attached does iterate all block objects and NODs, but is it possible for proxies to not be blocks, but plain entities?  Should I be iterating other graphical objects other than blocks?
The attached file was a file generated in house, and although we have xdata on the centerlines (CADWorx), it's not proxy objects (CADWorx 2012 - pre-proxy).  The only other proxy source is Civil3d which is the version one Structural guy has in our organization, and I have the object enabler for Civil3d manually installed.  Because there's nothing wrong with dicts that are littered in a drawing, unless it bloats a file, therefore I don't care that there happens to be tons in this particular file.

In addition, what about an effort to simply clean AEC/Civil3D from a drawing.  At least doing that, I'd be able to identify if AEC was the reason Vanilla CAD says it can't bind "because of proxies".  I couldn't fine efforts at this forum, but is anybody aware of efforts at this forum to programatically clean AEC from a drawing? (CAB?) lol.
Title: Re: Proxy & AEC Clean/Remove
Post by: gile on July 03, 2013, 12:30:03 PM
Hi,

Coincidently, I was playing to this right now.
Here's where I am using .NET (work in progress inspired by this thread (http://adndevblog.typepad.com/autocad/2013/04/search-and-erase-proxies.html)).
The routine try to scan the whole database searching for proxy entities / objects in block table records, named dictionaries, symbol table records and in eventual extension dictionaries for all.

Code - C#: [Select]
  1.         [CommandMethod("RemoveProxies")]
  2.         public void RemoveProxies()
  3.         {
  4.             EraseAllProxies(HostApplicationServices.WorkingDatabase);
  5.         }
  6.  
  7.         private void EraseAllProxies(Database db)
  8.         {
  9.             RXClass proxyEntity = RXClass.GetClass(typeof(ProxyEntity));
  10.             ObjectIdCollection entries = new ObjectIdCollection();
  11.             using (BlockTable bt = (BlockTable)db.BlockTableId.Open(OpenMode.ForRead))
  12.             {
  13.                 foreach (ObjectId btrId in bt)
  14.                 {
  15.                     using (BlockTableRecord btr = (BlockTableRecord)btrId.Open(OpenMode.ForRead))
  16.                     {
  17.                         foreach (ObjectId id in btr)
  18.                         {
  19.                             using (Entity ent = (Entity)id.Open(OpenMode.ForWrite))
  20.                             {
  21.                                 if (id.ObjectClass.IsDerivedFrom(proxyEntity))
  22.                                 {
  23.                                     ent.Erase();
  24.                                 }
  25.                                 SearchInExtensionDictionary(ent, entries);
  26.                             }
  27.                         }
  28.                     }
  29.                 }
  30.             }
  31.  
  32.             ObjectId[] tableIds =
  33.                 {
  34.                     db.BlockTableId,
  35.                     db.LinetypeTableId,
  36.                     db.DimStyleTableId,
  37.                     db.LayerTableId,
  38.                     db.RegAppTableId,
  39.                     db.TextStyleTableId,  
  40.                     db.UcsTableId,
  41.                     db.ViewTableId,
  42.                     db.ViewportTableId
  43.                 };
  44.             foreach (ObjectId tableId in tableIds)
  45.             {
  46.                 SearchInTable(tableId, entries);
  47.             }
  48.  
  49.             using (DBDictionary NOD = (DBDictionary)db.NamedObjectsDictionaryId.Open(OpenMode.ForRead))
  50.             {
  51.                 SearchInDictionaries(NOD, entries);
  52.                 foreach (ObjectId id in entries)
  53.                 {
  54.                     using (DBDictionary newDict = new DBDictionary())
  55.                     using (DBObject proxy = (DBObject)id.Open(OpenMode.ForWrite))
  56.                     {
  57.                         try
  58.                         {
  59.                             proxy.HandOverTo(newDict, true, true);
  60.                         }
  61.                         catch { }
  62.                     }
  63.                 }
  64.             }
  65.         }
  66.  
  67.         private void SearchInTable(ObjectId tableId, ObjectIdCollection entries)
  68.         {
  69.             using (SymbolTable table = (SymbolTable)tableId.Open(OpenMode.ForRead))
  70.             {
  71.                 foreach (ObjectId strId in table)
  72.                 {
  73.                     using (SymbolTableRecord record = (SymbolTableRecord)strId.Open(OpenMode.ForRead))
  74.                     {
  75.                         SearchInExtensionDictionary(record, entries);
  76.                     }
  77.                 }
  78.             }
  79.         }
  80.  
  81.         private void SearchInExtensionDictionary(DBObject dbObj, ObjectIdCollection entries)
  82.         {
  83.             ObjectId xdictId = dbObj.ExtensionDictionary;
  84.             if (xdictId != ObjectId.Null)
  85.             {
  86.                 using (DBDictionary xdict = (DBDictionary)xdictId.Open(OpenMode.ForRead))
  87.                 {
  88.                     SearchInDictionaries(xdict, entries);
  89.                 }
  90.             }
  91.         }
  92.  
  93.         private void SearchInDictionaries(DBDictionary dict, ObjectIdCollection entries)
  94.         {
  95.             RXClass proxyObject = RXClass.GetClass(typeof(ProxyObject));
  96.             RXClass dictionary = RXClass.GetClass(typeof(DBDictionary));
  97.             SearchInExtensionDictionary(dict, entries);
  98.             foreach (DBDictionaryEntry entry in dict)
  99.             {
  100.                 ObjectId id = entry.Value;
  101.                 if (id.ObjectClass.IsDerivedFrom(proxyObject))
  102.                 {
  103.                     try
  104.                     {
  105.                         using (DBObject obj = (DBObject)id.Open(OpenMode.ForWrite))
  106.                         {
  107.                             obj.Erase();
  108.                         }
  109.                     }
  110.                     catch
  111.                     {
  112.                         entries.Add(id);
  113.                     }
  114.                 }
  115.                 else if (entry.Value.ObjectClass == dictionary)
  116.                 {
  117.                     using (DBDictionary subDict = (DBDictionary)id.Open(OpenMode.ForRead))
  118.                     {
  119.                         SearchInDictionaries(subDict, entries);
  120.                     }
  121.                 }
  122.             }
  123.         }
Title: Re: Proxy & AEC Clean/Remove
Post by: CADDOG on July 03, 2013, 02:33:31 PM
I have only an entry level understanding of proxies, so perhaps that lack of knowledge is the reason I can't find the proxy in the file attached in first post (there are hundreds).

As mentioned, I only know of two ways to identify a proxy object:
[Entity].IsAProxy and [objectId].Value.ObjectClass.Name == "AcDbZombieObject"
or one way in lisp:
(vlax-get obj 'ObjectName)

Alternatives?  What to look for on an object?

My understanding is that a proxy is a proxy because the object enablers are not present.  If the xml object type definition of a custom ARX object is present, then it is no longer a proxy object, therefore testing for a proxy is irrelevant as the object is no longer an AcDbZombieObject, rather the name of the actual object class.  Also, if object enablers were present, then autocad would not be restricting a bind-insert on an xref, as the enabler would handle the operation (presumably).  So in the .net example above, it does iterate the entire db in both model and layouts.

The only thing I'm not doing is iterating the block definitions, as I remember reading that autocad doesn't allow proxies to be inside the blocks themselves.  Actually I read that in the source of SuperFlatten (http://www.theswamp.org/index.php?topic=18153.msg455517#msg455517).  Even that being stated in the comments, Joe (http://www.theswamp.org/index.php?action=profile;u=801) still takes the time to cycle the block definition, as he states (line 296), it is "very rare case since the block command does not allow proxies to bew included in a block definition".  Who knows with ObjectARX?

While I was composing this, gile posted...
Coincidently, I was playing to this right now.
Here's where I am using .NET (work in progress inspired by this thread (http://adndevblog.typepad.com/autocad/2013/04/search-and-erase-proxies.html)).
This thread also started my journey.
My only goal difference, is that I want to keep the primatives of any graphical object, if possible - so I use HandOverTo() for everything.

Question premise...  I don't want any non-vanila acad data to get through.  I noticed that you are using IsDerivedFrom.  Property of IsAProxy evaluates false if object enablers installed for that object type.
Question... Will IsDerivedFrom RXClass.GetClass(typeof(ProxyObject)) return true even if object enablers for that object are installed?
Title: Re: Proxy & AEC Clean/Remove
Post by: CADDOG on July 03, 2013, 02:35:09 PM
Moderator,

Looks like this is going to be a .net solution.
If possible, can this be moved to the .net forum.

Sincerely,
Ricky
Title: Re: Proxy & AEC Clean/Remove
Post by: CADDOG on July 03, 2013, 05:51:40 PM
symbol table records and in eventual extension dictionaries for all.

My problem was in here (quote above).  I stopped overthinking and ran your code.
So my problems reside in either SymbolTableRecords or in Extension Dictionaries.

You know that giddy feeling, when you realize this might work?   :ugly:

Ok.. pulled the symbol table run out, and it still cleaned successfully so it was extension dictionaries.  Which btw, you are the only one, I'm aware of, that's checking extension dictionaries and symbol tables for proxies.

Amazing gile.

I looked, and couldn't find - and is probably the reason you aren't using such a thing, but is there a collection of symbol tables to iterate, or must the default nine be iterated explicitly?
Title: Re: Proxy & AEC Clean/Remove
Post by: Jeff H on July 03, 2013, 08:02:10 PM
The attached drawing is creating a proxy object from a custom object defined by Civil3D as shown in picture below.
 
AutoCAD is responsible for creating proxy objects.
 
There are 2 different types created a ProxyEntity and ProxyObject.
 
So if a custom object is saved in drawing and the object enabler is not present then a ProxyEntity is created for objects derived from AcDbEntity(Objects with graphics)  and a ProxyObject is created for objects derived from just AcDbObject.
 
When these custom objects are created and the classes are first loaded the ACRX_DXF_DEFINE_MEMBERS macro is used which has a PROXY_FLAGS argument.
Quote
AcDbProxyEntity::kEraseAllowed = 0x1
AcDbProxyEntity::kTransformAllowed = 0x2
AcDbProxyEntity::kColorChangeAllowed = 0x4
AcDbProxyEntity::kLayerChangeAllowed = 0x8
AcDbProxyEntity::kLinetypeChangeAllowed = 0x10
AcDbProxyEntity::kLinetypeScaleChangeAllowed = 0x20
AcDbProxyEntity::kVisibilityChangeAllowed = 0x40
AcDbProxyEntity::kCloningAllowed = 0x80
AcDbProxyEntity::kLineWeightChangeAllowed = 0x100
AcDbProxyEntity::kPlotStyleNameChangeAllowed = 0x200
or
AcDbProxyEntity::kAllButCloningAllowed = 0x37F
AcDbProxyEntity::kAllAllowedBits = 0x3FF
Looking at your drawing the ProxyFlag property is 129 so looks like allows cloning and erasing.
 
There is much info and other considerations which can be read in ObjectARX docs, but to keep it simple and short say a application is loaded into autocad that creates custom objects. That custom object must derive from a AcDbObject if it will be able to be saved in the drawing. If it has graphics it must derive from a AcDbEntity or another object that does.
The object enabler tells AutoCAD or gives implementation on what to do when selected, how to transform it, display, etc......
 
Not always true but proxy objects exists in memory, and give access to the custom object data for members defined by AcDbEntity or AcDbObject. The PROXY_FLAGS setting can limit edit capabilities. And as I sure you can gues a ProxyObject derives from a DbObject and a ProxyEntity derives from a Entity.
 
Quickly looking at code from ADN blog the reason it misses proxy object is it searches all the entries in the Dictionaries in the NamedObjectDictionary, but the proxy object is a entry in the NameObjectDictionary, so if  it searched the entries in the NameObjectDictionary it would find it.
 
There is a lot more to it, but basically proxies are just data holders for custom objects allowing you to read the data for members it derive from.
 
A ProxyEntity should only be in BlockTablerecords and ProxyObjects can be anywhere a AcDbObject can be stored but you should be able to find all them.
 
 
 
 
 
 
Title: Re: Proxy & AEC Clean/Remove
Post by: gile on July 04, 2013, 01:33:08 AM
Hi,

Thanks for these precisions Jeff.

I don't know much about proxies, but from time to time, I'm asked for cleaning a drawing from proxies. The files I was working on yesterday came from MicroStation and had a proxy object in a line type record extension dictionary.

I never searched for proxies in symbol tables extension dictionaries before...

This made me think: as proxies are made by AutoCAD every times it encounters an object it do not recognize, whatever the application which created it, rather than adding some new location to search every time I encounter a new one, I should search everywhere it can be.
So, may be there never been a proxy in an extension dictionary of an UCS table record but as an application can create a custom object using an ucs table record, I have to search here to.

That said, the code posted upper is still a 'work in progress' and haven't been deeply tested yet.
Title: Re: Proxy & AEC Clean/Remove
Post by: WILL HATCH on July 04, 2013, 02:43:41 PM
Great stuff guys! I've wondered about this before but haven't actually had the time to address it.
Title: Re: Proxy & AEC Clean/Remove
Post by: Alexander Rivilis on July 04, 2013, 04:45:55 PM
Hi, Gile!
What about direct iteration of database as I've proposed in comment (http://adndevblog.typepad.com/autocad/2013/04/search-and-erase-proxies.html#comment-6a0167607c2431970b017c38893859970b)?
Title: Re: Proxy & AEC Clean/Remove
Post by: gile on July 04, 2013, 04:55:01 PM
Yes Alexander, thanks.
My intent was to try this too as soon as I have some free time.
Title: Re: Proxy & AEC Clean/Remove
Post by: gile on July 04, 2013, 05:38:10 PM
Alexander's algorithm

<EDIT: safer (and a little fater) implementation>
Code - C#: [Select]
  1. private void EraseProxies(Database db)
  2.         {
  3.             RXClass zombieEntity = RXClass.GetClass(typeof(ProxyEntity));
  4.             RXClass zombieObject = RXClass.GetClass(typeof(ProxyObject));
  5.             ObjectId id;
  6.             for (long l = db.BlockTableId.Handle.Value; l < db.Handseed.Value; l++)
  7.             {
  8.                 if (!db.TryGetObjectId(new Handle(l), out id))
  9.                     continue;
  10.                 if (id.ObjectClass.IsDerivedFrom(zombieObject) && !id.IsErased)
  11.                 {
  12.                     try
  13.                     {
  14.                         using (DBObject proxy = id.Open(OpenMode.ForWrite))
  15.                         {
  16.                             proxy.Erase();
  17.                         }
  18.                     }
  19.                     catch
  20.                     {
  21.                         using (DBDictionary newDict = new DBDictionary())
  22.                         using (DBObject proxy = id.Open(OpenMode.ForWrite))
  23.                         {
  24.                             try
  25.                             {
  26.                                 proxy.HandOverTo(newDict, true, true);
  27.                             }
  28.                             catch { }
  29.                         }
  30.                     }
  31.                 }
  32.                 else if (id.ObjectClass.IsDerivedFrom(zombieEntity) && !id.IsErased)
  33.                 {
  34.                     try
  35.                     {
  36.                         using (DBObject proxy = id.Open(OpenMode.ForWrite))
  37.                         {
  38.                             proxy.Erase();
  39.                         }
  40.                     }
  41.                     catch { }
  42.                 }
  43.             }
  44.         }
Title: Re: Proxy & AEC Clean/Remove
Post by: WILL HATCH on July 05, 2013, 12:04:27 PM
How do these compare in speed?
Title: Re: Proxy & AEC Clean/Remove
Post by: Jeff H on July 05, 2013, 05:57:28 PM
This might help speed it up a bit, and have not tested it, but could check ProxyFlags property to see if it can be erased instead of using catch block.
 
 
Since the ProxyEntity and Proxyobject use a int for its ProxyFlags property maybe something like
Code - C#: [Select]
  1.  
  2.          public static bool IsErasable(this ProxyEntity proxy)
  3.        {
  4.            return (proxy.ProxyFlags & 1) == 1;
  5.        }
  6.  
Title: Re: Proxy & AEC Clean/Remove
Post by: CADDOG on July 08, 2013, 10:48:09 PM
Repackaged gile's iterative method, which is nifty without having to expose acdbHandEnt  :kewl:, into a transaction, but that's the only major difference.  I say this pre-emptively because the screen shot will give the impression the error shown is because of the differences.  I used gile's version in it's form, which worked awwweeeesome, until a random drawing was hit (using dbx and lisp), which was just as random as the screen shot version.

Code: [Select]
Autodesk.AutoCAD.Runtime.Exception was unhandled by user code
  Message=eInvalidOpenState
  Source=Acdbmgd
  StackTrace:
       at Autodesk.AutoCAD.DatabaseServices.OpenCloseTransaction.GetObject(ObjectId id, OpenMode mode, Boolean openErased, Boolean forceOpenOnLockedLayer)
       at Autodesk.AutoCAD.DatabaseServices.OpenCloseTransaction.GetObject(ObjectId id, OpenMode mode)
       at CADDOG_LispExpansionPack1.LispFunctions.CleanProxies(Database db, Transaction tr) in C:\Users\medleyr\Documents\Visual Studio 2010\Projects\CADDOG_LispExpansionPack1\CADDOG_LispExpansionPack1\CADDOG_LispExpansionPack1.cs:line 374
       at CADDOG_LispExpansionPack1.LispFunctions.ProxyClean(ResultBuffer rb) in C:\Users\medleyr\Documents\Visual Studio 2010\Projects\CADDOG_LispExpansionPack1\CADDOG_LispExpansionPack1\CADDOG_LispExpansionPack1.cs:line 357
       at Autodesk.AutoCAD.Runtime.CommandClass.InvokeWorker(MethodInfo mi, Object commandObject, Boolean bLispFunction)
       at Autodesk.AutoCAD.Runtime.CommandClass.InvokeWorkerWithExceptionFilter(MethodInfo mi, Object commandObject, Boolean bLispFunction)
       at Autodesk.AutoCAD.Runtime.PerDocumentCommandClass.Invoke(MethodInfo mi, Boolean bLispFunction)
       at Autodesk.AutoCAD.Runtime.CommandClass.CommandThunk.InvokeLisp()
  InnerException:


So I figured workingdatabase could be an issue, as it was in the past, but that proved a dead-end.

If I open the file into the editor, the function would run and effectively clean the drawing of proxies.  But when in batch, random drawings would give a fatal error.  Handling the error is not a solution, as that would leave a proxy behind.

Any ideas?

Code - C#: [Select]
  1.         [LispFunction("cd:ProxyClean")]
  2.         public Boolean ProxyClean(ResultBuffer rb)
  3.         {
  4.             if (rb == null)
  5.             { return false; }
  6.             TypedValue[] args = rb.AsArray();
  7.             if (args.Length != 1 || (LispDataType)args[0].TypeCode != LispDataType.ObjectId)
  8.             { return false; }
  9.             //we accept any object from the drawing that is to be processed.  I trend in sending modelspace, but it doesn't matter
  10.             //just need an object so the owning database can be retrieved
  11.             ObjectId objId = (ObjectId)args[0].Value;
  12.             Database db = objId.Database;
  13.             using (Transaction tr = db.TransactionManager.StartOpenCloseTransaction())
  14.             {
  15.                 CleanProxies(db, tr);
  16.                 tr.Commit();
  17.                 return true;
  18.             }
  19.         }
  20.  
  21.         //Alexander's algorithm ??
  22.         //gile - http://www.theswamp.org/index.php?topic=44880.msg501032#msg501032
  23.         private void CleanProxies(Database db, Transaction tr)
  24.         {
  25.             RXClass zombieEntity = RXClass.GetClass(typeof(ProxyEntity));
  26.             RXClass zombieObject = RXClass.GetClass(typeof(ProxyObject));
  27.             ObjectId id;
  28.             for (long l = db.BlockTableId.Handle.Value; l < db.Handseed.Value; l++)
  29.             {
  30.                 if (!db.TryGetObjectId(new Handle(l), out id))
  31.                     continue;
  32.                 if ((id.ObjectClass.IsDerivedFrom(zombieObject) || id.ObjectClass.IsDerivedFrom(zombieEntity))
  33.                     && !id.IsErased)
  34.                 {
  35.                     DBObject proxy = tr.GetObject(id, OpenMode.ForWrite) as DBObject;
  36.                     using (DBObject newObj = new Xrecord())
  37.                     {
  38.                         //try
  39.                         //{
  40.                             proxy.HandOverTo(newObj, false, false);
  41.                             newObj.Erase();
  42.                         //}
  43.                         //catch
  44.                         //{
  45.  
  46.                         //}
  47.  
  48.                     }
  49.                 }
  50.             }
  51.         }
  52.  

Current lisp calls for cd:setworkingdatabase

Code - C#: [Select]
  1.         [LispFunction("cd:SetWorkingDatabase")]
  2.         public object SetWorkingDatabase(ResultBuffer rb)
  3.         {
  4.             if (rb == null)
  5.             {return false;}
  6.             TypedValue[] args = rb.AsArray();
  7.             if (args.Length != 1)
  8.             {return false; }
  9.             try
  10.                 {              
  11.                 Database db = ((ObjectId)args[0].Value).Database;
  12.                 HostApplicationServices.WorkingDatabase = db;
  13.                 return true;
  14.                 }
  15.                 catch (System.Exception)
  16.                 {
  17.                 return false;
  18.                 }
  19.         }
  20.  

And the lisp that uses it.

Code: [Select]
;;ProxyClean v0.1
;;
;;by Ricky Medley - CADDOG Computing
;;ricky.medley@caddog.net
;;Additional Credit: LeeMac LM:GetFiles LM:ODBX Wrapper LM:GetFilesAll LM:BrowseForFolder
;; Thank you Lee for making my life easier
;;
;;GNU GPLv3 licensed
;;http://www.gnu.org/licenses/gpl.html for preamble
;;This means you can modify, and distribute as long as the source code or any derivitive of this
;;code is provided and distributed as well.  You can even compile and sell, but you must provide
;;this source code in the package, and notify the recipient of the license as well.  Porting to
;;other languages as far as form and structure are also applicable to this license.
;;Modification to any code must be submitted back to ricky.medley@caddog.net for as long as the
;;address is valid.
;;The idea is not copyrighted.
;;Feel free to change the name of the command
;;
;;Program Description: Cleans all proxy objects from drawing or batch of drawings.
;; The command used, cd:ProxyClean, is indescriminate, and will
;; clean proxy objects from their proxy data regardless if object
;; enablers are installed.  Please use with caution.
;;
;;Dependencies:
;;;LM:GETFILES - http://www.lee-mac.com/getfilesdialog.html
;;;LM:ODBX - http://www.lee-mac.com/odbxbase.html
;;;LM:GETALLFILES
;;;LM:BROWSEFORFOLDER
;;
;;Future: cd:proxyclean to report back the name/description of the application of the proxy that
;; was cleaned.  This data will need to be handled and will be in a list format.
;;
;;History:
;;6/26/2013 v0.1 - Initial Build v0.1:
;; -
;;

(defun c:ProxyClean (/
      *Error*
      releaseobj
      get-time
      get-utime
      _getsavefile
      process
      cd:dependency
      cd:Get-WebUtils
      vl:list->string

      doc
      process
      opt
      )
  (vl-load-com)

    (defun *error* (msg)
      (cd:setworkingdatabase (vlax-vla-object->ename (vla-get-modelspace (vla-get-activedocument (vlax-get-acad-object)))))
      (releaseobj doc)
    (if (not (wcmatch (strcase msg) "*BREAK,*CANCEL*,*EXIT*"))
      (progn
(princ (strcat "\nXref Clean Error: " msg))
(vl-bt)
)
      (princ)
      );if
    );defun *Error*
 
  ;from LeeMac
  (defun releaseobj (obj)
    (if
      (and
(eq 'VLA-OBJECT (type obj))
(not (vlax-object-released-p obj))
)
      (vl-catch-all-apply 'vlax-release-object (list obj))
      )
    );defun releaesobj

  (defun Get-Time (format rev ;return time in two string parts ("5-2-2011";format derives 12 or 24 (feed integer) - default is 24
   ;rev refers to reversing the order from american(relevance order) to logical(numberical order)
   ;rev is t or nil
   /
   ctime cy cm cd ch cm cs suf
   )
    (setq ctime (rtos (getvar "CDATE") 2 6)
  cy (substr ctime 1 4)
  cm (substr ctime 5 2)
  cd (substr ctime 7 2)
  ch  (atoi (substr ctime 10 2))
  cmn  (substr ctime 12 2)
  cs  (substr ctime 14 2)
  suf "")
    (if (= format 12)
      (progn
(if (> ch 12)
  (setq ch (- ch 12)
suf " PM")
  (setq suf " AM")
  );if
);progn
      );if
    (if (= rev t)
      (list (strcat cm "-" cd "-" cy) (strcat (itoa ch) ":" cmn ":" cs suf))
      (list (strcat cy "-" cm "-" cd) (strcat (itoa ch) ":" cmn ":" cs suf))
      );if
    );defun
 
  (defun get-utime ()
    (* 86400 (getvar "tdusrtimer"))
    )


  (defun vl:list->string (delim ls / out)
    (setq out "")
    (mapcar (function (lambda (x)
(setq out (strcat out x delim))))
    ls)
    (if out (vl-string-right-trim delim out))
    );defun list->string

  (defun _getsavefile (prmpt dfltFile fileExt / saveFileName)
    (if (not prmpt) (setq prmpt "Choose Save File"))
    (if (not dfltFile) (setq dfltFile "Output"))
    (if (not fileExt) (setq fileExt "txt"))
    (setq saveFileName (getfiled prmpt dfltFile fileExt 129))
    );defun _getsavefile

   ;need to optimize to accept a list of dependencies
  (defun cd:dependency (/
LM:GetFiles_url
LM:ODBX_url
opt
resource_directory
)
    ;;lee-macs site is not responding... for the past month or so.
    (setq LM:GetFiles_url "http://www.caddog.net/opensource/lisp/lee-mac/GetFilesV1-1.lsp")
    (setq LM:ODBX_url "http://www.caddog.net/opensource/lisp/lee-mac/ObjectDBXWrapperV1-1.lsp")
    (setq resource_directory (strcat (getenv "appdata") "\\Autodesk\\ApplicationPlugins\\Lee-Mac-Utils"))
   
    ;check for required functions
    (if (or (not LM:ODBX)(not LM:GetFiles)(not LM:GetAllFiles)(not LM:BrowseForFolder))
      ;check for variations of the lisp files that might contain the functions
      (if (or (and (setq LM:GetFiles (findfile "GetFiles.lsp"))
   (or (setq LM:ODBX (findfile "ObjectDBXWrapper.lsp"))
       (setq LM:ODBX (findfile "ODBX.lsp"))))
      (and (setq LM:GetFiles (findfile (strcat resource_directory "\\" (vl-filename-base LM:GetFiles_url) (vl-filename-extension LM:GetFiles_url))))
   (setq LM:ODBX (findfile (strcat resource_directory "\\" (vl-filename-base LM:ODBX_url) (vl-filename-extension LM:ODBX_url)))))
      );or
;found them, so load them
(progn
  (load LM:GetFiles)
  (load LM:ODBX)
  );
;else, let's ask to download them.
(progn
  (princ "\nIn order to proceed, Lee-Mac lisp functions must be downloaded:")
  (princ (strcat "\n\t" LM:GetFiles_url))
  (princ (strcat "\n\t" LM:ODBX_url))
  (initget "Yes No")
  (setq opt (getkword "\nAllow connection to CADDOG.net [Yes/No] <Yes>: "))
  (cond ((= opt "No")
;(vl-exit-with-error "Cannot Continue") ;this exit method may not be right
;(princ "Cannot Continue")
;(exit)
)
;(cd:Get-LeeMacUtils DownloadDestination
(t
(if (setq return (cd:Get-WebUtils resource_directory (list LM:GetFiles_url LM:ODBX_url)))
   (mapcar 'load return))
)
);cond
  );progn
);if LM files exists
      );if LM functions defined
    (if (and LM:ODBX LM:GetFiles)
      t
      nil)
    );defun cd:dependency
 
  (defun cd:Get-WebUtils (dest url_list
  /
  *error*
  dest
  url
  msxml
  ADOstream
  return
  )
    (defun *error* (msg /)
      (if msxml (vlax-release-object msxml))
      (if ADOstream (vlax-release-object ADOstream))
      (setq msxml nil
    ADOstream nil)
      (vl-exit-with-error "Get-WebUtils Failed")
      )
   
    (if (setq msxml (vlax-create-object "MSXML2.XMLHTTP"))
      (progn
(vl-mkdir dest)
(foreach url url_list
  ;test if web location exists
  (vlax-invoke-method msxml 'Open "GET" url :vlax-false)
  (vlax-invoke-method msxml 'Send)
  (princ (strcat "\nInquiring: " url))
  (setq status (vlax-get-property msxml "status"))
  (if (= 200 status)
    (progn
      ;start a stream to download
      (setq ADOstream (vlax-create-object "ADODB.Stream"))
      (vlax-invoke-method ADOstream 'Open nil nil nil nil nil)
      (vlax-put-property ADOstream "Type" 1)
      (vlax-invoke-method ADOstream 'Write (vlax-get-property msxml "responseBody"))
      (vlax-put-property ADOstream "Position" 0)
      (setq return (append (list (strcat dest "\\" (vl-filename-base url) (vl-filename-extension url))) return))
      (vlax-invoke-method ADOstream 'SaveToFile (car return) 2)
      (vlax-invoke-method ADOstream 'Close)
      (vlax-release-object ADOstream)
      (princ (strcat "\nSaving To: " (car return)))
      );progn status = 200
    ;else, tell user
    (princ (strcat "\nFileGet Error: " (vlax-get-property msxml "statusText")
   "\n\t" url))
    );if status = 200
  );for each url url_list
(vlax-release-object msxml)
);progn
      (princ "Connection could not be established.")
      );if createobject
    return
    );defun cd:Get-WebUtils
 
 
  (defun _write_data (data filename
      /
      file
      dwg
      xref
      )
    (if (and data filename
     (setq file (open filename "w"))
     );and
      (progn
(write-line (vl:list->string "\t" '("dwg" "proxy_clean_status")) file)
(foreach dwg data
  (foreach xref (cdr dwg)
    (setq clean (mapcar (function (lambda (x) (if x x ""))) xref))
    (write-line (vl:list->string "\t" (append (list (car dwg)) clean)) file)
    );foreach xref
  );foreach dwg
(close file)
);progn
      );if data and filename
    );defun _write_data


  (defun cd:proxy_clean (doc
/
docpath
outdata
logdata
clean-status
)
    ;this test is the difference between dbx and current file
    (if (vlax-property-available-p doc 'FullName) ;only available in UI drawings
      (setq docpath (vl-filename-directory (vla-get-fullname doc)))
      ;else we are in DBX, and need to get a different property and load xrefs
      ;and try set dbx doc as working database
      (progn
(setq docpath (vl-filename-directory (vla-get-name doc)))
(cd:setworkingdatabase (vlax-vla-object->ename (vla-get-modelspace doc)))
);progn
      );if current dwg
    (if (cd:proxyclean (vlax-vla-object->ename (vla-get-modelspace doc)))
      (setq clean-status "clean-ran")
      (setq clean-status "clean-failed")
      );if cd:proxyclean
    (cd:setworkingdatabase (vlax-vla-object->ename (vla-get-modelspace (vla-get-activedocument (vlax-get-acad-object)))))
    (list (list clean-status))
    );defun cd:proxy_clean

(defun process (runfolder simulate /
  doc
  dwgname
  data
  name
  save
  )
    (setq save (not simulate)) ;usage is opposite on LM:ODBX which is a save or notsave var
    ;so simulate=true is save=false
    (setq doc (vla-get-activedocument (vlax-get-acad-object))
  dwgname (vla-get-fullname doc))
    (cond ((not runfolder)
   (setq data (list (append (list dwgname) (cd:proxy_clean doc))))
   )
  ((and (setq name (strcat (vl-filename-directory dwgname) "\\"))
(setq runfolder (LM:GetFiles "Select Files to Strip Proxy Data" name "dwg"))
(setq name (strcat (vl-filename-directory dwgname) "\\ProxyClean-Log_" (car (get-time 12 nil))))
(setq name (_getsavefile "Choose Log Save File" name "TSV"))
(setq ctime (get-utime))
(setq data (LM:ODBX 'cd:proxy_clean runfolder save))
);and
   )
  );cond
    (cond
      ((and data name)
       (_write_data data name)
       (princ "\nProxyClean Complete")
       (princ (strcat "\nOpening " name))
       (startapp "notepad" name)
       (princ (strcat "\Time: " (rtos (- (get-utime) ctime) 2 2) " seconds"))
       )
      (data
       (princ "\nProxyClean Complete")
       )
      (t (princ "\nProxyClean Failed!!!"))
      );cond
    );defun process

  (setq doc (vla-get-activedocument (vlax-get-acad-object)))
  (setq version "v0.1")

  (initget "Select Current")
  (setq opt (getkword "ProxyClean: Process [Select files/Current dwg] <Current dwg>: "))
  (cond
    ((= opt "Select")
     (if (cd:dependency)
       (progn
(initget "Yes No")
(setq opt (getkword "\nSimulated Run? [Yes/No] <No>: "))
(cond
   ((= opt "Yes")
    (process t t))
   (t (process t nil)
    )
   );cond
);t all components in place, can batch
       (princ "Required Items Not Loaded for Batch")
       );if cd:dependency
     )
    ;else
    (t ;t Current drawing only
     (if (cd:dependency)
       (progn
(setq doc (vla-get-activedocument (vlax-get-acad-object)))
(vla-endundomark doc)
(vla-startundomark doc)
(process nil nil)
(vla-endundomark doc)
);progn
       );if cd:dependency
     );t Current drawing only
    );cond
  (releaseobj doc)
  (princ)
  );defun c:XrefBindAll
Title: Re: Proxy & AEC Clean/Remove
Post by: irneb on July 09, 2013, 01:23:30 AM
It might be a situation of the batch-mode falling over its own feet. As it's reading the DB to "open" the DWG, most objects are still locked for reading. Then your function is called and tries to open said object for write - these locks are mutually exclusive, so the open fails.

It might be an idea to give it some time to complete opening and only then run the code. Either some form of Wait in your lisp/script (might need to program such function through DotNet since you can't send commands to DBX), or look for an event about database open completed, or run the lisp through a script, or run the code through the s::startup function ...

Many ways you could try to give the DBX engine time to complete its open. You could even simply add the proxy objects to a queue and start a thread which runs through this queue until it's empty. The try can then simply fail and leave said object on the queue - seeing as it's going to loop and try again later. I'd definitely try to do this through a separate thread though - otherwise acad might feel locked up. You could then have another method checking for the progress of that queue - allowing something like a progress bar for user fed-back.
Title: Re: Proxy & AEC Clean/Remove
Post by: CADDOG on July 09, 2013, 09:56:06 PM
You could even simply add the proxy objects to a queue and start a thread which runs through this queue until it's empty. The try can then simply fail and leave said object on the queue -

Good call.  Solved that problem.  Now another one has reared it's ugly head.  In dbx, all objects are of type AcRxObject.   :ugly:   I'm sorry... I just can't stop giggling in total frustration.

Gonna go back to casting to see if that can trap it in either a proxyentity or proxyobject.
Title: Re: Proxy & AEC Clean/Remove
Post by: CADDOG on July 09, 2013, 11:29:05 PM
Casting works in both dbx db's and mdiactivedocument db's.  See last function GetDBProxyCol.

This implementation implements irneb's suggestion of finding, collecting, then cleaning in a while loop.  Worked like a charm.

Also of relevance: I removed the call to lispfunction to change working databaes.  I didn't think I needed it, and was only a test.  The only time I've needed to set working database was when working on attributes.

Code - C#: [Select]
  1.         [LispFunction("cd:ProxyClean")]
  2.         public Boolean ProxyClean(ResultBuffer rb)
  3.         {
  4.            
  5.             if (rb == null)
  6.             { return false; }
  7.             TypedValue[] args = rb.AsArray();
  8.             if (args.Length != 1 || (LispDataType)args[0].TypeCode != LispDataType.ObjectId)
  9.             { return false; }
  10.             //we accept any object from the drawing that is to be processed.  I trend in sending modelspace, but it doesn't matter
  11.             //just need an object so the owning database can be retrieved
  12.             ObjectId objId = (ObjectId)args[0].Value;
  13.             Database db = objId.Database;
  14.             using (Transaction tr = db.TransactionManager.StartOpenCloseTransaction())
  15.             {
  16.                 CleanProxies(GetDBProxyCol(db, tr), db, tr);
  17.                 tr.Commit();
  18.                 return true;
  19.             }
  20.         }
  21.  
  22.         //recieves a collection of proxy ONLY objectids.  No checking to see if that's true
  23.         //will loop until all proxies are successfully stripped (opened for write)
  24.         private void CleanProxies(ObjectIdCollection proxyCol, Database db, Transaction tr)
  25.         {
  26.             while (proxyCol.Count > 0)
  27.             {
  28.                 try
  29.                 {
  30.                     DBObject proxy = tr.GetObject(proxyCol[0], OpenMode.ForWrite) as DBObject;
  31.                     using (DBObject newObj = new Xrecord())
  32.                     {
  33.                         proxy.HandOverTo(newObj, false, false);
  34.                         proxyCol.RemoveAt(0);
  35.                     }
  36.                 }
  37.                 catch
  38.                 { } //catch to try try again
  39.             }
  40.         }
  41.  
  42.         //Alexander's algorithm ??
  43.         //gile - http://www.theswamp.org/index.php?topic=44880.msg501032#msg501032
  44.         //managed transaction base to retrieve all proxy objects in drawing by cycling through every object id db
  45.         private ObjectIdCollection GetDBProxyCol(Database db, Transaction tr)
  46.         {
  47.             ObjectIdCollection proxyCol = new ObjectIdCollection();
  48.             ObjectId id;
  49.             for (long l = db.BlockTableId.Handle.Value; l < db.Handseed.Value; l++)
  50.             {
  51.                 if (!db.TryGetObjectId(new Handle(l), out id))
  52.                     continue;
  53.                 if (!id.IsErased)
  54.                 {
  55.                     DBObject dbObj = tr.GetObject(id, OpenMode.ForRead) as DBObject;
  56.                     ProxyObject proxyObj = dbObj as ProxyObject;
  57.                     ProxyEntity proxyEnt = dbObj as ProxyEntity;
  58.                     if (proxyObj != null || proxyEnt != null)
  59.                         proxyCol.Add(id);
  60.                 }
  61.             }
  62.             return proxyCol;
  63.         }
  64.  

I can finally rest.  Thanks to everyone who's input made this happen.
Gile for the primary source
Irneb for the directional guidance.
Jeff H for the base understanding.
And Alexander Rivilis for the suggestion of db iteration so that nothing is missed

I can't thank you guys enough, as I've been struggling through this one for three weeks (almost full time), and I was worried that I could have cycled the drawing manually by now.
Title: Re: Proxy & AEC Clean/Remove
Post by: irneb on July 10, 2013, 02:57:23 AM
This implementation implements irneb's suggestion of finding, collecting, then cleaning in a while loop.  Worked like a charm.
Glad it worked for you.

Edit: Though I'd definitely try to change that loop into a thread instead. I know it's a lot more complex, but you'll find that you can do a lot more very easily. E.g. what if there's some error outside your control (say ACad trying to open a corrupt DWG)? That loop will just continue indefinitely. With a thread you can always abort after a specified maximum time so that the rest of your code can continue.

You could probably also write something like that into your while loop. Say set a start-time variable from DateTime.Now() (http://msdn.microsoft.com/en-us/library/system.datetime.now.aspx), then have your loop check the ellapsed time as a second exit option.
Code - C#: [Select]
  1. private void CleanProxies(ObjectIdCollection proxyCol, Database db, Transaction tr) {
  2.     DateTime stop_at  = DateTime.Now().AddMilliseconds(60000); // Set maximum stop time 1 minute after starting
  3.     while ((proxyCol.Count > 0) && (stop_at > DateTime.Now())) {
  4.         try {
  5.             DBObject proxy = tr.GetObject(proxyCol[0], OpenMode.ForWrite) as DBObject;
  6.             using (DBObject newObj = new Xrecord()) {
  7.                 proxy.HandOverTo(newObj, false, false);
  8.                 proxyCol.RemoveAt(0);
  9.             }
  10.         }
  11.         catch { } //catch to try try again
  12.     }
  13. }
Title: Re: Proxy & AEC Clean/Remove
Post by: CADDOG on July 15, 2013, 10:54:25 AM
Inerb, Ok.  Will do.  Thanks for the example.

Everything has run and completed, and files are on their way to client (clean as a whistle).  Thanks to everybody again.