Topic: Unit Testing in AutoCAD

[kdub_nz]

Has anyone done any work incorporating Unit Testing (Test First Development) with AutoCAD apps ??

If so, care to share opinions ?

Regards
kdub

[kaefer]

Gallio Release Note 3.0.5

AutoCAD Integration

Mike Sandberg has added support for testing AutoCAD plugins.

It turns out that AutoCAD has a managed extensibility model so you can create your own plugins using .Net and the ObjectARX toolkit.  Unfortunately it is somewhat difficult to write unit tests for plugins becuase they must run within the main UI thread of AutoCAD.

The AutoCAD integration for Gallio works by loading a shim into the AutoCAD application from which it can launch tests.  To enable this integration, specify the "AutoCAD" runner type to the Echo, Icarus, MSBuild, NAnt or PowerShell runners.

For example:

    Gallio.Echo.exe MyTestAssembly.dll /r:AutoCAD [other options...]

AutoCAD integration is not yet available from within the IDE.  We will be working to improve this use case in the future.

[kdub_nz]

Thanks kaefer ..

 seems that I have a heap more study to do


regards,

[kaefer]

I was just about able to deploy stand-alone tests by referencing Gallio.dll and MbUnit.dll and using the Icarus test runner. Barring a certain amount of study I have no idea how to implement the technique described above, let alone devising meaningful tests against the AutoCAD drawing Database.

[Andrey Bushman]

2 kaefer

Can you show an source code of your unit test example (for AutoCAD)?

Regards

[kaefer]

Can you show an source code of your unit test example (for AutoCAD)?

Sorry, can't. I've still no idea. Anyone?

[Andrey Bushman]

I am able to use NUnit and MSTest a little, but Gallio to me isn't clear. Whether correctly I understood your question: are you want, that I gave an example, when it can be necessary in AutoCAD?

[CADbloke]

I use NUnit in the Resharper Test runner in Visual Studio 2012. If you try to use an AutoCAD-dependent object in the test it crashes the test (but not the runner) with  System.InvalidProgramException : Common Language Runtime detected an invalid program but you can test classes that have AutoCAD objects as properties by calling a constructor that sets the AutoCAD-based property to null. I evolved this using properties which were actually a class that contained AutoCAD objects (RXClass) properties so this can work down down down into layers.

I use a constructor with a boolean flag for unit tests. The other constructor(s) call that flagged constructor before they run. This means that the constructor you are usually looking for to use in anger is as you expect. The unit testing constructor shows up in Intellisense so that is why I have the boolean parameter an obvious name. I

You can test test some AutoCAD elements, namely UI elements that are handled by the SDK DLLs. I used a Palette tab in a test that checked some UI elements against a settings class and it works fine. "handled", did I say? Yes, that's how the SDK DLLs work with the Visual Studio designer and they seem to work in the tests too.

To use the SDK DLLs in the Unit Tests you need to reference them in the test project (duh!) and also set Copy Local to True (wut!). Yes, the tests need to be in their own project but you already do that, right?

In my example below you may notice I use internal classes. The class you're testing doesn't necessarily need to be public ... BUT if you want to test internal classes then you need to add `[assembly:InternalsVisibleTo("CADFindReplaceTests")]` in your app's AssemblyInfo.cs. Google InternalsVisibleTo and also be aware of the security implications of shipping assemblies that have a back-door like the one this creates.

Here's a highly-contrived example of what I am on about ...

Code - C#: [Select]

1. using NUnit.Framework;
2. using Lots.Of.Google;
3.  
4.  
5. [Test()]
6. [STAThread]
7. public void Test_Breaks()
8. {
9.   ThingWithAutocadBits thing = new ThingWithAutocadBits(unitTesting); // Bang !
10.         Assert.Fail("FAIL");  // You won't get this far
11. }
12.  
13.  
14. [Test()]
15. [STAThread]
16. public void Test_Runs_But_My_Code_Sucks()
17. {
18.   ThingWithAutocadBits thing = new ThingWithAutocadBits(unitTesting: true);
19.         Assert.Pass("#FAIL");  // aw, damn! There is no justice.
20. }

Code - C#: [Select]

1. internal class ThingWithAutocadBits
2. {
3.         internal string ThingName { get; set; }
4.         internal bool TellMeItsNotTrue { get; set; }
5.         internal AutoCADObjectOfSomeSorts iBreakThings { get; set; }
6.  
7.  /// <summary>
8.  /// This Ctor is for Unit Testing - it avoids the dependency on AutoCAD objcts of doom
9.  /// It is always called by the default Ctor.
10.  /// </summary>
11.  /// <param name="UnitTesting">Value here doesn't matter, just its presence.</param>
12.         internal SearchOptions(bool UnitTesting = true)
13.         {
14.         ThingName = "Nigel";
15.         TellMeItsNotTrue = true;
16.         iBreakThings = null;
17.         }
18.  
19.  
20.         internal ThingWithAutocadBits() : this (true)
21.         {
22.         iBreakThings = Whatever.You.Like.Eh;
23.         }
24. }

[Andrey Bushman]

I use the Gallio. It works fine.

[CADbloke]

I've just discovered https://bitbucket.org/IsaacRodriguez/civilizeddevelopment/src/ - this approach looks interesting. It should be relatively easy (compared to solving the wars over politics and religion) to adapt this to run tests in Visual Studio Test Runners for any framework, or just use the frame work here. There seems to be some quite good ideas in here. Can't type ... reading.

[Andrey Bushman]

this approach looks interesting.
...  to adapt this to run tests in Visual Studio Test Runners for any framework, or just use the frame work here.

What exactly? Can MSVS use the Gallio platform also? Did you try it?

[CADbloke]

What exactly?

Good Question. Thank you, you made me think about it a lot more because I didn't have an instant "good answer".

You could use the Test runner in that app to run a set of tests inside AutoCAD that needed AutoCAD objects. These test would be a different set of test than you would run in something like NUnit against your code that has no external dependencies. The more I think about this the more I think it is a bad idea, mostly because it is too different from the normal, accepted way of doing things.

Can MSVS use the Gallio platform also? Did you try it?

Older versions of ReSharper had a test runners for Gallio. That's not a big deal, really, I don't mind using other Test Runners. NUnit's test runner can run a batch of tests so I can build all the different versions of an app and then run all the tests at once in NUnit. Gallio's Test runners look good, from what I have seen.

I looked at Gallio's source code but I have not tried it yet. My concern is that development has stopped on it. I could update it myself as it is on Github at https://github.com/Gallio/mbunit-v3/commits/master but that looks like it would be a lot of work in the future. Gallio is very big and complex and could easily turn out being more work to maintain than my projects I am supposed to be working on. Test frameworks like NUnit and XUnit are still very active and popular. Popular frameworks have a lot more support, code samples, documentation (Gallio's documentation is gone, only on the Web Archive).


I am currently considering using http://nsubstitute.github.io/ to use fake AutoCAD objects. NSubstitute builds fake objects from Interfaces, It is possible to build Interfaces for AutoCAD managed objects by decompiling AutoCAD's managed wrapper DLLs (I used Dotpeek) then use ReSharper to extract an Interface from the classes you want to fake (Resharper also extracts Interface members from the class's dependencies so you get a lot of members). It would probably be easier to just make the Interfaces yourself if they were simple.

I know that testing code with external dependencies is considered bad by unit-testing purists but I would also like to test the expected behaviour of those objects. Creating something like a fake DBText would allow me to set its properties and then test that my code is using those properties as I expect. Using fakes would also allow me to ignore any other behaviour I am not interested in testing...and probably save me from a lot of acad.exe crashes.

For those of you who got nervous when I mentioned decompiling AutoCAD DLLs, there is a precedent for this: http://adndevblog.typepad.com/autocad/2012/08/a-rich-source-of-autocad-net-sample-code.html and also at http://through-the-interface.typepad.com/through_the_interface/2010/02/debugging-into-autocads-net-api-layer-using-reflector-part-1.html - I'm not reverse-engineering AutoCAD, merely deconstructing it to test things. Well, that's my story.

@Andrey - I tried to keep my language clear here because I know your first language is not English, please let me know if anything is not clear to you. I would like to hear your thoughts on this.

[Jeff H]

I just can't see testing any code that relies on AutoCAD to ever be a reality.

It would think it would take so much time and effort to build fakes to behave like they would in AutoCAD environment and then with changes to maintain you would never have time to actually use it.
....
Number of fakes required to build, mass number of system variables whose settings will change the behavior of objects, some objects require a certain order for assigning properties, so many different contexts, etc....
 

[Andrey Bushman]

@Jeff H,

It is necessary to separate independent and integration testing. I did it for my some application. It requires many forces and time, but it gives me big confidence about the quality of my software.

[Andrey Bushman]

I know nobody who is interested the AutoCAD extensions testing (except me).

The more I think about this the more I think it is a bad idea, mostly because it is too different from the normal, accepted way of doing things.

It is necessary to separate independent and integration testing as I told in my previous message. It is usual common practice. I've learned this through this book. This is good book.

Older versions of ReSharper had a test runners for Gallio. That's not a big deal, really, I don't mind using other Test Runners. NUnit's test runner can run a batch of tests so I can build all the different versions of an app and then run all the tests at once in NUnit.

I prefer to use NUnit and Gallio.

I looked at Gallio's source code but I have not tried it yet.

I read its sources too and I have some expirience of Gallio using for AutoCAD testing.

My concern is that development has stopped on it.

It is not a problem for me because the sources are exist and I know how to extend it for newer AutoCAD versions (I did it already as a practice and it work fine). Current capabilities of Gallio are fine for me.

I am currently considering using http://nsubstitute.github.io/ to use fake AutoCAD objects. NSubstitute builds fake objects from Interfaces, It is possible to build Interfaces for AutoCAD managed objects by decompiling AutoCAD's managed wrapper DLLs (I used Dotpeek) then use ReSharper to extract an Interface from the classes you want to fake (Resharper also extracts Interface members from the class's dependencies so you get a lot of members).

I know what is it and how it works (the book has the materials about this). I use it too in my tests.

I know that testing code with external dependencies is considered bad by unit-testing purists but I would also like to test the expected behaviour of those objects.

Such tests are inevitable and usefull too.

For those of you who got nervous when I mentioned decompiling AutoCAD DLLs, there is a precedent for this

I have this official description, so I know what is .Net Reflector.

@Andrey - I tried to keep my language clear here because I know your first language is not English, please let me know if anything is not clear to you. I would like to hear your thoughts on this.

Thank you. I understood you. I am not a professional in the writting of the tests for AutoCAD extensions, but if this theme is interesting for you, then we can discovery it together later. For me is interesting to write the extensions for NUnit for capabilities of testing AutoCAD, nanoCAD, BricsCAD and any Teigha-based application.

Now I spend my time for learning of Haskell and WebGL. I didn't write a code for AutoCAD long ago.