Topic: Same Code, Multiple Projects, Multiple Builds

[CADbloke]

Edit: Oct 31, 2015. - Name change, Github repo name is changed.

Code Linker will only work in Visual Studio. It is not a build process, it is a way to share the same code between C# projects.

All Code Linker does is adds links to existing code files into a target csproj. No file is copied, no source code is parsed. The original csproj file is mined for its source code <Include=...> things and that information is used to add links to the target csproj file. That's all. It ends there. The target csproj is all it touches. It automates the add a file as a link to a C# project process. That link explains it really well, and also some of the reasons you would do this.

The code repo is at https://github.com/CADbloke/CodeLinker

There is an Illustrated example at https://github.com/CADbloke/CodeLinker/wiki/Illustrated-Example

There's an extensive readme there so I won't duplicate it here.



...original text for posterity's sake (posterior being a synonym for "ass" which is what I made of myself)...

(edit Oct12, 2015)  Scroll down to this post, that's where I started to make sense. I will post a better description when I have had enough sleep.

...
I'm having another crack at this at https://github.com/CADbloke/CodeLinker

This time around I'm focusing on propagating source code to destination Projects which control all their own build settings. Apparently some (all) of us though that 72 builds in one project was a little unmanageable </YesItWas>.

(edit Oct12, 2015)  No, I lied, I am creating links to existing source code. The source project is mined for code links and these links are written to the target project.

There's an extensive readme there so I won't duplicate it here. I'd love to hear any thoughts, suggestions, WTFs.

For those unaware of the history ...
http://www.theswamp.org/index.php?topic=41868.msg497509#msg497509
http://www.theswamp.org/index.php?topic=49039.msg541744#msg541744
http://www.theswamp.org/index.php?topic=49961

are just a few.

It's like looking at any of your work from years ago ... WTF? Hopefully this one will have a few less WTFs / Minute

[JohnK]

Git - Merging from a template project - no subdirectory
http://stackoverflow.com/questions/9106387/git-merging-from-a-template-project-no-subdirectory

[JohnK]

Generate C# project using CMake
http://stackoverflow.com/a/15418074

[CADbloke]

Git - Merging from a template project - no subdirectory
http://stackoverflow.com/questions/9106387/git-merging-from-a-template-project-no-subdirectory

This has potential but requires good Git-fu. Also, Google "csproj merge conflict"

Generate C# project using CMake
http://stackoverflow.com/a/15418074

I might be missing something there but keeping changing source code in sync with that doesn't sound easy. Can you elaborate ?

[JohnK]

Git - Merging from a template project - no subdirectory
http://stackoverflow.com/questions/9106387/git-merging-from-a-template-project-no-subdirectory

This has potential but requires good Git-fu. Also, Google "csproj merge conflict"

Generate C# project using CMake
http://stackoverflow.com/a/15418074

I might be missing something there but keeping changing source code in sync with that doesn't sound easy. Can you elaborate ?

`Git-fu' lol (that's made me laugh) it's just one command git remote add.... I googled the csproject merge conflict but I have zero context (I don't have any reliance on the VS IDE) as to what it is about. Is this something you can live with? This option fits your criteria the best and offers you the ability to update or not (Very flexible).
If you're worried about the git-fu you can build a few batch scripts to checkout and update from the template.

The second option is a bit of an enigma for me (the VS stuff confuses me; I don't use it and the VS IDE holds your hand too much) but how I understand the CMake option is that your template project is referenced in place (no copying of files to new projects).
This option is nice but I have to warn you though, it would be far easier to learn Git better then to pick up CMake. CMake is a beast (and a fickle one at that)! ...don't get me wrong, I love it, but man can it be a pain to set up your project the way you want to sometimes.

I recommend setting up a quick test of the first one (git option) and see how you like it.

[CADbloke]

Thanks of the follow-up. The CMake approach looks like a departure for Visual Studio's csproj projects? That's not a direction I'm currently interested in going.  I really like Visual Studio hence my latest approach to clone the code between projects inside the IDE, if you change the code in one project it changes for all of them, although you need to add and remove any "universal" files from the original project which could be a pain-point if you stray from the original source project. The only reason I expect to be editing code in target projects is to fix any differences in code between the various APIs or to fix failing unit tests (after I fix CADtest). I also like seeing in the IDE what code the #if directives are affecting and what API classes are actually being called. Yes, I like having my hand held.

If CMake does embrace csproj then it looks a bit like the proliferation of project imports I wrote about at  http://www.theswamp.org/index.php?topic=41868.msg497509#msg497509 which eventually breaks for me because importing a project drags in framework and build settings, References as well as any Nuget package specifics. It also tends to bury things a bit too deeply to quickly make sense of them, especially if I am coming back to something I haven't looked at for quite a while. I have zero experience with CMake so I may be way off the mark here.

The Git approach has potential but does this mean you need to write the code in a project that doesn't compile (gasp! No Intellisense) and juggle Git Remotes. No, not so hard really but I am trying to keep it all "simple". In theory. Your Git approach works for you perhaps, as you said, because you don't work in Visual Studio?

I also wanted a system I can drop in on top of stuff I've already written without breaking it. You never break the source project because it never writes to it, although your code in it will change with various #if directives if your platforms vary widely.

This article: http://haacked.com/archive/2014/04/16/csproj-merge-conflicts/ explains what I mean by .csproj merge conflicts. Understanding what is going on in the csproj files, and keeping an eye on it is important with this approach. They are just XML text files so the differences will be very obvious in commit diffs but you need to know what to expect. Once you've seen a few sets of changes then you will know what looks odd. Also, reverting changes is easy.

I tested Code Cloner with a WPF AutoCAD Project and it worked well, it all came together, looks good in Visual Studio (hey, I care) and builds fine.

[JohnK]

I'm sorry CADbloke but I am this close to giving up on you. Have you run a test (because I just did)? I'm sorry if that seems rude but I just don't know how you structure your code/projects so you will have to be the one driving this, not me.

CMake is a cross platform build system. That means you can take code and compile it on a windows, Mac, Linux, and Unix machine. You do this by writing a "recipe of how to build" your project. CMake takes that recipe and builds a build system for your platform (so in windows, cmake can build a VS project from your code and recipe. In Mac, it would build an xcode project from your code and recipe. In unix, it would build a Makefile. etc. etc.). Forget CMake, this would work, but it would be too much command line for you (you'd have to type way too much).

Git, just tracks your files. It has nothing to do with your code compiling or not (or Intellisense; that is a feature of your editor/IDE not your code). You can decide to track anything you want; if you want to track csproject or not is up to you and how you structure your template. Visual studio has built in support for Git but it is a really weird experience; you will have to open the command line and type.

Run a test!!
1. Create a template project: "[EX]_VS_Template-Project"
2. Close and create another project: "[EX]_VS_Test-Project-One"
3. Open the command line and add a reference to the previous one: git remote add -f template ../[EX]_VS_Template-Project
4. Merge the template project to your new project: git merge template/master
5. Add template project to your new one (I right clicked on the "Solution '[EX]_VS_Test-Project-One'" in the solution explorer and choose ADD EXISTING PROJECT).

NOTE:
1. Apparently, you can use wildcards in csproject files so I don't know if that helps you at all (with the compile/intellisense statement).

Code: [Select]

<ItemGroup>
  <Compile Include="${DOS_STYLE_SOURCE_DIR}\**\*.cs" />
</ItemGroup>
Now, all that said, this is the hand holding that I do NOT like. -i.e. The directory structure that VS keeps is horrendous! I would go bat-crap crazy trying to look at this all day. I solve this problem and other problems like this in my C++ projects with a combination of a directory structure, and CMake.

[CADbloke]

I'm sorry CADbloke but I am this close to giving up on you.

Perhaps you should. I have no desire to waste your time and if Visual Studio projects don't interest you (or even offend you) then perhaps this topic is not for you. You clearly haven't read the readme at the Github project I pointed to in the first post. My objectives all there - it is clear there (although not so much here) that I want this to work in Visual Studio. I also note how I structure my projects there.

Nor do you seem to have referred to any of the historical posts I linked to. I mention the wildcard usage in C# project files and its fragility both in the Github readme and in linked posts. Wildcards are an MSBuild thing, Visual studio tolerates them to a point but doesn't natively embrace them - to the extent that it will enumerate them into individual links if you happen to add a file into the folder tree you linked with wildcards. I can't beging to tell you how much fun this is if you happen to miss it at the time.

Git, just tracks your files. It has nothing to do with your code compiling or not (or Intellisense; that is a feature of your editor/IDE not your code).

Yes, I know, I have this Github thing and some experience with Mercurial. My point is that you should be writing most of the code in the original  template project. If that doesn't compile, have SDK references etc. then you are basically coding blind. Yes, you could write the code in one of the target projects but then you need to remember to add, remove, rename, refactor etc in the original template project. Any build settings in the template project will clash with the target projects. I also like Visual Studio's autocomplete, it types faster than me. See also Resharper which holds my other hand.

You hate Visual Studio (you said so) and seem to want to save me from it. Thank you for your concern but I'll be ok. I make no apologies for my lesser calibre in this field (with the possible exception of csproj files) and my affection for Visual Studio's "hand-holding", you clearly have a broader and deeper experience than me in these things. So did Tony Tanzillo and his condescending tone killed a lot of discussions too. I am sure there are plenty of people reading this who won't want to add to the discussion for fear of being made to look stupid. A quick peruse of my post history will show that I have overcome this fear but most have not. The same elitism has also stunted Stackoverflow.com.

As an aside, I have been known to type thing on a command line from time to time. I even once broke a nail doing so.

I will start a new post with a more clearly defined headline, my apologies for the overly ambitious scope I declared in my original headline.

Thank you for your explanation of CMake, that is a much better one than they have on their page.

I do sincerely thank you for your suggestions and help but I will call you out for your tone, your bias against Visual Studio is your problem (not mine) and making me look like a dill is both a community killer and also redundant - I don't need you help on that one .

cheers
Ewen

[JohnK]

VS doesn't 'offend' me at all. I've used it several times, I just prefer to use my editor and a compiler instead of conforming to an IDE. I've seen both sides of the fence so feel as though I can have an opinion.

Feel free to call me out for any tone you feel you've perceived.

Yes, I have read all the links you provided. I read your read me twice. I do not understand most of the links because we speak different languages. Nonetheless, I am still trying to help. No, I don't have time to read everything nor do I have time to do a bunch of tests or back reading up on for some concept I do not fully grasp yet.

Tony was a little rough at times but he was always very helpful towards me but I've always understood that someone with his abilities was probably very busy so I tried not to read too much into "tone" -I.e tried to take the written word at face value and not read much into it other then help and direction.

Again, I've used VS a few times. I even ported one of my projects for fun. ...I have zero desire to 'save' you from anything (why should I care what you use). I was only trying to give you perspective into my lack VS abilities so you can add salt to any direction/suggestions I offered up and/or added further...*sigh* Good luck.

[CADbloke]

I took Tony's comments in the same way, I miss him around here. I am personally not offended at all by your posts (for exactly the same reasons you were not offended by Tony) but I think some lurkers may be a little more sensitive and thus more reluctant to post their input lest they are shown up. I guess that's a risk anywhere and probably their problem - I personally came here to be wrong and to be corrected. I don't think I'm being clear about this, I just don't want to scare the newbs into silence.

Thank you for your input so far. I certainly don't wish to convey any ingratitude - far from it, I have learned things here and that's what I came for. I think you and I may probably never share common opinions on Visual Studio and CMake and that's a good thing, forums like this need that dissonance and difference of opinion. If you got the feeling I devalued your opinions and experience then I'm sorry, that was not my intention, far from it.

I, personally, would avoid the Git approach simple because I think big project could be a handful to merge consistently. Code Cloner puts all the code in one place. I think I need to clarify in the workflows in my readme that it's a good idea to commit the cloned code right away so you can tell if you accidentally add something local in the middle of it after the fact. It still needs work (it is only early days), I do need to make it check that anything has actually changed before it rewrites the csproj file because the whole reloading recently saved projects gets tiresome pretty quickly.

I think the best demonstration of Code Cloner is to clone it locally, build it a couple of times until it clones into its own .NET 3.5 project and then see what the next Git commit looks like. You will see the lines it has added to the .NET 3.5 project.

As an aside, linking files with wildcards in Visual Studio breaks a lot because of the different file extensions (*.xaml, *.config, *.resx etc) and also the "DependsOn" Attributes, you need that meta information from the original project, well you need it in Visual Studio, it will probably build just fine without it.

Code: [Select]

<ItemGroup>
  <Compile Include="${DOS_STYLE_SOURCE_DIR}\**\*.cs" />
</ItemGroup>

Linking with *.* can create a bit of a mess and you eventually end up with progressively unmaintainable stuff like ...

Code - XML: [Select]

1. <Compile Include="..\Path\To\ParentFolder\*.*"
2.         Exclude="..\Path\To\ParentFolder\NotThisOne.cs;..\Path\To\ParentFolder\NotThisOneEither.cs">
3.   <Link>ParentFolder\%(RecursiveDir)%(Filename)%(Extension)</Link>
4.  </Compile>
5. <!-- Visual Studio loves to refactor these things so I always add it as a comment too so I can restore it.
6.        Source Code repo check-ins are an easy way to spot this when it happens.
7. <Compile Include="..\Path\To\ParentFolder\*.*"
8.         Exclude="..\Path\To\ParentFolder\NotThisOne.cs;..\Path\To\ParentFolder\NotThisOneEither.cs">
9.  <Link>ParentFolder\%(RecursiveDir)%(Filename)%(Extension)</Link>
10. </Compile>
11. -->
12. <!-- This sort of abomination is also possible, if not advisable -->
13. <Compile Include="..\zSourceCode\**\*.*"       Exclude="..\zSourceCode\Properties\AssemblyInfo.cs;..\zSourceCode\bin\**\*.*;..\zSourceCode\obj\**\*.*;..\zSourceCode\**\*.csproj;..\zSourceCode\**\*.user;..\zSourceCode\**\*.vstemplate;..\zSourceCode\readme.txt;..\zSourceCode\**\*.lsp;..\zSourceCode\**\*.scr;..\zSourceCode\**\*.ico;..\zSourceCode\**\*.txt">
14.  
15. <!-- This has an absolute path with a  variable previously declared in this or another .csproj file -->
16. <Compile Include="$CadExtensionClassesRootFolder$\Common\*.*" >
17.   <Link>Common\%(RecursiveDir)%(Filename)%(Extension)</Link>
18. </Compile>

At the end of the day I am trying to find the best way that works for me, I posted it here and at Github in case it helps someone else along the way, or at least helps to clarify why they like their way by showing an alternative they don't like.

[JohnK]

Too many merges, too often? Now its starting to sound to me like you're building a library (dll) or should be. Failing that you could use a preprocessor to generate code files for you (and update them before you compile). Heck, I have two preprocessors myself (well, only one compiles unless you want to customize the source code of a compiler so you can compile the other one) you could use.

Sounds to me like you are making this far too complicated and you should step back and work on not only your read me but your criteria as well.

[JohnK]

Found this while surfing this morning.

T4 Tutorial: Creating reusable code generation templates
http://www.olegsych.com/2008/09/t4-tutorial-creating-reusable-code-generation-templates/

[Jeff H]

I do this type of stuff everyday using VIM

[JohnK]

I do this type of stuff everyday using VIM

Does Vim have a script to remove my foot from your a55?

[CADbloke]

Sounds to me like you are making this far too complicated and you should step back and work on not only your read me but your criteria as well.

<TLDR>
Um, yeah. I think you got me there. This one is less complicated than my previous efforts but it's still not a "click...next..Yay!" solution. By the time I'm 80 I'll have this nailed. Maybe. The readme definitely needs an edit, it's a midnight braindump.

If all you have in your source project is *.cs files then importing the files with a wildcard into your new project is the easiest solution. Job Done. Ditto if the source project is quite static, nothing much changes. I want a solution that can automatically keep the projects in sync, even if the original source changes radically. I want it to be Visual Studio friendly. I want more coffee.
</TLDR>

Code Cloner is definitely half-baked. Here's the proof (I wrote the following first)...

...you could use a preprocessor to generate code files for you (and update them before you compile)...

That's basically what I want Code Cloner to do, sort of. I want to work with the original files, not proliferate copies all over the place that need to be kept track of and in sync with the original. If I have a bright idea and add it to a target project (because I'm tired and forgot to switch to the original source project) then I want the changes to propagate to all projects - linked files solves that. One version of the Truth. I also want it to work as an easy drop-in on an existing project. You make a call to CodeCloner.exe whenever you want to refresh the target projects. You can check the projects against each other in a text-diff to see what is missing, as a confidence test. As an example, diff the 2x csproj files in the Github project after you build them a couple of times, you will see the added lines in the XML and how they compare to the original.

a use case is ...

• I have a project (or a few, let's stick to one here) in Visual Studio I have written for AutoCAD 2015
• I decide I want to build a version for, say, BrisCAD with the same code (more or less)
• I create a new Solution folder in the solution root called "_Builds". I just pulled that name from the same place John just put his foot. Call it anything you like
• oh, thank you (for nothing!) Visual Studio, now I got to the actual physical folder tree on disk and create the folder in real life. I give it the same name for consistency's sake.
• In this folder I add / copy / whatever  new csproj Project. Hmm, this sounds a little like "magic just happened, you got that right?". ok, you can
  
  • copy the existing csproj and hand-delete all the compile stuff then add it as an existing project
  • or you can create a new project of the same type. You will need to make sure the namespaces etc match.
  • Physically copying the csproj may be easier. Actually, something that automagically clones the guts of the project would be easier
• the new project is the target project. Delete all the code, add the "Clone Zone" placeholders.
• Add a call to CodeCloner.exe as a post-build process. Or pre-build. I made it post-build in the original project because it actually has to build before it can use itself, it was a chicken/egg thing.
• Most of your editing is done in the original source project, you're really only tweaking the target(s) to make them compatible with their target platform.
• ok, I'm bored now, somebody should automate all this crap and get back to me when it works
• time passes...

Well, that was crap. Next!

• I have a project (or a few, let's stick to one here) in Visual Studio I have written for AutoCAD 2016 (time passed, for those of you who noticed)
• I decide I want to build a version for, say, BrisCAD with the same code (more or less)
• I drop CodeCloner.exe in the Solution root (optional, the .exe can live anywhere, that's up to you)
• I call CodeCloner.exe init "_Builds", "_Builds" being the folder name I want to use for the new builds
• New project(s) magically appear under a new Solution Folder. At this stage they are basically clones of the existing project(s).
• In the target project(s) change the AutoCAD References for BricsCAD references and the .NET framework, if required. I will probably want to  add/remove/change some conditional compilation symbols. Or not, whatever is your preference
• Fix what broke
• Ship it. Tests? Meh.

I still think that, for a Visual Studio addict like me, linked source code is still the most usable way of using the same code multiple times. I, personally, want to look through the code and see what is going on.

I have (more) work to do....after the day job ...after the kids go to bed. Sleep? Meh.

...unless you want to customize the source code of a compiler...

Hey, there's an idea...what could possibly go wrong?