Author Topic: Same Code, Multiple Projects, Multiple Builds  (Read 26477 times)

0 Members and 1 Guest are viewing this topic.

Jeff H

  • Needs a day job
  • Posts: 6144
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #15 on: October 08, 2015, 01:01:39 AM »
I do this type of stuff everyday using VIM

Does Vim have a script to remove my foot from your a55?
:2funny:

JohnK

  • Administrator
  • Seagull
  • Posts: 10605
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #16 on: October 08, 2015, 09:18:27 AM »
Okay.

Your criteria/explanations are still a bit jumbled up and I'm still having a hard time envisioning this but I'm getting a better idea of your plan (I think). -i.e. sometimes you are describing a library, and sometimes you're describing a file sync thing, and other times you're describing a processor of sorts.

I think a library (a set of DLLs you can use in your projects) is the best solution:
Pros:
1. No code to copy around.
2. Conforms to the whole .Net concept.
3. Easy to use.
4. Updates are automatic.
5. Quick turn around (fast to get up and running).
Cons:
1. Not sure how VS handles distributions (how to get all referenced DLLs in a package for distribution).

However, if you're dead-set on *not* building and maintaining a library I have some contentions with the processor/file sync thing concepts we'd need to work out later. Namely:
1. What sync method(s) are you planing on, and how are you going to determine which file is outdated?
2. What protocols are looking to sync over (local drive, LAN, WAN)?
3. Are you planning on using compression to keep things small and fast?

But for now we probably start formulating a plan of attack for this thing you have envisioned.

I took a quick look at your github code and I don't really see any "processor" code at all. I mean, I didn't see any sort of tokenizer, lexer, or a sold parser really (I see a bunch of code that looks more procedural in nature than anything). ...I think you should scrap this and start over. Your thing will have to be able to operate on at least a few different file types/languages source code files (C# language), IDE project files (XML).

Assuming the simpler/easier to write/maintain recursive-decent type parser, here's how I think you should start:
1. Build a in/out buffer to handle the file/console read/writes.
2. Pick one of the file types to work on first (C# or XML).
3. Develop a good set of tokens and a rock solid lexer base for handling some parts of the file type.
4. Start building an abstract syntax tree for what you have so far.
5. Start building your base-level and your top-level parsers.
6. Build a main driver.
7. Start building tests.
8. Repeat steps 3-5 and 7 to add to your parser parts until you are able to parse one/all of the file types.

Obviously, you need to work on your criteria and this initial plan will change but this should at least get us speaking the same `language'. I also think XML will kill the recursive-decent type processor plan but we need to start somewhere and I think if you only offer limited abilities in your parser thing you'll be alright.

Sounds to me like you are making this far too complicated and you should step back and work on not only your read me but your criteria as well.
<TLDR>
Um, yeah. I think you got me there. This one is less complicated than my previous efforts but it's still not a "click...next..Yay!" solution. By the time I'm 80 I'll have this nailed. Maybe. The readme definitely needs an edit, it's a midnight braindump.

If all you have in your source project is *.cs files then importing the files with a wildcard into your new project is the easiest solution. Job Done. Ditto if the source project is quite static, nothing much changes. I want a solution that can automatically keep the projects in sync, even if the original source changes radically. I want it to be Visual Studio friendly. I want more coffee.
</TLDR>

Code Cloner is definitely half-baked. Here's the proof (I wrote the following first)...

...you could use a preprocessor to generate code files for you (and update them before you compile)...

That's basically what I want Code Cloner to do, sort of. I want to work with the original files, not proliferate copies all over the place that need to be kept track of and in sync with the original. If I have a bright idea and add it to a target project (because I'm tired and forgot to switch to the original source project) then I want the changes to propagate to all projects - linked files solves that. One version of the Truth. I also want it to work as an easy drop-in on an existing project. You make a call to CodeCloner.exe whenever you want to refresh the target projects. You can check the projects against each other in a text-diff to see what is missing, as a confidence test. As an example, diff the 2x csproj files in the Github project after you build them a couple of times, you will see the added lines in the XML and how they compare to the original.

a use case is ...
  • I have a project (or a few, let's stick to one here) in Visual Studio I have written for AutoCAD 2015
  • I decide I want to build a version for, say, BrisCAD with the same code (more or less)
  • I create a new Solution folder in the solution root called "_Builds". I just pulled that name from the same place John just put his foot. Call it anything you like
  • oh, thank you (for nothing!) Visual Studio, now I got to the actual physical folder tree on disk and create the folder in real life. I give it the same name for consistency's sake.
  • In this folder I add / copy / whatever  new csproj Project. Hmm, this sounds a little like "magic just happened, you got that right?". ok, you can
    • copy the existing csproj and hand-delete all the compile stuff then add it as an existing project
    • or you can create a new project of the same type. You will need to make sure the namespaces etc match.
    • Physically copying the csproj may be easier. Actually, something that automagically clones the guts of the project would be easier
  • the new project is the target project. Delete all the code, add the "Clone Zone" placeholders.
  • Add a call to CodeCloner.exe as a post-build process. Or pre-build. I made it post-build in the original project because it actually has to build before it can use itself, it was a chicken/egg thing.
  • Most of your editing is done in the original source project, you're really only tweaking the target(s) to make them compatible with their target platform.
  • ok, I'm bored now, somebody should automate all this crap and get back to me when it works
  • time passes...
Well, that was crap. Next!
  • I have a project (or a few, let's stick to one here) in Visual Studio I have written for AutoCAD 2016 (time passed, for those of you who noticed)
  • I decide I want to build a version for, say, BrisCAD with the same code (more or less)
  • I drop CodeCloner.exe in the Solution root (optional, the .exe can live anywhere, that's up to you)
  • I call CodeCloner.exe init "_Builds", "_Builds" being the folder name I want to use for the new builds
  • New project(s) magically appear under a new Solution Folder. At this stage they are basically clones of the existing project(s).
  • In the target project(s) change the AutoCAD References for BricsCAD references and the .NET framework, if required. I will probably want to  add/remove/change some conditional compilation symbols. Or not, whatever is your preference
  • Fix what broke
  • Ship it. Tests? Meh.

I still think that, for a Visual Studio addict like me, linked source code is still the most usable way of using the same code multiple times. I, personally, want to look through the code and see what is going on.

I have (more) work to do....after the day job ...after the kids go to bed. Sleep? Meh.

...unless you want to customize the source code of a compiler...
Hey, there's an idea...what could possibly go wrong?
TheSwamp.org (serving the CAD community since 2003)
Member location map - Add yourself

Donate to TheSwamp.org

CADbloke

  • Bull Frog
  • Posts: 342
  • Crash Test Dummy
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #17 on: October 08, 2015, 07:05:22 PM »
Okay.

Your criteria/explanations are still a bit jumbled up and I'm still having a hard time envisioning this but I'm getting a better idea of your plan (I think). -i.e. sometimes you are describing a library, and sometimes you're describing a file sync thing, and other times you're describing a processor of sorts.
Yeah, the state of my head this week is not good.

Code Cloner...wait, I think it needs a better, more applicable name like "Code Linker"
I don't think I have explicitly said this but ... This will only work in Visual Studio. It is not a build process, it is a way to share the same code between projects.

All Code Cloner does is adds links to existing code files into a target csproj. No file is copied, no source code is parsed. The original csproj file is mined for its source code <Include=...> things and that information is used to add links to the target csproj file. That's all. It ends there. The  target csproj is all it touches. It automates the add existing file as a link process. That link explains it really well, and also some of the reasons you would do this.
I want to work with the original files, not proliferate copies all over the place that need to be kept track of and in sync with the original. If I have a bright idea and add it to a target project (because I'm tired and forgot to switch to the original source project) then I want the changes to propagate to all projects - linked files solves that. One version of the Truth.

Libraries are great. This has nothing to do with libraries, it is all about re-using existing code in a new Visual Studio Project - building the same thing a different way.

Try it, it already works. I just pushed a fix to Github that fixes the issue of refreshing the target project when nothing changed - it leaves it alone now. Changing code inside linked files does not affect the target project, a link only changes if you add / delete / move a file.

You make a call to CodeCloner.exe whenever you want to refresh the target projects. You can check the projects against each other in a text-diff to see what is missing, as a confidence test. As an example, diff the 2x csproj files in the Github project after you build them a couple of times, you will see the added lines in the XML and how they compare to the original.

Here is an illustrated process. I cloned the Github source and opened it in Visual Studio

Note the post-build process (Project - Properties). In this particular instance I put it there because it actually needed to be built for the demo to work. It's also not the standard command line call because I used a macro variable to call the actual EXE. You would normally call CodeCloner ..\..\..\_Builds\CodeCloner-3.5\CodeCloner-3.5.csproj. Putting it in a pre or post build process means it happens automatically, you don't have to think about it.


Note the emptiness of the .NET 3.5 project. Now, right-click the CodeCloner Project in Solution Explorer and click "Build"


Visual Studio will (or not, this may depend on your settings) see the 3.5 project has been tampered with and ask to reload it

Reload it.

Job done. Note the new linked files in the 3.5 project

You can now build the 3.5 project and it will build a .NET 3.5 version of CodeCloner (the original is .NET 4.5.2). You're finished. No, really. Done. That's it.

There are no new files in the 3.5 project. Open the directory on disk and have a look. Have a look at the changes for the next Git commit in the Solution, the only changes are the linked files in _Builds\CodeCloner-3.5\CodeCloner-3.5.csproj



That's all it does. All you need to use it is the EXE. All this Github repo does is builds the exe. I used it as an example of itself which is probably the cause of the confusion here.

JohnK

  • Administrator
  • Seagull
  • Posts: 10605
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #18 on: October 09, 2015, 08:55:15 AM »
I'm glad you got your project rolling on the right track.

Next time share a link like the MSDN one early in the conversation (that link would have saved *hours* of my time)! I'm sorry but, as far as I'm concerned, this post deserves the LHTAQTSW tag.
TheSwamp.org (serving the CAD community since 2003)
Member location map - Add yourself

Donate to TheSwamp.org

CADbloke

  • Bull Frog
  • Posts: 342
  • Crash Test Dummy
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #19 on: October 11, 2015, 05:15:57 PM »
...this post deserves the LHTAQTSW tag.
Agreed. Poor effort on my part. Sorry I wasted your time.

CADbloke

  • Bull Frog
  • Posts: 342
  • Crash Test Dummy
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #20 on: October 13, 2015, 05:55:20 PM »
Updated the first post (again), added an Illustrated example and updated the extensive readme, both at Github.

Let me know if something (everything?) doesn't make sense.

Apologies again (especially to John K) for not making sense previously, it has been that kind of month.

I hope you find this useful. I would be thrilled to hear suggestions or alternatives we haven't yet explored.

JohnK

  • Administrator
  • Seagull
  • Posts: 10605
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #21 on: October 14, 2015, 08:04:57 AM »
I would add a release to the downloads section of your github project (include help documents).
https://github.com/CADbloke/CodeCloner/releases

I would also remove the "informal" feel to the readme.md writeup. If I stumble on your project, I want to know quickly:
1. What it does.
2. How to get it.
3. Why it does it.
4. What doesn't work about it.
5. How can I contribute.

Another thing you might want to do is to take a look at how other software documents their program. Unix utilities are good examples because they are all individual programs but they all have a common format to their readme's.
Here is the man page (help) for GREP (a basic tool)
http://linuxcommand.org/man_pages/grep1.html

I would also build an installer for people to use. This installer could add the location of CodeCloner to the env path so it can be called via the command line.

Lastly, I hate to suggest this but I would think about changing the name. CodeCloner sounds a bit misleading.
« Last Edit: October 14, 2015, 08:09:33 AM by John Kaul (Se7en) »
TheSwamp.org (serving the CAD community since 2003)
Member location map - Add yourself

Donate to TheSwamp.org

CADbloke

  • Bull Frog
  • Posts: 342
  • Crash Test Dummy
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #22 on: October 14, 2015, 05:53:06 PM »
Thanks for the feedback John, All your suggestions are good ones. I'll go through them when I get some time. Most of that readme should be separate wiki pages.

I agree with you on the naming - it is misleading. I named it before I wrote it. It should mention reuse ... recycle ... something like that. Clone infers that something is copied and it's not.

This thing needs a complete reboot. Well, there goes the weekend.

CADbloke

  • Bull Frog
  • Posts: 342
  • Crash Test Dummy
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #23 on: October 21, 2015, 08:37:27 AM »
Still working on it.

Here's a teaser ...

Everything is draggy-droppy so you can just drag Files, folders etc from Windows Exploder into it. The GUI version recycles the original project(s) and creates the destination project(s) with the right placeholder, source designation(s) and links to the original source code. You can type stuff into the GUI as well, delete rows, mix up tour source folders. It's just a garden-variety editable DataGridView.

Then you add it to a Visual Studio Solution and edit it. Initially it will build exactly the same thing as the source project.

Yes, it also works with VB.NET. At least it's s supposed to.

The command line version now also clones as well as stripping the relevant content. You use it in a batch file to keep things up to date. The GUI is to get you started.

Code: [Select]
Usages...
CODERECYCLER /?
CODERECYCLER [Folder [/s]]
CODERECYCLER [[Source.csproj] Destination.csproj]
CODERECYCLER clone [[Source.csproj] Destination Folder]
CODERECYCLER clone [[Source Folder] Destination Folder] [/s]
CODERECYCLER strip [Destination Folder [/s]]
CODERECYCLER strip [Destination.csproj]
CODERECYCLER ... /noconfirm

/?           This help text.
/noconfirm   Switch. Don't ask about overwrites etc. Use in batch jobs.

Folder   Recycles the source(s) into all CSPROJ files in the folder
         This is the destination folder that has recycled projects.
         The Destination projects need to have the source in their placeholder.
/s       Also iterates all subfolders. You just forgot this, right?
         use . for the current folder, add /s for all subfolders.

Source.csproj        optional Path to the CSPROJ with the source to be recycled.
                     if only 1 CSPROJ is specified then it is the Destination.
                     This source overrides all sources in the Destination.
Destination.csproj   Path to the existing Destination project.

clone    Clones the Source to the Destination path and strips the contents.
         Creates new project file(s) with a placeholder + Source.
         Using folders it clones all Projects to the same destination Folder.

Source Solution Root  (optional)
         The root of the solution containing the projects to recycle.
         Default is the current directory.
Destination Solution Folder
         The new Folder Name for the recycled project(s).
         If only one Folder is specifed then it is the destination folder.

strip    Creates new recycled projects from ALL existing CSPROJ in the folder.
         /s iterates all subfolders . Strips out ALL code. Adds a placeholder.
         usage: copy an existing CSPROJ. Strip it. Add a Source.
         This is like clone but doesn't copy anything.
                Update References, build settings etc. Build it. Rejoice.

It's not on Github yet, I'll start another thread and try to make more sense when it's ready.

Here's what a couple of recycled projects look like in Visual Studio.
Note the tiny arrows on the file icons - they are linked to the original.
Also note Help.cs has pending changes which show on all of its appearances.


The GUI placeholder has a couple of exclusions that you'll see in the Solution Explorer if you squint (they don't have the link icon, they are local files)...
Code - XML: [Select]
  1. <!-- CodeRecycler
  2. Source: ..\CodeRecycler\CodeRecycler.csproj
  3. Exclude: app.config
  4. Exclude: *\assemblyinfo.cs
  5. Exclude: Program.cs
  6. Exclude: YesOrNo.cs
  7. -->

Coming Soon.
If you're interested.
There's a lot of lookers but only John has said anything.
Is this actually useful for anyone (apart from me)?

JohnK

  • Administrator
  • Seagull
  • Posts: 10605
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #24 on: October 21, 2015, 09:01:09 AM »
I think it's a good project; keep going.

Are you using Git to it's fullest extent? -i.e. using branches and whatnot. The reason I ask is because of your `it's not on github yet'. My initial thought at that moment was that it should be in some sort of branch or another.
TheSwamp.org (serving the CAD community since 2003)
Member location map - Add yourself

Donate to TheSwamp.org

CADbloke

  • Bull Frog
  • Posts: 342
  • Crash Test Dummy
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #25 on: October 21, 2015, 09:12:56 AM »
It's in a private repo at kilnhg.com

That's about 75 commits ahead if the GitHub repo but I'll scrap that & start clean.

The public repo is a condensed history. Yes, separate branch.

JohnK

  • Administrator
  • Seagull
  • Posts: 10605
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #26 on: October 21, 2015, 09:59:22 AM »
*facepalm* ...okay, I'll try to just keep my mouth shut (*COUGH: GET ORGANIZED!!!!*).

A Fog creek user? I was kind of disappointed when I heard about them and I found out they were paid only. Do you like it?
TheSwamp.org (serving the CAD community since 2003)
Member location map - Add yourself

Donate to TheSwamp.org

CADbloke

  • Bull Frog
  • Posts: 342
  • Crash Test Dummy
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #27 on: October 21, 2015, 02:59:26 PM »
*facepalm* ...okay, I'll try to just keep my mouth shut (*COUGH: GET ORGANIZED!!!!*).

A Fog creek user? I was kind of disappointed when I heard about them and I found out they were paid only. Do you like it?

No, please, keep up the conversation- you have been a great help. You got me organised!

I don't use the Fogbugz part of the deal much these days, mostly because Trello is so good but it has definitely evolved since I started using it.. Btw it's free for 1-2 users. Follow the link in my post at http://www.theswamp.org/index.php?topic=32381.msg486114#msg486114 and find the startups link on their pricing page. That post may also have other info you may find interesting.

I still prefer Kiln, mostly because you can organise things into projects, something the I've only seen in Gitlab and, recently, Bitbucket. Both of them are free for private repos.

I actually use Kiln & Bitbucket, I push to both so I have a mirror. Gitlab looks good, I haven't used it much and it's not established enough to be my primary, yet.

Try Smartgit, if I can convince you into trying a GUI.

RIghtyo, off to work.

JohnK

  • Administrator
  • Seagull
  • Posts: 10605
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #28 on: October 21, 2015, 03:28:58 PM »
Nah, I don't need another code host. Nor do I need to slow myself down by using a GUI. I evaluated smartgit when I was recommending a GUI for user who was looking to get into using VC (nice but git was intended to be used from the command line and any GUI seems like a kludge--besides, you know Git has a built in GUI right?).

My primary hosts are Bitbucket and Visual Studio Online, I have a github account but I really don't like github that much. Lately my primary has been VSO because I can create Backlogs, Stories, and all that agile stuff right there and even have VSO build my project for me. Very, very nice.

Are you using you .git/config file for pushing/pulling branches to each host or are you doing things some weird way you invented on the fly?
TheSwamp.org (serving the CAD community since 2003)
Member location map - Add yourself

Donate to TheSwamp.org

CADbloke

  • Bull Frog
  • Posts: 342
  • Crash Test Dummy
Re: Same Code, Multiple Projects, Multiple Builds
« Reply #29 on: October 21, 2015, 06:08:56 PM »
Quote from: John Kaul (Se7en) on Today at 05:28:58 am
Quote
Nah, I don't need another code host. Nor do I need to slow myself down by using a GUI. I evaluated smartgit when I was recommending a GUI for user who was looking to get into using VC (nice but git was intended to be used from the command line and any GUI seems like a kludge--besides, you know Git has a built in GUI right?).

The thing I most like about the GUI (SmartGit, TortoiseHg & SourceTree) is the way they manage a list of repos and can show you the status of all of them at once, or at least easily find the one you're looking for. I work on a few different PCs that I need to keep in sync but I'm generally the only one editing the code. I do most of my commits in Visual Studio's git palette. Visual Studio's diff is nice and convenient too. I also like Smartgit's diff. The GUIs that come with Git are a bit fugly, no improvement on the command line and they don't do Hg which I still use a lot.

Quote from: John Kaul (Se7en) on Today at 05:28:58 am
Quote
Lately my primary has been VSO because I can create Backlogs, Stories, and all that agile stuff right there and even have VSO build my project for me. Very, very nice.
Yes, VSO has come a long way - I haven't used it much because I work alone so I can get away with my dodgy Trello workflow. It would be my 1st choice if I had the reasons you just mentioned. When I regain momentum with my bigger projects I will reconsider moving them to VSO.

Quote from: John Kaul (Se7en) on Today at 05:28:58 am
Quote
Are you using you .git/config file for pushing/pulling branches to each host or are you doing things some weird way you invented on the fly?
The latter, I'm not that clever with Git - yet. I'm pleading "too old and too busy with small children" on that one. I keep a separate folder on disk for the public repo and sync it with my working copy using Free File Sync, then push it to Github with a saner history - my private commit message list reads like a Tourette's transcript. I use GitHub flow branching on big projects but on this one it's just one branch.

I also use Free File Sync to manage drawings between local working copies and client networks while keeping old versions in a sane, discoverable manner. See http://stackoverflow.com/a/33138827/492 for how I do that.
</offtopic>



< .. >  I keep a separate folder on disk for the public repo and sync it with my working copy using Free File Sync, then push it to Github with a saner history 
< .. >


That looks like an interesting App; thanks Ewen.


added;
Oh bollocks .. Sorry CADbloke, I stuffed up. I modified your post instead of quoting it.  :oops:
John, is there a way to restore Ewens post ??


added : revisionist history ... the names have been changed to implicate the guilty
« Last Edit: October 21, 2015, 08:43:27 PM by Kerry »