Author Topic: [challenge]: Compare File Modification Dates and Times  (Read 845 times)

0 Members and 1 Guest are viewing this topic.

cmwade77

  • Swamp Rat
  • Posts: 1361
[challenge]: Compare File Modification Dates and Times
« on: March 30, 2022, 01:11:24 PM »
Challenge is to compare File_1 to File_2 and return if File_1 is newer than File_2

I have looked around and found some routines that only compared the date, but not the time and others that only worked some of the time.

Here is my entry that appears to work all of the time:
Code: [Select]
(defun IsNewer (File_1 File_2 / FileSystem Date_1 Date_2 Results Obj_File_1 Obj_File_2) ;Returns T if File_1 is newer than File_2
  (setq File_1 (findfile File_1)
        File_2 (findfile File_2)
  )
  (if (and File_1 File_2)
    (progn
      (setq FileSystem (vlax-get-or-create-object "scripting.filesystemobject"))
      (if FileSystem
        (progn
          (setq Obj_File_1 (vlax-invoke FileSystem 'getfile File_1)
                Obj_File_2 (vlax-invoke FileSystem 'getfile File_2)
          )
          (if (and Obj_File_1 Obj_File_2)
            (progn
              (setq Date_1 (vlax-get Obj_File_1 'datelastmodified)
                    Date_2 (vlax-get Obj_File_2 'datelastmodified)
              )
              (if (and Date_1 Date_2)
                (progn
                  (if (> Date_1 Date_2)
                    (setq Results T)
                  )
                )
              )
            )
          )
          (vlax-release-object FileSystem)
        )
      )
    )
  )
  Results
)

ronjonp

  • Needs a day job
  • Posts: 7437
Re: [challenge]: Compare File Modification Dates and Times
« Reply #1 on: March 30, 2022, 01:23:04 PM »
What would you use this for? If it's updating files, then I'd defer to XCOPY or ROBOCOPY to do the heavy lifting.

Windows 11 x64 - AutoCAD /C3D 2022

Custom Build PC

cmwade77

  • Swamp Rat
  • Posts: 1361
Re: [challenge]: Compare File Modification Dates and Times
« Reply #2 on: March 30, 2022, 01:44:42 PM »
What would you use this for? If it's updating files, then I'd defer to XCOPY or ROBOCOPY to do the heavy lifting.

XCOPY or ROBOCOPY actually aren't robust enough, we are syncing files to and from a server in Tokyo, creating essentially local caches and with a total of over 50 users, we can be talking about thousands of files a day and some can be quite large, some non-AutoCAD files can reach into the 3-4 GB range syncing over 10 Gbps Up/Down connections on each end and yes, I mean 10 gigabits per second, as we are using AWS servers for everything and that is the connection they provide for our file servers, even our workstations are on EC2 instances with at least 1 Gbps, given the speeds we had just tried mapping the same dive to both ends, but whichever is furthest away is very slow due to latency, so hence the cache/sync approach. So, we use Resilio Sync, which is the fastest and most reliable method we have found so far and works well most of the time. We already have methods in place to check if the file is currently open on the remote server and force opening in read only if it is. But, occasionally given the number and size of the files, there are sometimes some slight delays in the sync process.

This can lead to issues where people on both ends may open the file and modify it on both ends before it syncs, which then causes issues, as Resilio Sync can't readily resolve the issue. Ultimately, this is nothing more than an extra failsafe option to ensure that the files are in sync and only allow it to be opened in a read only mode if they aren't.

Alternatively, if you know of a better solution, we are always reevaluating how we have things setup.

ronjonp

  • Needs a day job
  • Posts: 7437
Re: [challenge]: Compare File Modification Dates and Times
« Reply #3 on: March 30, 2022, 02:11:43 PM »
That's a lot to keep track of! I don't envy you.  :evil:

Windows 11 x64 - AutoCAD /C3D 2022

Custom Build PC

JohnK

  • Administrator
  • Seagull
  • Posts: 10140
Re: [challenge]: Compare File Modification Dates and Times
« Reply #4 on: March 30, 2022, 02:40:18 PM »
This isn't a "challenge". I understand your need for a good/fast/better/etc. solution but please use a 'regular thread' for that.

As to the alternate solutions to your problems. Resillo--unison, syncthing, etc--are meant for "desktop type" situations. You may as well be using Dropbox. You need something like:
https://github.com/LINBIT/csync2
Which is meant for scaled up situations (if you need to keep local fileservers at each location; otherwise checkout a azure server).

You may also want to try something like: https://www.nasuni.com/
TheSwamp.org (serving the CAD community since 2003)

Donate to TheSwamp.org

cmwade77

  • Swamp Rat
  • Posts: 1361
Re: [challenge]: Compare File Modification Dates and Times
« Reply #5 on: March 30, 2022, 05:50:04 PM »
This isn't a "challenge". I understand your need for a good/fast/better/etc. solution but please use a 'regular thread' for that.

As to the alternate solutions to your problems. Resillo--unison, syncthing, etc--are meant for "desktop type" situations. You may as well be using Dropbox. You need something like:
https://github.com/LINBIT/csync2
Which is meant for scaled up situations (if you need to keep local fileservers at each location; otherwise checkout a azure server).

You may also want to try something like: https://www.nasuni.com/
My original post was indeed meant to be a challenge, as I was truly only interested to see others ideas on how to compare the last modification date. Ronjon asked why I needed it, so I explained it, but my apologies, as my intention was indeed just a challenge to see if others had a better of different way to compare the dates. Again, my apologies for allowing it to go off on a tangent.

We havde tried the dropbox approach and honestly, it is even worse at needing a modification stamp check and is very slow when there are 20+ xrefs in a single drawing (and yes we get to deal with that joy).

Also, p0lease feel free to move this thread if you think there is a more appropriate place since ir did go off on a tangent.
« Last Edit: March 30, 2022, 05:59:24 PM by cmwade77 »

JohnK

  • Administrator
  • Seagull
  • Posts: 10140
Re: [challenge]: Compare File Modification Dates and Times
« Reply #6 on: March 30, 2022, 07:06:16 PM »
I understand. I just don't want this to turn into a flaming war because someone thinks this is a "fishing expedition".

Dropbox, etc are far below your needs if we're talking 10's of gb a second, but with 'Resillo' and "Syncthing" they need to be set up on each workstation and/or multiple hosts at different locations to be effective/fast but that much data per second will be stretching their needs. The direct sync'ing between two hosts(servers) for a peer-to-peer tool is "counter-productive". If you want a direct copy/sync between two servers(clients) you need something like the Cync2 which is designed to handle 100'sgb a second but that tool isn't designed for--or will compile on--simple/stupid Windows based file servers. Nassuni is okay but you'll probably see little speed improvements with just two servers/hosts/etc..
TheSwamp.org (serving the CAD community since 2003)

Donate to TheSwamp.org

cmwade77

  • Swamp Rat
  • Posts: 1361
Re: [challenge]: Compare File Modification Dates and Times
« Reply #7 on: March 30, 2022, 07:42:42 PM »
I understand. I just don't want this to turn into a flaming war because someone thinks this is a "fishing expedition".

Dropbox, etc are far below your needs if we're talking 10's of gb a second, but with 'Resillo' and "Syncthing" they need to be set up on each workstation and/or multiple hosts at different locations to be effective/fast but that much data per second will be stretching their needs. The direct sync'ing between two hosts(servers) for a peer-to-peer tool is "counter-productive". If you want a direct copy/sync between two servers(clients) you need something like the Cync2 which is designed to handle 100'sgb a second but that tool isn't designed for--or will compile on--simple/stupid Windows based file servers. Nassuni is okay but you'll probably see little speed improvements with just two servers/hosts/etc..
Oh, I understand and I don't want a flaming war either, I really enjoy seeing others approaches to the problems and this is definitely now a tangent that I hadn't intended, but is proving useful too, which is why I suggested moving the thread, so it doesn't turn into a flame war and maybe you can suggest the original challenge again without it going on a tangent, as there are other related items to file system that I may want to work with down the road, but want ideas of different approaches, as mine may not be the best.

As for syncing, here is what we presently do:
  • Oregon File Server does a two way sync with Tokyo File Server using Resilio Sync (I guess I wasn't very clear about which offering we are using from Resilio, we are currently using Resilio Sync for Business with the Servers Syncronization added on, that does make a difference in speed of the sync, but looking, it looks like they have a new, even faster option)
    NOTE: Content required to remain only in the U.S. is excluded from the sync
  • In the U.S., users connect via RDP for their workstations to EC2 instances located in Oregon that map a network drive from the Oregon file server
  • In the Philippines, users connect via RDP for their workstations to EC2 instances that map a network drive from the Tokyo file server

This setup has had its challenges, such as making sure if a file is open on say the server in Tokyo that it can only be opened as read only in the U.S., which I did find a way to accomplish. Additionally, there is the issue of what to do if the files are not presently in sync, which is where the original premise of this thread was, by comparing file modification dates, I can determine if they are in sync or not and I imagine there are multiple ways to compare the modification dates.

And yes, if you have tried using AutoCAD via RDP before, you know I am massively over simplifying the work that has gone into making this setup a viable choice for being a daily driver. But the fact that we were using this basic setup before COVID happened helped us to keep running mostly as if nothing happened, people just had to take their computers home and work from there instead. There have been many other benefits as well, I just get the task of being the BIM, CAD and IT Manager keeping all of this running from a behind the scenes standpoint.

We did try the csync2 at one point, but unfortunately Revit doesn't always play nicely with Linux file servers as we discovered, but that may be a long term solution, as we are in the process of moving all of our Revit projects to BIM360 for a variety of reasons.

Lee Mac

  • Seagull
  • Posts: 12696
  • London, England
Re: [challenge]: Compare File Modification Dates and Times
« Reply #8 on: April 24, 2022, 03:41:44 PM »
In the interest of participating in the 'challenge', I posted the following example some time ago, which leverages the standard vl-file-systime function:

https://forums.autodesk.com/t5/visual-lisp-autolisp-and-general/need-to-compare-quot-lsp-quot-last-modified-date/m-p/4638975#M317151

However, data returned by the FSO is likely to be more reliable.

cmwade77

  • Swamp Rat
  • Posts: 1361
Re: [challenge]: Compare File Modification Dates and Times
« Reply #9 on: April 26, 2022, 12:12:52 PM »
In the interest of participating in the 'challenge', I posted the following example some time ago, which leverages the standard vl-file-systime function:

https://forums.autodesk.com/t5/visual-lisp-autolisp-and-general/need-to-compare-quot-lsp-quot-last-modified-date/m-p/4638975#M317151

However, data returned by the FSO is likely to be more reliable.
Lee, what I discovered was the code you had posted only compared the date, I also needed to compare the time, down to the second as well. I am not sure if that is an issue in newer versions of AutoCAD, this was tested with 2022 and 2023 to discover that or if it was something else.

But thank you for trying to get this back on track, I really do want to see how different people come up with different ways to compare the file modification dates and times, regardless of the reason why I want it.