Topic: Remove-if ... etal

[kdub_nz]

A sort of challenge ..

(vl-directory-files .... includes  ("." ".." etc if the folder is not a root drive

To remove these strings if existing I tried these

 ... any others ??

Code: [Select]


(defun _test01 ()
  (setq bom:ParentFolder "F:\\WDT-BOM-Data"
        Folders          (vl-remove-if (function (lambda (g) (or (= g ".") (= g ".."))))
                                       (vl-directory-files bom:ParentFolder NIL -1)
                         )
  )
)
(defun _test02 ()
  (setq bom:ParentFolder "F:\\WDT-BOM-Data"
        Folders          (vl-remove ".." (vl-remove "." (vl-directory-files bom:ParentFolder NIL -1)))
  )
 
)
(defun _test03 ()
  (setq bom:ParentFolder "F:\\WDT-BOM-Data"
        Folders          (if (= "." (car (setq Folders (vl-directory-files bom:ParentFolder NIL -1))))
                           (if (= ".." (car (setq Folders (cdr Folders))))
                             (setq Folders (cdr Folders))
                           )
                         )
  )
)

(benchMark '((_test01) (_test02) (_test03)))
Benchmarking [M.P. 2005] .........Elapsed milliseconds for 64 iteration(s)/ relative Timing :

    (_TEST02).....796 / 1.0845 <slowest>
    (_TEST01).....766 / 1.0436
    (_TEST03).....734 / 1 <fastest>

edit: clumsy and stupid

[kdub_nz]

... and does anyone know for definite if there is ever only ONE elipsis returned or always either NONE or TWO ??

[MP]

Sorry, I don't know the answer to your last question and I can't run AutoCAD right now to find out.

Untested but what if you use the member function (member g '("." "..")) in _test01 instead of the or construct? It's fewr function calls than the other variants so it might bench faster.

[kdub_nz]

good thought ..  close to the money

Code: [Select]

(defun _test04 ()
  (setq bom:ParentFolder "F:\\WDT-BOM-Data"
        Folders          (vl-remove-if (function (lambda (g) (member g '("." "..") )))
                                       (vl-directory-files bom:ParentFolder NIL -1)
                         )
  )
)

(benchMark '((_test01) (_test02) (_test03) (_test04)))

Benchmarking [M.P. 2005] .........Elapsed milliseconds for 64 iteration(s)/ relative Timing :

    (_TEST04).....797 / 1.0858 <slowest>
    (_TEST03).....781 / 1.064
    (_TEST02).....765 / 1.0422
    (_TEST01).....734 / 1 <fastest>

[MP]

Great, the worst. So much for that idea <facepalm>.

[kdub_nz]

Great, the worst. So much for that idea <facepalm>.

but it's only 'worse' by a poofteenth   

[MP]

Worse by 8.6%. That ain't poofteenth.

[MP]

If memory serves me you'll always get the "." ".." if the directory location is not the root folder, but I'm assuming vl-directory-files behavior mirrors that of the DOS dir command.

[MP]

Code: [Select]

(defun _test? ()
    (setq
        bom:ParentFolder "F:\\WDT-BOM-Data"
        Folders          (if (member ".." (setq Folders (vl-directory-files bom:ParentFolder NIL -1)))
                             (cddr Folders)
                             Folders
                         )
    )
)
untested but ... if the "." ".." always appear in the noted order (and always first in the list) then the above should work.

[kdub_nz]

Thanks Michael,
I'm doing an animated stepthrough at the moment so I'll try that option as soon as the process is finished.

[MP]

You're welcome KB. I gotta crash here pretty quick so I won't see the results 'til tomorrow, but thanks for the opportunity to pitch a few ideas.

[Kerry]

On the old box at home .....

Code: [Select]


(defun _test01 ()
  (setq bom:ParentFolder "F:\\_Develop"
        Folders          (vl-remove-if (function (lambda (g) (or (= g ".") (= g ".."))))
                                       (vl-directory-files bom:ParentFolder NIL -1)
                         )
  )
)
(defun _test02 ()
  (setq bom:ParentFolder "F:\\_Develop"
        Folders          (vl-remove ".." (vl-remove "." (vl-directory-files bom:ParentFolder NIL -1)))
  )
 
)
(defun _test03 ()
  (setq bom:ParentFolder "F:\\_Develop"
        Folders          (if (= "." (car (setq Folders (vl-directory-files bom:ParentFolder NIL -1))))
                           (if (= ".." (car (setq Folders (cdr Folders))))
                             (setq Folders (cdr Folders))
                           )
                         )
  )
)
(defun _test04 ()
  (setq bom:ParentFolder "F:\\_Develop"
        Folders          (vl-remove-if (function (lambda (g) (member g '("." "..") )))
                                       (vl-directory-files bom:ParentFolder NIL -1)
                         )
  )
)

(defun _test05 ()
    (setq
        bom:ParentFolder "F:\\_Develop"
        Folders          (if (member ".." (setq Folders (vl-directory-files bom:ParentFolder NIL -1)))
                             (cddr Folders)
                             Folders
                         )
    )
)
(defun _test06 ()
    (setq bom:ParentFolder "F:\\_Develop")
    (if (= ".." (cadr (setq Folders (vl-directory-files bom:ParentFolder NIL -1))))
        (setq Folders (cddr Folders))
    )
)


Benchmarking [M.P. 2005] .............Elapsed milliseconds for 1024 iteration(s)/ relative Timing :

    (_TEST04).....500 / 1.0684 <slowest>
    (_TEST01).....484 / 1.0342
    (_TEST02).....469 / 1.0021
    (_TEST03).....469 / 1.0021
    (_TEST05).....469 / 1.0021
    (_TEST06).....468 / 1 <fastest>

[MP]

[ronjonp]

Kerry,

What kind of speed does this have?

Code: [Select]

(defun subfolders (path / folder fs lst subs)
  (setq fs (vlax-create-object "Scripting.FileSystemObject"))
  (if (and (setq folder (vlax-invoke fs 'getfolder path))
           (setq subs (vlax-get folder 'subfolders))
      )
    (vlax-for sub subs (setq lst (cons (vlax-get sub 'Path) lst)))
  )
  (vlax-release-object fs)
  (reverse lst)
)
(subfolders "C:\\")
Nevermind  :ugly:

    (_TEST03)........1500 / 3.47 <fastest>
    (_TEST05)........1515 / 3.43
    (_TEST02)........1516 / 3.43
    (_TEST06)........1516 / 3.43
    (_TEST01)........1578 / 3.30
    (_TEST04)........1578 / 3.30
    (SUBFOLDERS).....5203 / 1.00 <slowest>

[VovKa]

Code: [Select]

(defun _test07 ()
  (setq bom:ParentFolder "C:\\Program Files")
  (if (> (strlen bom:ParentFolder) 3)
    (cddr (vl-directory-files bom:ParentFolder NIL -1))
    (vl-directory-files bom:ParentFolder NIL -1)
  )
)
03 and 06 do not work for the root

[kdub_nz]

Oooops 

Thanks VovKA .. missed that


tried 3 run-throughs ..
Benchmarking [M.P. 2005] ............Elapsed milliseconds for 512 iteration(s)/ relative Timing :

    (_TEST01).....610 / 1.1152 <slowest>
    (_TEST04).....594 / 1.0859
    (_TEST02).....578 / 1.0567
    (_TEST05).....563 / 1.0293
    (_TEST03).....547 / 1
    (_TEST06).....547 / 1
    (_TEST07).....547 / 1 <fastest>

 
Benchmarking [M.P. 2005] ............Elapsed milliseconds for 512 iteration(s)/ relative Timing :

    (_TEST01).....625 / 1.1447 <slowest>
    (_TEST04).....625 / 1.1447
    (_TEST02).....578 / 1.0586
    (_TEST03).....578 / 1.0586
    (_TEST05).....563 / 1.0311
    (_TEST07).....563 / 1.0311
    (_TEST06).....546 / 1 <fastest>

 
Benchmarking [M.P. 2005] ............Elapsed milliseconds for 512 iteration(s)/ relative Timing :

    (_TEST01).....594 / 1.0859 <slowest>
    (_TEST02).....578 / 1.0567
    (_TEST04).....578 / 1.0567
    (_TEST06).....563 / 1.0293
    (_TEST03).....562 / 1.0274
    (_TEST07).....562 / 1.0274
    (_TEST05).....547 / 1 <fastest>



Copy of Michaels Benchmark routine with my mods ..

[kdub_nz]

These are the reworked functions, revised to RETURN the Folder List for ALL functions.
The Benchmark Test can't make up it's mind ( consistently ) which function is faster

Code: [Select]

(defun _test01 ( / Folders)
  (setq Folders (vl-remove-if (function (lambda (g) (or (= g ".") (= g ".."))))
                              (vl-directory-files bom:ParentFolder NIL -1)
                )
  ) 
)

(defun _test02 ( / Folders)
  (setq Folders
         (vl-remove ".." (vl-remove "." (vl-directory-files bom:ParentFolder NIL -1)))
  )
)

(defun _test03 ( / Folders)                                                     
  (if (= "." (car (setq Folders (vl-directory-files bom:ParentFolder NIL -1))))
    (if (= ".." (car (setq Folders (cdr Folders))))
      (setq Folders (cdr Folders))
    )
  )
  Folders
)

(defun _test04 ( / Folders)
  (setq Folders (vl-remove-if (function (lambda (g) (member g '("." ".."))))
                              (vl-directory-files bom:ParentFolder NIL -1)
                )
  )
)

(defun _test05 ( / Folders)
  (setq Folders
         (if (member ".." (setq Folders (vl-directory-files bom:ParentFolder NIL -1)))
           (cddr Folders)
           Folders
         )
  )
)

(defun _test06 ( / Folders)
  (if (= ".." (cadr (setq Folders (vl-directory-files bom:ParentFolder NIL -1))))
    (setq Folders (cddr Folders))
  )
  Folders
)

(defun _test07 ( / Folders) 
  (if (> (strlen bom:ParentFolder) 3)
    (cddr (vl-directory-files bom:ParentFolder NIL -1))
    (vl-directory-files bom:ParentFolder NIL -1)
  )
)

;; (setq bom:ParentFolder "C:\\Program Files")
(setq bom:ParentFolder "C:\\")
(benchMark '((_test01) (_test02) (_test03) (_test04) (_test05) (_test06) (_test07) ))


[MP]

I believe the variability of the results has more to do with the unpredictability of what the system greater is doing when you're performing I/O via the vl-directory-files in each test invocation rather than stability of the benchmark function, tho I'm not inferring it is perfect. Anyway, since that portion of the code (vl-directory-files bom:ParentFolder NIL -1) is common to all functions it would be better to store the result of such to a global variable and then use said variable in each of the functions to be tested. After all, what you're wishing to bench is the performance of the code that deals with the "." .".." entries, not the I/O that retrieves them. To wit:

Code: [Select]

(progn

    (defun _test01 ( / Folders)
      (setq Folders (vl-remove-if (function (lambda (g) (or (= g ".") (= g ".."))))
                                  _folders
                    )
      )
    )

    (defun _test02 ( / Folders)
      (setq Folders
             (vl-remove ".." (vl-remove "." _folders))
      )
    )

    (defun _test03 ( / Folders)
      (if (= "." (car (setq Folders _folders)))
        (if (= ".." (car (setq Folders (cdr Folders))))
          (setq Folders (cdr Folders))
        )
      )
      Folders
    )

    (defun _test04 ( / Folders)
      (setq Folders (vl-remove-if (function (lambda (g) (member g '("." ".."))))
                                  _folders
                    )
      )
    )

    (defun _test05 ( / Folders)
      (setq Folders
             (if (member ".." (setq Folders _folders))
               (cddr Folders)
               Folders
             )
      )
    )

    (defun _test06 ( / Folders)
      (if (= ".." (cadr (setq Folders _folders)))
        (setq Folders (cddr Folders))
      )
      Folders
    )

    (defun _test07 ( / Folders)
      (if (> (strlen bom:ParentFolder) 3)
        (cddr _folders)
        _folders
      )
    )

    ;; (setq bom:ParentFolder "C:\\Program Files")
    (setq bom:ParentFolder "C:\\")
   
    (setq _folders (vl-directory-files bom:ParentFolder NIL -1))

    (repeat 5

        (benchMark
           '(   (_test01)
                (_test02)
                (_test03)
                (_test04)
                (_test05)
                (_test06)
                (_test07)
            )
        )

    )

    (princ)

)
Result (very consistant):

Elapsed milliseconds / relative speed for 65536 iteration(s):

    (_TEST03).....1641 / 1.91 <fastest>
    (_TEST06).....1672 / 1.88
    (_TEST07).....1719 / 1.83
    (_TEST02).....1797 / 1.75
    (_TEST05).....1906 / 1.65
    (_TEST01).....2844 / 1.10
    (_TEST04).....3141 / 1.00 <slowest>

Elapsed milliseconds / relative speed for 65536 iteration(s):

    (_TEST03).....1672 / 1.88 <fastest>
    (_TEST06).....1672 / 1.88
    (_TEST07).....1703 / 1.84
    (_TEST02).....1797 / 1.75
    (_TEST05).....1891 / 1.66
    (_TEST01).....2875 / 1.09
    (_TEST04).....3141 / 1.00 <slowest>

Elapsed milliseconds / relative speed for 65536 iteration(s):

    (_TEST03).....1641 / 1.91 <fastest>
    (_TEST06).....1672 / 1.88
    (_TEST07).....1718 / 1.83
    (_TEST02).....1781 / 1.76
    (_TEST05).....1921 / 1.64
    (_TEST01).....2859 / 1.10
    (_TEST04).....3141 / 1.00 <slowest>

Elapsed milliseconds / relative speed for 65536 iteration(s):

    (_TEST03).....1656 / 1.89 <fastest>
    (_TEST06).....1656 / 1.89
    (_TEST07).....1687 / 1.85
    (_TEST02).....1781 / 1.75
    (_TEST05).....1921 / 1.63
    (_TEST01).....2829 / 1.10
    (_TEST04).....3125 / 1.00 <slowest>

Elapsed milliseconds / relative speed for 65536 iteration(s):

    (_TEST03).....1641 / 1.91 <fastest>
    (_TEST06).....1656 / 1.90
    (_TEST07).....1688 / 1.86
    (_TEST02).....1797 / 1.75
    (_TEST05).....1891 / 1.66
    (_TEST01).....2813 / 1.12
    (_TEST04).....3141 / 1.00 <slowest>

Cheers.

[kdub_nz]

Yes Michael, I came to the same conclusion, and reworked my test code in a similar manner.
.. the discrepancy is further exacerbated by accessing a network folder, albeit on a GB system

[MP]

cool well I'm out-a-here, been a long day :doa: