Author Topic: Remove Duplicates from list?  (Read 17364 times)

0 Members and 1 Guest are viewing this topic.

ASMI

  • Guest
Re: Remove Duplicates from list?
« Reply #15 on: September 28, 2007, 04:26:09 PM »
Another test with 100000 iterations and list length = 30.

Code: [Select]
RemoveDuplicates-mp1       (min:sec.msek): 0:06.98 <fastest>
remove_doubles (gile)         (min:sec.msek): 0:08.28
remove_dup-se7en3           (min:sec.msek): 0:13.38
Remove_Dublicates2-ASMI   (min:sec.msek): 0:13.41
remove_dup-se7en             (min:sec.msek): 0:13.62
remove-duplicates-tony       (min:sec.msek) 0:17.42
RemoveDuplicates-mp2        (min:sec.msek) 0:18.06
remove-dup-se7en2            (min:sec.msek) 20:19.14
Remove_Dublicates-ASMI     (min:sec.msek)  0:21.44 <slowlest>


Binky

  • Guest
Re: Remove Duplicates from list?
« Reply #16 on: September 28, 2007, 05:19:44 PM »
Lamba hurts my brain and anything that starts with VL is still ahead of me.  So here is a non-gifted monkey behind a typewriter approach.. (may not be even close to fast, but it does work)

Code: [Select]
(defun remove_duplicate_list-Binky (xlist / ylist)

  (foreach x xlist
   (foreach y ylist
    (if (= x y)
      (setq fg 1)))
   (if (/= fg 1)
     (setq ylist (cons x ylist)))
   (setq fg 0))
  ylist)

Here are the results
Code: [Select]
Elapsed milliseconds / relative speed for 16384 iteration(s):

    (REMOVEDUPLICATES-MP1 MYLST)..................1437 / 9.25 <fastest>
    (REMOVEDUPLICATES-MP2 MYLST)..................2391 / 5.56
    (REMOVE_DUP-SE7EN MYLST)......................2812 / 4.73
    (REMOVE_DUP-SE7EN3 MYLST).....................2875 / 4.63
    (REMOVE-DUPLICATES-TONY MYLST)................3547 / 3.75
    (APPLY (QUOTE APPEND) (REMOVE-DUP-SE...)......5703 / 2.33
    (REMOVE_DUBLICATES-ASMI MYLST)................6454 / 2.06
    (REMOVE_DUPLICATE_LIST-BINKY MYLST)..........13297 / 1.00 <slowest>
Elapsed milliseconds / relative speed for 16384 iteration(s):

    (REMOVEDUPLICATES-MP1 MYLST)..................1594 / 8.71 <fastest>
    (REMOVEDUPLICATES-MP2 MYLST)..................2640 / 5.26
    (REMOVE_DUP-SE7EN MYLST)......................2984 / 4.66
    (REMOVE_DUP-SE7EN3 MYLST).....................3047 / 4.56
    (REMOVE-DUPLICATES-TONY MYLST)................3594 / 3.87
    (APPLY (QUOTE APPEND) (REMOVE-DUP-SE...)......5641 / 2.46
    (REMOVE_DUBLICATES-ASMI MYLST)................6765 / 2.05
    (REMOVE_DUPLICATE_LIST-BINKY MYLST)..........13891 / 1.00 <slowest>
Elapsed milliseconds / relative speed for 16384 iteration(s):

    (REMOVEDUPLICATES-MP1 MYLST)..................1578 / 21.11 <fastest>
    (REMOVEDUPLICATES-MP2 MYLST)..................2531 / 13.16
    (REMOVE_DUP-SE7EN MYLST)......................3063 / 10.88
    (REMOVE_DUP-SE7EN3 MYLST).....................3156 / 10.56
    (REMOVE-DUPLICATES-TONY MYLST)................3719 / 8.96
    (APPLY (QUOTE APPEND) (REMOVE-DUP-SE...)......5641 / 5.91
    (REMOVE_DUBLICATES-ASMI MYLST)................6750 / 4.94
    (REMOVE_DUPLICATE_LIST-BINKY MYLST)..........33312 / 1.00 <slowest>

so here I find some motivation to learn more. 

MP

  • Seagull
  • Posts: 17750
  • Have thousands of dwgs to process? Contact me.
Re: Remove Duplicates from list?
« Reply #17 on: September 28, 2007, 05:30:20 PM »
... non-gifted monkey behind a typewriter ...

I'm sorry, that position is taken.
Engineering Technologist • CAD Automation Practitioner
Automation ▸ Design ▸ Drafting ▸ Document Control ▸ Client
cadanalyst@gmail.comhttp://cadanalyst.slack.comhttp://linkedin.com/in/cadanalyst

Binky

  • Guest
Re: Remove Duplicates from list?
« Reply #18 on: September 28, 2007, 06:13:36 PM »
Can I be his apprentice then??????

FengK

  • Guest
Re: Remove Duplicates from list?
« Reply #19 on: September 28, 2007, 07:06:58 PM »

MP

  • Seagull
  • Posts: 17750
  • Have thousands of dwgs to process? Contact me.
Re: Remove Duplicates from list?
« Reply #20 on: September 28, 2007, 07:36:02 PM »

Code: [Select]
(defun remove_doubles (lst)
  (if lst
    (cons (car lst) (remove_doubles (vl-remove (car lst) lst)))
  )
)

...

I like the way you think.

:)
Engineering Technologist • CAD Automation Practitioner
Automation ▸ Design ▸ Drafting ▸ Document Control ▸ Client
cadanalyst@gmail.comhttp://cadanalyst.slack.comhttp://linkedin.com/in/cadanalyst

MP

  • Seagull
  • Posts: 17750
  • Have thousands of dwgs to process? Contact me.
Re: Remove Duplicates from list?
« Reply #21 on: September 28, 2007, 07:53:30 PM »
What's interesting is that the original --

Code: [Select]
(defun remove_doubles (lst)
  (if lst
    (cons (car lst) (remove_doubles (vl-remove (car lst) lst)))
  )
)

Should be outperformed by this --

Code: [Select]
(defun remove_doubles2 (lst)
  (if lst
    (cons (car lst) (remove_doubles2 (vl-remove (car lst) (cdr lst))))
  )
)

But the difference in performance is negligible.

(Based on a very cursory test).

:)
« Last Edit: September 28, 2007, 08:31:44 PM by MP »
Engineering Technologist • CAD Automation Practitioner
Automation ▸ Design ▸ Drafting ▸ Document Control ▸ Client
cadanalyst@gmail.comhttp://cadanalyst.slack.comhttp://linkedin.com/in/cadanalyst

gile

  • Gator
  • Posts: 2507
  • Marseille, France
Re: Remove Duplicates from list?
« Reply #22 on: September 29, 2007, 04:59:11 AM »
What's interesting is that the original --

Code: [Select]
(defun remove_doubles (lst)
  (if lst
    (cons (car lst) (remove_doubles (vl-remove (car lst) lst)))
  )
)

Should be outperformed by this --

Code: [Select]
(defun remove_doubles2 (lst)
  (if lst
    (cons (car lst) (remove_doubles2 (vl-remove (car lst) (cdr lst))))
  )
)

But the difference in performance is negligible.

(Based on a very cursory test).

:)

May be a joke, but my poor English unable me to really understand it.

Anyway, I thank you (Se7en, Kelie, MP) for your comments, I receive them according to how I consider your great knowledge and experience in LISP writing (you and some other ones are my "Mentors").

PS: this routine is the example I use when I try to explain the recursive form.
« Last Edit: September 29, 2007, 05:24:02 AM by gile »
Speaking English as a French Frog

mkweaver

  • Bull Frog
  • Posts: 352
Re: Remove Duplicates from list?
« Reply #23 on: September 29, 2007, 09:10:14 AM »
Thanks for the response, folks.  These are great!

Mike Weaver

JohnK

  • Administrator
  • Seagull
  • Posts: 10604
Re: Remove Duplicates from list?
« Reply #24 on: October 01, 2007, 08:58:45 AM »
...
Anyway, I thank you (Se7en, Kelie, MP) for your comments, I receive them according to how I consider your great knowledge and experience in LISP writing (you and some other ones are my "Mentors").
...

That is funny because i study and learn from your code!?
TheSwamp.org (serving the CAD community since 2003)
Member location map - Add yourself

Donate to TheSwamp.org

FengK

  • Guest
Re: Remove Duplicates from list?
« Reply #25 on: October 01, 2007, 12:52:42 PM »
That is funny because i study and learn from your code!?
[/quote]
...
Anyway, I thank you (Se7en, Kelie, MP) for your comments, I receive them according to how I consider your great knowledge and experience in LISP writing (you and some other ones are my "Mentors").
...

That is funny because i study and learn from your code!?

i suppose he really meant MP.

VovKa

  • Water Moccasin
  • Posts: 1626
  • Ukraine
Re: Remove Duplicates from list?
« Reply #26 on: July 06, 2008, 05:17:49 AM »
found my way to here from a faraway thread
mine:
Code: [Select]
(defun vk_RemoveDupl
     (InList / OutList First)
  (while InList
    (setq First   (car InList)
  InList  (cdr InList)
  OutList (cons First OutList)
    )
    (if (vl-position First InList)
      (setq InList (vl-remove First InList))
    )
  )
  (reverse OutList)
)
it's slower that gile's recursive one, but has no "19975" limitation.

CAB

  • Global Moderator
  • Seagull
  • Posts: 10401
Re: Remove Duplicates from list?
« Reply #27 on: July 06, 2008, 07:22:40 AM »
Thanks for your contribution VovKa

Another look back.  http://www.theswamp.org/index.php?topic=7891.0
I've reached the age where the happy hour is a nap. (°¿°)
Windows 10 core i7 4790k 4Ghz 32GB GTX 970
Please support this web site.

VovKa

  • Water Moccasin
  • Posts: 1626
  • Ukraine
Re: Remove Duplicates from list?
« Reply #28 on: July 06, 2008, 08:02:21 AM »
interesting look back :)
sometimes duplicates are not so frequent or there could be no duplicates at all in a given list, that's why i use vl-position
try this
Code: [Select]
(progn (setq mylst nil
     i 0
       )
       (repeat 1000 (setq mylst (cons (setq i (1+ i)) mylst)))
       (benchmark '((unique mylst) (vk_RemoveDupl mylst)))
)

domenicomaria

  • Swamp Rat
  • Posts: 723
Re: Remove Duplicates from list?
« Reply #29 on: March 12, 2022, 12:55:18 PM »
(defun :LST-REMOVE-DUPLICATES (l / r)
   (while (setq i (car l) )  (setq r (cons i r)   l (vl-remove i l)   )   )
   (reverse r)
)

this one seems to be faster