TheSwamp

Code Red => AutoLISP (Vanilla / Visual) => Topic started by: whdjr on December 02, 2005, 09:39:24 AM

Title: Remove duplicates in a list
Post by: whdjr on December 02, 2005, 09:39:24 AM
I have this list:

an example:
Code: [Select]
(list '("me" "5" "3" "4" "5") '("you" "6" "7" "8" "10") '("him" "6" "7" "8" "10")
      '("our" "6" "7" "8" "10") '("you" "6" "7" "8" "10") '("her" "6" "7" "8" "10")
      '("we" "6" "7" "8" "10"))
)
All I am concerned about is the first element.  I want to be able to scan this list and remove all but but one copy of the items that are duplicate only.

Using the above list as an example I want it to return this:
Code: [Select]
(list '("me" "5" "3" "4" "5")
      '("you" "6" "7" "8" "10")
      '("him" "6" "7" "8" "10")
      '("our" "6" "7" "8" "10")
      '("her" "6" "7" "8" "10")
      '("we" "6" "7" "8" "10"))

The second item that started with "you" got removed because it was a duplicate.

Can anyone help me out here? Maybe some suggestions?

Thanks,
Title: Re: Remove duplicates in a list
Post by: MP on December 02, 2005, 09:55:26 AM
Code: [Select]
(defun unique ( lst / result )
    (foreach item lst
        (if (null (member item result))
            (setq result (cons item result))
        )
    )
    (reverse result)
)

Coded blind but should do it.
Title: Re: Remove duplicates in a list
Post by: whdjr on December 02, 2005, 10:06:08 AM
Thanks MP.

Very close but needs a little 'bit 'o tweeking.
It removes a list if every item is the two lists are the same but not if only some are the same.

Using this example list:
Code: [Select]
(list '("me" "5" "3" "4" "5")
       '("you" "6" "7" "8" "10")
       '("him" "6" "7" "8" "10")
       '("you" "6" "7" "8" "10")
       '("her" "6" "7" "8" "10")
       '("me" "6" "7" "8" "10"))

It removes the second 'you' because each element is the same but it doesn't remove the second 'me'.
Title: Re: Remove duplicates in a list
Post by: whdjr on December 02, 2005, 10:24:35 AM
MP upon further thinking I decided yo use your tool just the way it is.  I decided that my duplicate lists would always be the same and if they weren't then I needed to know that.

Thanks,
Title: Re: Remove duplicates in a list
Post by: MP on December 02, 2005, 10:26:15 AM
I thought you wanted duplicates removed??

Do you want items removed on the basis of duplicated cars (first item)? Then perhaps --

Code: [Select]
(defun unique2 ( lst / result )
    (foreach item lst
        (if (null (assoc (car item) result))
            (setq result (cons item result))
        )
    )
    (reverse result)
)

Title: Re: Remove duplicates in a list
Post by: Fatty on December 02, 2005, 10:51:38 AM
And another one

 
Code: [Select]
   
(defun remove-first (lst /)
  (while
    (member (caar lst) (mapcar 'car (cdr lst)))
     (setq lst (vl-remove-if
(function (lambda (x)
     (eq (car x) (caar lst))
   )
)
lst
       )
     )
  )
  lst
)
Title: Re: Remove duplicates in a list
Post by: Fatty on December 02, 2005, 10:55:36 AM
Oops

Sorry, forget about it
I did misplaced

Fatty
Title: Re: Remove duplicates in a list
Post by: whdjr on December 02, 2005, 10:59:46 AM
I thought you wanted duplicates removed??

Do you want items removed on the basis of duplicated cars (first item)? Then perhaps --

...All I am concerned about is the first element. I want to be able to scan this list and remove all but but one copy of the items that are duplicate only...

This is what I asked for MP but you knew better and provided what I needed -- I just didn't realize it then. :-)

Thanks a bunch MP,
 :-) :-)
Title: Re: Remove duplicates in a list
Post by: MP on December 02, 2005, 11:04:38 AM
Thanks a bunch MP,
 :-) :-)

My pleasure Will.

:) :) :)
Title: Re: Remove duplicates in a list
Post by: MP on December 02, 2005, 11:20:46 AM
Mr. Oleg my friend. I think yours needs a tweak (perhaps that's what you meant by "I did misplaced"), it removes all items that have duplicates, that is, it returns --

(   
    ("him" "6" "7" "8" "10")
    ("her" "6" "7" "8" "10")
)

for Will's test list of --

'( 
    ("me" "5" "3" "4" "5")
    ("you" "6" "7" "8" "10")
    ("him" "6" "7" "8" "10")
    ("you" "6" "7" "8" "10")
    ("her" "6" "7" "8" "10")
    ("me" "6" "7" "8" "10")
)

When I believe he desired --

(   
    ("me" "5" "3" "4" "5")
    ("you" "6" "7" "8" "10")
    ("him" "6" "7" "8" "10")
    ("her" "6" "7" "8" "10")
)

Also, a little incidental note here, while vl-remove is convenient, and indeed I used it (lots), in cases like this its performance sometimes lags behind a roll your own variant using "classic" lisp equivalents (primarilly because of the insitu mapcar calls I'm guessing).

An informal test, based on unique2 and remove-first (as is) and Will's test list suggests unique2 is about 60% faster, though admittedly, sometimes this matters not.

Hope you don't mind my observations.

:)
Title: Re: Remove duplicates in a list
Post by: MP on December 02, 2005, 11:29:32 AM
!! If you use a list like this --

(setq lst
   '(   
        ("me" "5" "3" "4" "5")
        ("you" "6" "7" "8" "10")
        ("him" "6" "7" "8" "10")
        ("you" "6" "7" "8" "10")
        ("her" "6" "7" "8" "10")
        ("me" "6" "7" "8" "10")
    )
)

(repeat 8
    (setq lst
        (append lst lst)
    )
)


The difference in performance is even more dramatic, like > 300% !!
Title: Re: Remove duplicates in a list
Post by: CAB on December 02, 2005, 11:40:54 AM
Just to see if I could find another way.
Not better mind you just different. :-)

Code: [Select]
  (defun unique (lst / result)
    (while (setq itm (car lst))
      (setq lst    (vl-remove itm lst)
            result (cons itm result)
      )
    )
  )
Title: Re: Remove duplicates in a list
Post by: Fatty on December 02, 2005, 11:47:37 AM
Quote
..............

Hope you don't mind my observations.

:)

   Thank you Michael, I certainly shall not begin to object, especially
    that yours explanations very detailed and correct.
    In addition I have incorrectly understood a problem that it was necessary to solve
    and because of it I has given the wrong answer to a question :oops:

    Oleg
Title: Re: Remove duplicates in a list
Post by: MP on December 02, 2005, 11:48:17 AM
Interesting performance Alan!

Elapsed milliseconds / relative speed for 256 iteration(s):

    (UNIQUE_CAB LST).....1172 / 2.52 <fastest>
    (UNIQUE_MP LST)......2953 / 1.00 <slowest>
Title: Re: Remove duplicates in a list
Post by: CAB on December 02, 2005, 12:34:36 PM
Wow, that was unexpected! :-o
Although the loop is shortened by using the while and therefore
duplicates are not tested, I think it must be the vl-remove