TheSwamp
Code Red => AutoLISP (Vanilla / Visual) => Topic started by: Kerry on February 22, 2005, 12:07:39 AM
-
Remove duplicate Strings in List
Anyone have any alternatives to these ??
(setq full_list (list "c" "a" "b" "a" "d" "b" "c" "e"))
( RemoveDuplicateStrings full_list) ;;->> ("c" "a" "b" "d" "e")
( SortAndRemoveDuplicateStrings full_list) ;;->> ("a" "b" "c" "d" "e")
;;option
(vl-sort ( RemoveDuplicateStrings full_list) '<) ;;->> ("a" "b" "c" "d" "e")
(defun SortAndRemoveDuplicateStrings (stringlist / newlist lastin)
;;kwb 20050221
(foreach var (vl-sort stringlist '<)
(if (/= var lastin)
(setq newlist (cons (setq lastin var) newlist))
)
)
(reverse newlist)
)
(defun RemoveDuplicateStrings (stringlist / newlist)
;;kwb 20050221
(foreach var stringlist
(if (not (vl-position var newlist))
(setq newlist (cons var newlist))
)
)
(reverse newlist)
)
I'm brain dead, and would appreciate any insight ..
.. probably time for MP's benchmarker too.
-
Here is an alternate 'Remove Duplicates'
Did not record the author
;---Remove Duplicate members from a list.
; Given list = '(1 1 2 3 4 "a" "b" "a")
; Returns '(1 2 3 4 "a" "b")
(defun Remove_Dup (LST / CL)
(mapcar
'(lambda (X)
(cond
( (member X LST)
(cond
( (not (member X CL)) ;if element not already in CL
(setq CL (cons X CL))
)
) ; end cond.
)
) ; end cond.
) ;end lambda
LST
) ; end mapcar
(reverse CL)
) ;end defun
-
Perhaps it could be simplied to this:
(defun remove_dup (lst / cl)
(mapcar
'(lambda (x)
(if (not (member x cl)) ;if element not already in CL
(setq cl (cons x cl))
)
)
lst
)
(reverse cl)
) ;end defun
Which is what you have :shock:
-
Not quite what I have.
CAB, This will rock you a little perhaps < or not > ...
The REMOVE_DUP2 is the same as yours, with the MEMBER changed to VL-POSITION.
.. so the difference will be in the MAPCAR and LAMBDA where I used FOREACH
The test list was a list of duplicate DWG name strings [ 2 instances of 25 for a total of 50 ]
Benchmarking ................Elapsed milliseconds / relative speed for 8192 iteration(s):
(REMOVE_DUP FULL_LIST).................5938 / 3.1203 <slowest>
(REMOVE_DUP2 FULL_LIST)................4236 / 2.2260
(REMOVEDUPLICATESTRINGS FULL_LIST).....1903 / 1.0000 <fastest>
-
So what your saying is that Mapcar is slower.
-
uhmmmmm ..
I'm just looking at the numbers Se7en.
-
Oh, and Michael .. I flipped the report list around on my benchmark version so that the fastest was the lowest ratio .. and the slowest shows the larger number < elapsed time >.
-
No it doesn't rock me. I was just providing an alternative which I found somewhere else.
When I removed some of the code it looked similar to yours but using the mapcar / lambda
as you pointed out. I didn't think it would out preform yours because you obviously were
proud enough to put you name in the code. I just offered it up as food for thought as you
said "I'm brain dead, and would appreciate any insight .."
Sorry it wasn't what you were looking for.
-
Well your right in this instance.
Mapcar is recursive by nature, foreach isnt. Therefore every time you are using a mapcar+lambda you are waiting for the intrip. to add a function definition to the stack.
-
Hi CAB.
Wasn't dissing you :D
... or the routine
and I was seriously looking for feedback, not a pat on the back.
The problem is sometimes I get my head burried in code and miss the 'obvious' alternatives, hence the post.
-
Kerry, run this with the same paramaters as before and cross your fingers. I want to see if we can speed up this process a bit.
(defun remove_dup (lst / cl)
(defun workhorse (x)
(if (not (member x cl)) ;if element not already in CL
(setq cl (cons x cl))))
(mapcar 'workhorse lst)
(reverse cl)
)
-
Wrapping it like that doesn't help, sorry.
Benchmarking ................Elapsed milliseconds / relative speed for 8192 iteration(s):
(REMOVE_DUP FULL_LIST).................5978 / 3.0609 <slowest>
(REMOVE_DUP2 FULL_LIST)................4347 / 2.2258
(REMOVE_DUP_Se7en FULL_LIST)...........3765 / 1.9278
(REMOVEDUPLICATESTRINGS FULL_LIST).....1953 / 1.0000 <fastest>
-
Compare the performance of this --
(defun RemoveDuplicates ( lst / temp )
(vl-remove-if
'(lambda (x)
(cond
((vl-position x temp) t)
((setq temp (cons x temp)) nil)
)
)
lst
)
)
Against preceeding variants using this list --
(setq lst '(1 2 3 4 5 6 7 8))
(repeat 10
(setq lst
(append lst lst)
)
)
Also (Kerry) please tell me about the speedometer in your vehicle.
:cheesy:
-
Even more amusing is the performance of this --
(defun RemoveDuplicates ( lst / foo temp )
(defun foo (x)
(cond
((vl-position x temp) t)
((setq temp (cons x temp)) nil)
)
)
(vl-remove-if
'foo
lst
)
)
-
Benchmarking ................Elapsed milliseconds / relative speed for 8192 iteration(s):
(REMOVE_DUP FULL_LIST).................5938 / 3.1203 <slowest>
(REMOVE_DUP2 FULL_LIST)................4236 / 2.2260
(REMOVEDUPLICATESTRINGS FULL_LIST).....1903 / 1.0000 <fastest>
Benchmarking ................Elapsed milliseconds / relative speed for 8192 iteration(s):
(REMOVE_DUP FULL_LIST).................5978 / 3.0609 <slowest>
(REMOVE_DUP2 FULL_LIST)................4347 / 2.2258
(REMOVE_DUP_Se7en FULL_LIST)...........3765 / 1.9278
(REMOVEDUPLICATESTRINGS FULL_LIST).....1953 / 1.0000 <fastest>
But it looks like it did. If I'm reading your benchmark correct. I imporved the preformance of the procedure by over 60% by just changing one thing. (using a NAMED procedure instead of an UNNAMED.) Am i right?
-
Michael,
(benchmark '((removeduplicatestrings full_list)
(RemoveDuplicates full_list)
(remove_dupSe7en full_list)
(remove_dup2 full_list)
(remove_dup full_list)
)
)
Benchmarking ...............Elapsed milliseconds / relative speed for 4096 iteration(s):
(REMOVE_DUP FULL_LIST).................3075 / 3.0719 <slowest>
(REMOVE_DUP2 FULL_LIST)................2193 / 2.1908
(REMOVE_DUPSE7EN FULL_LIST)............1902 / 1.9001
(REMOVEDUPLICATES FULL_LIST)...........1532 / 1.5305
(REMOVEDUPLICATESTRINGS FULL_LIST).....1001 / 1.0000 <fastest>
(setq lst '(1 2 3 4 5 6 7 8))
(repeat 10
(setq lst
(append lst lst)
)
)
(benchmark '((removeduplicatestrings lst)
(RemoveDuplicates lst)
(remove_dupSe7en lst)
(remove_dup2 lst)
(remove_dup lst)
)
)
Benchmarking ...........Elapsed milliseconds / relative speed for 256 iteration(s):
(REMOVE_DUP LST).................10004 / 5.3440 <slowest>
(REMOVE_DUP2 LST).................8072 / 4.3120
(REMOVE_DUPSE7EN LST).............6119 / 3.2687
(REMOVEDUPLICATESTRINGS LST)......3756 / 2.0064
(REMOVEDUPLICATES LST)............1872 / 1.0000 <fastest>
-
Michael, looks like your REMOVEDUPLICATESFOO is a screamer for the String List.
.. and your REMOVEDUPLICATES for the numberlist
The differences are significant, heh ..
Benchmarking ...........Elapsed milliseconds / relative speed for 256 iteration(s):
(REMOVE_DUP LST).................10496 / 5.4020 <slowest>
(REMOVE_DUP2 LST).................8302 / 4.2728
(REMOVE_DUPSE7EN LST).............6359 / 3.2728
(REMOVEDUPLICATESTRINGS LST)......3936 / 2.0257
(REMOVEDUPLICATESFOO LST).........2233 / 1.1493
(REMOVEDUPLICATES LST)............1943 / 1.0000 <fastest>
Benchmarking ................Elapsed milliseconds / relative speed for 8192 iteration(s):
(REMOVE_DUP FULL_LIST).................6129 / 3.7555 <slowest>
(REMOVE_DUP2 FULL_LIST)................4476 / 2.7426
(REMOVE_DUPSE7EN FULL_LIST)............3886 / 2.3811
(REMOVEDUPLICATES FULL_LIST)...........2834 / 1.7365
(REMOVEDUPLICATESTRINGS FULL_LIST).....1903 / 1.1661
(REMOVEDUPLICATESFOO FULL_LIST)........1632 / 1.0000 <fastest>
-
Also (Kerry) please tell me about the speedometer in your vehicle.
:cheesy:
You mean this ?
System:
Microsoft Windows XP
Professional
Version 2002
Service Pack 2
Computer:
Intel(R)
Pentium(R) 4 CPU 2.00GHz
2.02 GHz, 1.00 GB of RAM
-
< .. >But it looks like it did. If I'm reading your benchmark correct. I imporved the preformance of the procedure by over 60% by just changing one thing. (using a NAMED procedure instead of an UNNAMED.) Am i right?
ahh , I see what you mean now .. Yes, That change is better.
-
Small tweak makes it faster yet --
(defun RemoveDuplicates ( lst / foo index )
(defun foo (x)
(cond
((vl-position x index))
((null (setq index (cons x index))))
)
)
(vl-remove-if
'foo
lst
)
)
-
And still greater performance is realized by separating the functions --
(defun RemoveDuplicatesAux ( x )
(cond
((vl-position x index))
((null (setq index (cons x index))))
)
)
(defun RemoveDuplicates ( lst / index )
(vl-remove-if
'RemoveDuplicatesAux
lst
)
)
I'm guessing you'd like to mention that variable 'index' is not local in RemoveDuplicatesAux. Tell me what you think the implications are. :)
-
Thanks for your continued input on this topic Michael.
I think we may have discussed the nature of the personality aberration that manifests itself in this sort of dedication.
:D
My results indicate that this approach is not as fast as the wrapped foo approach for your number lists and is a winner for the Strings, which is where we started. The relative speed differences are that minimal now that other issues will come into consideration.
The declaration of the local variable fascinated me. I was presumptuous enough to consider it redundant, untill I saw the results of removing it and realised why it was required.
..that would be a nightmare to debug for the unwary and is an excellent demonstration of variable scope.
Thanks Michael
Kerry.
-
Ahhh, I see you read the small print; excellent. I'll resist participating in dialogues about my abberant fascinations, we banned someone for that yesterday. Anyway ...
For those that may desire more illumination about variable scope consider the following (observe variable 'temp') ...
;; demo function
;; works with variable
;; temp appropriate to
;; lexical scope
(defun ConsXToVariableTemp ( x )
(princ
(strcat
"=> temp : "
(vl-princ-to-string
(setq temp
(cons x temp)
)
)
"\n"
)
)
)
;; separate function definition
;; with nested (local) functions
(defun c:Test ( / implicit explicit temp )
;; variable temp is local to function
;; because it is an argument to the
;; function
(defun implicit ( temp )
(ConsXToVariableTemp 1)
)
;; variable temp is local to function
;; because it is formally declared as
;; having local scope in the function
;; definition
(defun explicit ( / temp )
(setq temp '(7 8 9))
(ConsXToVariableTemp 6)
)
;; call function implicit, passing a
;; small list as the parameter
(implicit '(2 3 4))
;; call function explicit, no parameters
(explicit)
;; define and invoke an annonymous function
;; that explicitly declares temp as having
;; local scope; parameterless
( (lambda ( / temp )
(setq temp '(b c d))
(ConsXToVariableTemp 'a)
)
)
;; define and invoke an annonymous function
;; that implicitly declares temp as having
;; local scope because it's a parameter
( (lambda (temp)
(ConsXToVariableTemp '+)
)
'(- * /) ;; passed as the parameter
)
;; call ConsXToVariableTemp directly after
;; setting variable temp that has scope local
;; to function c:Test
(setq temp '(@ # $))
(ConsXToVariableTemp '!)
;; shhh
(princ)
)
;; before we call c:Test let's set
;; global variable 'temp' to something
(setq temp "Lexical Scope, whoa.")
;; show whats bound to temp
!temp
=> "Lexical Scope, whoa."
;; perform the test run
(c:Test)
=> temp : (1 2 3 4)
=> temp : (6 7 8 9)
=> temp : (A B C D)
=> temp : (+ - * /)
=> temp : (! @ # $)
;; show whats bound to temp
!temp
=> "Lexical Scope, whoa."
Kerry is 100% correct - code like this can be difficult to debug if you lose track of the scope. As such I normally avoid it myself and do not recommend it as normal practice.
And now for something different ...
(defun ConsToXQuotedVariable ( x QuotedVariable )
(set QuotedVariable
(cons x
(eval QuotedVariable)
)
)
)
(setq temp '(2 3 4))
!temp
=> (2 3 4)
(ConsToXQuotedVariable 1 'temp)
!temp
=> (1 2 3 4)
Cheers.
-
Ahhh, I see you read the small print;
ohhh, no, sneaky ..
-
Guilty as charged.
-
re the ConsToXQuotedVariable
I use these complimentary routines where applicable
(defun pop_in (val qvar)
(set qvar (cons val (eval qvar)))
)
(defun pop_out (val qvar)
(set qvar (vl-remove val (eval qvar)))
)
-
It's like you can see my library; scary.
-
this is my code...
(setq full_list (list "c" "a" "b" "a" "d" "b" "c" "e"))
(setq #list (vl-list-length full_list))
(setq ts1 "")
(repeat #list
(if (not (vl-string-search (car full_list) ts1))
(progn
(setq ts1 (strcat ts1 (car full_list)))
(setq full_list (cdr full_list))
)
(progn
(setq #list (+ #list 1))
(setq full_list (cdr full_list))
)
))