Author Topic: Remove sequential duplicates from a list  (Read 2367 times)

0 Members and 1 Guest are viewing this topic.

ifncdylan

  • Guest
Remove sequential duplicates from a list
« on: March 15, 2017, 08:29:16 PM »
Hey all,

I was given some tasks to achieve, one of them was to re-format some text that had various double/triple spaces back to a single space.

I couldn't quite figure out how to do this with the existing string-trim or char-replace methods - basically it would be removing all duplicates or all spaces. So I wrote a function which will process a list and remove any duplicates that are sequential. I thought I'd post the code here as I couldn't find sequential duplicate processing stuff easily, and also I bet there are some improvements to be made to my super-newbie code that I can learn from if anyone is willing to elaborate.  :mrgreen:

At the moment you can give it a list and a character to remove the sequential duplicates - in my case I'm using vl-string->list to convert the string to list and removing sequential 32s, then converting back, works like a charm. Improvements to be made could be the option to give 'nil' as a char in order to remove ALL sequential duplicates or to give it a list of chars to use, but I wasn't quite sure on the logic to be used there.

Code - Auto/Visual Lisp: [Select]
  1. ;; - DB:removeSeqDups
  2. ;; Removes sequential duplicates in a list, leaving the first occuring item in place
  3. ;;
  4. ;; - Inputs
  5. ;; [lst] - list of items
  6. ;; [char] - item to remove
  7. ;; - Outputs
  8. ;; [lst] - processed list of items
  9.  
  10. (defun DB:removeSeqDups ( in char / out prev)
  11.    (cond
  12.      ((listp in)
  13.         (foreach item in
  14.           (if (or (and (= item char) (/= item prev))
  15.                   (/= item char))
  16.             (setq out (cons item out))
  17.             )
  18.           (setq prev item)
  19.           )
  20.       )
  21.      (T nil)
  22.      )
  23.   (reverse out)
  24. )

_$ (DB:removeSeqDups '(1 1 2 3 4 4 4 5 6 6 7 7 7 7 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 10) 9)
(1 1 2 3 4 4 4 5 6 6 7 7 7 7 8 8 8 8 8 9 10)
« Last Edit: March 15, 2017, 08:33:24 PM by ifncdylan »

hanhphuc

  • Newt
  • Posts: 64
Re: Remove sequential duplicates from a list
« Reply #1 on: March 16, 2017, 04:18:46 AM »
Code: [Select]
(defun foo (lst n / l)
     (setq l (cons (nth (vl-position n lst) lst)
   (vl-remove-if ''((x) (= x n)) (cdr (member n lst)))))
     (append (vl-remove-if ''((x) (vl-position x l)) lst) l)
     )
maybe slower than iteration due to vl-remove-if (predicate) twice?
« Last Edit: March 17, 2017, 06:51:15 AM by hanhphuc »
( apply 'equal "hp" "happy" "hạnh phúc" "ハッピー" "幸福" "행복" ) ; error: too many arguments

irneb

  • Water Moccasin
  • Posts: 1794
  • ACad R9-2016, Revit Arch 6-2016
Re: Remove sequential duplicates from a list
« Reply #2 on: March 16, 2017, 05:16:51 AM »
My suggestion ... mostly functional programming instead of procedural.
Code - Auto/Visual Lisp: [Select]
  1. (defun StrRemDupes  (source chars /)
  2.   (setq source   (vl-string->list source)
  3.         chars (cond (chars (vl-string->list chars))))
  4.   (vl-list->string
  5.     (append (vl-remove 'nil
  6.               (mapcar (function (lambda (a b)
  7.                                   (cond ((and b (= a b) (or (null chars) (member a chars))) nil)
  8.                                         (t a))))
  9.                       source (cdr source)))
  10.             (list (last source)))))


Some tests
Code: [Select]
_$ (StrRemDupes "  This Test    space   rrrremmmovve" nil)
" This Test space remove"
_$ (StrRemDupes "  This Test    space   rrrremmmovve" " ")
" This Test space rrrremmmovve"
_$ (StrRemDupes "  This Test    space   rrrremmmovve" " r")
" This Test space remmmovve"
_$ (StrRemDupes "  This Test    space   rrrremmmovve" "")
" This Test space remove"
Common sense - the curse in disguise. Because if you have it, you have to live with those that don't.

ronjonp

  • Needs a day job
  • Posts: 7526
Re: Remove sequential duplicates from a list
« Reply #3 on: March 16, 2017, 09:07:01 AM »
Here's another to remove one defined duplicate:
Code - Auto/Visual Lisp: [Select]
  1. (defun _remove (string char / r)
  2.   (setq r (strcat char char))
  3.   (while (vl-string-search r string) (setq string (vl-string-subst char r string)))
  4.   string
  5. )
  6. (_remove "Test    123  45   YAY!" " ")
  7. ;;;_$
  8. ;;;_REMOVE
  9. ;;;"Test 123 45 YAY!"
  10. ;;;_$


And one to remove all: ( be aware of this one .. will trash words that should have duplicate letters 'running' = 'runing')
Code - Auto/Visual Lisp: [Select]
  1. (defun _removeall (string / r)
  2.   (foreach l (mapcar 'chr (vl-string->list string))
  3.     (setq r (strcat l l))
  4.     (while (vl-string-search r string) (setq string (vl-string-subst l r string)))
  5.   )
  6.   string
  7. )
  8. (_remove "Tesssstttt    12333  455555                 YYYYAY!   ")
  9. ;;;_$
  10. ;;;_REMOVEALL
  11. ;;;"Test 123 45 YAY! "
  12. ;;;_$
« Last Edit: March 16, 2017, 09:56:25 AM by ronjonp »

Windows 11 x64 - AutoCAD /C3D 2023

Custom Build PC

MP

  • Seagull
  • Posts: 17750
  • Have thousands of dwgs to process? Contact me.
Re: Remove sequential duplicates from a list
« Reply #4 on: March 16, 2017, 09:51:36 AM »
Code: [Select]
I may not understand the challenge -- just woke up, haven't had coffee, feel like my head was struck with a mallet -- so offered solely for fun:

[code(progn

    (defun _CleanStr ( str flag )
        (   (lambda ( foo lst / r )
                (foreach x lst (if (foo x r) (setq r (cons x r))))
                (vl-string-trim " " (vl-list->string r))
            )
            (if flag
                (lambda (x r) (/= x (car r)))
                (lambda (x r) (not (= x 32 (car r))))
            )
            (reverse (vl-string->list str))
        )
    )

    (setq str "    This    Test    space    rrrremmmovvve    ")
    (print (_CleanStr str nil))
    (print (_CleanStr str t))   
    (princ)

)

>> "This Test space rrrremmmovvve"
>> "This Test space remove"
   

Cheers.
« Last Edit: March 16, 2017, 12:28:25 PM by MP »
Engineering Technologist • CAD Automation Practitioner
Automation ▸ Design ▸ Drafting ▸ Document Control ▸ Client
cadanalyst@gmail.comhttp://cadanalyst.slack.comhttp://linkedin.com/in/cadanalyst

Lee Mac

  • Seagull
  • Posts: 12906
  • London, England
Re: Remove sequential duplicates from a list
« Reply #5 on: March 16, 2017, 02:41:12 PM »
Another variation:
Code - Auto/Visual Lisp: [Select]
  1. (defun deldupespaces ( s )
  2.     (   (lambda ( l ) (vl-list->string (vl-remove nil (mapcar '(lambda ( a b ) (if (not (= 32 a b)) b)) (cons 32 l) l))))
  3.         (vl-string->list (vl-string-trim " " s))
  4.     )
  5. )

Nice solution Ron :)

EDIT: Oops - just realised that this is very similar to Irneb's.

ronjonp

  • Needs a day job
  • Posts: 7526
Re: Remove sequential duplicates from a list
« Reply #6 on: March 16, 2017, 02:54:14 PM »
...

Nice solution Ron :)

...
Thanks!  :)

Windows 11 x64 - AutoCAD /C3D 2023

Custom Build PC

Lee Mac

  • Seagull
  • Posts: 12906
  • London, England
Re: Remove sequential duplicates from a list
« Reply #7 on: March 16, 2017, 02:55:10 PM »
Another, recursive & inefficient:
Code - Auto/Visual Lisp: [Select]
  1. (defun deldupespaces ( s )
  2.     (if (= "" s) s
  3.         (strcat (if (wcmatch s "  *") "" (chr (ascii s))) (deldupespaces (substr s 2)))
  4.     )
  5. )

Grrr1337

  • Swamp Rat
  • Posts: 812
Re: Remove sequential duplicates from a list
« Reply #8 on: March 16, 2017, 03:07:11 PM »
Lee always wins at the "lets shorten the code" game. :D
(apply ''((a b c)(a b c))
  '(
    (( f L ) (apply 'strcat (f L)))
    (( L ) (if L (cons (chr (car L)) (f (cdr L)))))
    (72 101 108 108 111 32 87 111 114 108 100)
  )
)
vevo.bg

irneb

  • Water Moccasin
  • Posts: 1794
  • ACad R9-2016, Revit Arch 6-2016
Re: Remove sequential duplicates from a list
« Reply #9 on: March 17, 2017, 02:27:53 AM »
Lee always wins at the "lets shorten the code" game. :D
You mean "code golf"?
Common sense - the curse in disguise. Because if you have it, you have to live with those that don't.

hanhphuc

  • Newt
  • Posts: 64
Re: Remove sequential duplicates from a list
« Reply #10 on: March 17, 2017, 06:28:24 AM »
My suggestion ... mostly functional programming instead of procedural.

Thanks for the advise.
yeah again Lee's recursive idea  8-)

another
Code: [Select]
(defun foo (l n / a b )
  (if (cdr l)
  (setq a (car l) b
(cons (if (or (and (= a n) (/= a (cadr l)))
           (/= a n))
a
)
  (foo (cdr l) n)
  )
)
    (setq b (append b (list (last l))))
    )
(vl-remove nil b)
  )

OP's list
(foo
'(1 1 2 3 7 7 7 7 7 4 4 4 5 6 6 7 7 7 7 8 8 8 8 8 9 9 9 9 7 7 7 9 9 9 9 9 9 10) 7)
;;;(1 1 2 3 7 4 4 4 5 6 6 7 8 8 8 8 8 9 9 9 9 7 9 9 9 9 9 9 10)


(defun remdupchr (str $ )
(vl-list->string
(foo  (vl-string->list str) (ascii $))))

;test
('
   (($)
   (foreach x '("r" "e" "m" "o" "v" "s" " ")
    (terpri)
    (prin1 (setq $ (remdupchr $ x)))
    )
    (textpage)(princ)
   )
  "        Thisss  isss   Tesst   sspace      rrrrreemmmooovvvee"
  )


"        Thisss  isss   Tesst   sspace      reemmmooovvvee"
"        Thisss  isss   Tesst   sspace      remmmooovvve"
"        Thisss  isss   Tesst   sspace      remooovvve"
"        Thisss  isss   Tesst   sspace      removvve"
"        Thisss  isss   Tesst   sspace      remove"
"        This  is   Test   space      remove"
" This is Test space remove"

« Last Edit: March 17, 2017, 09:09:54 AM by hanhphuc »
( apply 'equal "hp" "happy" "hạnh phúc" "ハッピー" "幸福" "행복" ) ; error: too many arguments