Author Topic: Split big list...  (Read 825 times)

0 Members and 1 Guest are viewing this topic.

John Kaul (Se7en)

  • Administrator
  • Needs a day job
  • Posts: 9445
Re: Split big list...
« Reply #30 on: May 27, 2020, 06:03:03 PM »
You wouldn't store that much in memory typically. But I've processed the entire "War and Peace" novel in milliseconds in C++ (time values that small are vey hard to get accurate). You can typically get about the same time frames from C and C++. C# is good as well (MS does a good job compiling).

Here was a fun challenge using a snip from "war and peace".
https://www.theswamp.org/index.php?topic=50378.msg555005#msg555005
TheSwamp.org (serving the CAD community since 2003)

Donate to TheSwamp.org

gile

  • Water Moccasin
  • Posts: 2259
  • Marseille, France
Re: Split big list...
« Reply #31 on: May 28, 2020, 07:15:05 AM »
Sorry for my ignorance (I don't know many languages and I'm not a programmer...) just out of curiosity: with other languages can a list long 4190208 strings be processed much faster?

A quick test with F# and an array (done on VS Code)
Code - F#: [Select]
  1. let arrayFirstN array =
  2.     let n = (Array.length array) / 2
  3.     array.[.. n - 1]
  4.  
  5. let arrayLastN array =
  6.     let n = (Array.length array) / 2
  7.     array.[n ..]
  8.  
  9. let benchArray array =
  10.     let sw = System.Diagnostics.Stopwatch()
  11.     printfn "Length: %d" (Array.length array)
  12.     sw.Start()
  13.     let a1 = arrayFirstN array
  14.     sw.Stop()
  15.     printfn "Length arrayFirstN: %d" (Array.length a1)
  16.     printfn "Elapsed time for arrayFirstN: %d milliseconds" sw.ElapsedMilliseconds
  17.     sw.Reset()
  18.     sw.Start()
  19.     let a2 = arrayLastN array
  20.     sw.Stop()
  21.     printfn "Length arrayLastN: %d" (Array.length a2)
  22.     printfn "Elapsed time for arrayLastN: %d milliseconds" sw.ElapsedMilliseconds

Code: [Select]
> benchArray [| 1 .. 4190208 |];;
Length: 4190208
Length arrayFirstN: 2095104
Elapsed time for arrayFirstN: 2 milliseconds
Length arrayLastN: 2095104
Elapsed time for arrayLastN: 2 milliseconds
val it : unit = ()
Speaking English as a French Frog

ribarm

  • Water Moccasin
  • Posts: 2355
  • Marko Ribar, architect
Re: Split big list...
« Reply #32 on: May 28, 2020, 08:36:34 AM »
Only to inform... Lee's sub is fastest, but on BricsCAD...

Here is my testing result of (extremum) vs ((car (vl-sort)) :

Code: [Select]
: TEST

Elapsed time for (extremum) : 15 milliseconds...
Elapsed time for (car (vl-sort)) : 171 milliseconds...

Thanks all for participating...
And BTW. If someone would need 2-10 times faster without changing LISP, he/she should just use BricsCAD... For me it worked well on 1 example (32 - depth, 50 solutions, 2 points per (processpt)), but on another one (32 - depth, 100 - solutions, 4 points per (processpt)) it crashed around (unique) - there was probably needed (gc) call while recursing...
Marko Ribar, d.i.a. (graduated engineer of architecture)

:)

M.R. on Youtube

Marc'Antonio Alessi

  • Swamp Rat
  • Posts: 1074
  • Marco
Re: Split big list...
« Reply #33 on: May 28, 2020, 05:07:10 PM »
A quick test with F# and an array (done on VS Code)
<clip>
Length arrayFirstN: 2095104
Elapsed time for arrayFirstN: 2 milliseconds
Length arrayLastN: 2095104
Elapsed time for arrayLastN: 2 milliseconds
val it : unit = ()
Thanks for sample!  :-) :roll:

gile

  • Water Moccasin
  • Posts: 2259
  • Marseille, France
Re: Split big list...
« Reply #34 on: May 28, 2020, 05:22:59 PM »
@Marc'Antonio Alessi

The same thing using a list instead of an array gives results similar to those of LISP (F # also uses linked lists like most functional programming languages).
This illustrates what I wanted to say: the problem is less the language itself than the data structures it provides. AutoLISP only provides linked lists which are ineffective when they become too large.

Code - F#: [Select]
  1. let listFirstN lst =
  2.     let n = (List.length lst) / 2
  3.     List.take n lst
  4.  
  5. let listLastN lst =
  6.     let n = (List.length lst) / 2
  7.     List.skip n lst
  8.  
  9. let benchList list =
  10.     let sw = System.Diagnostics.Stopwatch()
  11.     printfn "Length: %d" (List.length list)
  12.     sw.Start()
  13.     let l1 = listFirstN list
  14.     sw.Stop()
  15.     printfn "Length listFirstN: %d" (List.length l1)
  16.     printfn "Elapsed time for listFirstN: %d milliseconds" sw.ElapsedMilliseconds
  17.     sw.Reset()
  18.     sw.Start()
  19.     let l2 = listLastN list
  20.     sw.Stop()
  21.     printfn "Length listLastN: %d" (List.length l2)
  22.     printfn "Elapsed time for listLastN: %d milliseconds" sw.ElapsedMilliseconds

Code: [Select]
> benchList [1 .. 4190208];;
Length: 4190208
Length listFirstN: 2095104
Elapsed time for listFirstN: 141 milliseconds
Length listLastN: 2095104
Elapsed time for listLastN: 14 milliseconds
val it : unit = ()
Speaking English as a French Frog