Topic: Split big list...

[JohnK]

You wouldn't store that much in memory typically. But I've processed the entire "War and Peace" novel in milliseconds in C++ (time values that small are vey hard to get accurate). You can typically get about the same time frames from C and C++. C# is good as well (MS does a good job compiling).

Here was a fun challenge using a snip from "war and peace".
https://www.theswamp.org/index.php?topic=50378.msg555005#msg555005

[gile]

Sorry for my ignorance (I don't know many languages and I'm not a programmer...) just out of curiosity: with other languages can a list long 4190208 strings be processed much faster?

A quick test with F# and an array (done on VS Code)

Code - F#: [Select]

1. let arrayFirstN array =
2.     let n = (Array.length array) / 2
3.     array.[.. n - 1]
4.  
5. let arrayLastN array =
6.     let n = (Array.length array) / 2
7.     array.[n ..]
8.  
9. let benchArray array =
10.     let sw = System.Diagnostics.Stopwatch()
11.     printfn "Length: %d" (Array.length array)
12.     sw.Start()
13.     let a1 = arrayFirstN array
14.     sw.Stop()
15.     printfn "Length arrayFirstN: %d" (Array.length a1)
16.     printfn "Elapsed time for arrayFirstN: %d milliseconds" sw.ElapsedMilliseconds
17.     sw.Reset()
18.     sw.Start()
19.     let a2 = arrayLastN array
20.     sw.Stop()
21.     printfn "Length arrayLastN: %d" (Array.length a2)
22.     printfn "Elapsed time for arrayLastN: %d milliseconds" sw.ElapsedMilliseconds

Code: [Select]

> benchArray [| 1 .. 4190208 |];;
Length: 4190208
Length arrayFirstN: 2095104
Elapsed time for arrayFirstN: 2 milliseconds
Length arrayLastN: 2095104
Elapsed time for arrayLastN: 2 milliseconds
val it : unit = ()

[ribarm]

Only to inform... Lee's sub is fastest, but on BricsCAD...

Here is my testing result of (extremum) vs ((car (vl-sort)) :

Code: [Select]

: TEST

Elapsed time for (extremum) : 15 milliseconds...
Elapsed time for (car (vl-sort)) : 171 milliseconds...

Thanks all for participating...
And BTW. If someone would need 2-10 times faster without changing LISP, he/she should just use BricsCAD... For me it worked well on 1 example (32 - depth, 50 solutions, 2 points per (processpt)), but on another one (32 - depth, 100 - solutions, 4 points per (processpt)) it crashed around (unique) - there was probably needed (gc) call while recursing...

[Marc'Antonio Alessi]

A quick test with F# and an array (done on VS Code)
<clip>
Length arrayFirstN: 2095104
Elapsed time for arrayFirstN: 2 milliseconds
Length arrayLastN: 2095104
Elapsed time for arrayLastN: 2 milliseconds
val it : unit = ()

Thanks for sample! 

[gile]

@Marc'Antonio Alessi

The same thing using a list instead of an array gives results similar to those of LISP (F # also uses linked lists like most functional programming languages).
This illustrates what I wanted to say: the problem is less the language itself than the data structures it provides. AutoLISP only provides linked lists which are ineffective when they become too large.

Code - F#: [Select]

1. let listFirstN lst =
2.     let n = (List.length lst) / 2
3.     List.take n lst
4.  
5. let listLastN lst =
6.     let n = (List.length lst) / 2
7.     List.skip n lst
8.  
9. let benchList list =
10.     let sw = System.Diagnostics.Stopwatch()
11.     printfn "Length: %d" (List.length list)
12.     sw.Start()
13.     let l1 = listFirstN list
14.     sw.Stop()
15.     printfn "Length listFirstN: %d" (List.length l1)
16.     printfn "Elapsed time for listFirstN: %d milliseconds" sw.ElapsedMilliseconds
17.     sw.Reset()
18.     sw.Start()
19.     let l2 = listLastN list
20.     sw.Stop()
21.     printfn "Length listLastN: %d" (List.length l2)
22.     printfn "Elapsed time for listLastN: %d milliseconds" sw.ElapsedMilliseconds

Code: [Select]

> benchList [1 .. 4190208];;
Length: 4190208
Length listFirstN: 2095104
Elapsed time for listFirstN: 141 milliseconds
Length listLastN: 2095104
Elapsed time for listLastN: 14 milliseconds
val it : unit = ()

[Marc'Antonio Alessi]

@gile  Very interesting.