Topic: What's the fastest way to transfer string of coordinates to list of points ?

[Coder]

Hi everyone .

I have hundreds of coordinate points that are existed in a .txt file and I want to read them to lisp file to insert for
each coordinate as a point or block , so I wrote some codes .

Could and would you please help me with it ? 

Code: [Select]

(if (setq fl (getfiled "Select File" (getvar 'dwgprefix) "txt" 8))
  (progn
    (setq f (open fl "r"))
    (setq r (read-line f)) ;; =>  "15.53215 7.451254 -10.4551"
    .....
    ............
    ..................

Thank you so much guys

[Lee Mac]

Perhaps not the fastest program, but one that is already written 

http://lee-mac.com/ptmanager.html

[pBe]

Hi everyone .

I have hundreds of coordinate points that are existed in a .txt file and I want to read them to lisp file to insert for
each coordinate as a point or block , so I wrote some codes .

Could and would you please help me with it ? 

Code: [Select]

(if (setq fl (getfiled "Select File" (getvar 'dwgprefix) "txt" 8))
  (progn
    (setq f (open fl "r"))
    (setq r (read-line f)) ;; =>  "15.53215 7.451254 -10.4551"
    .....
    ............
    ..................

Thank you so much guys

(read (strcat "(" r ")")

[Coder]

Perhaps not the fastest program, but one that is already written 

http://lee-mac.com/ptmanager.html

I am sorry , your program did not work for me and I don't know why !!!!

I selected the txt file and selected point option from the list of your program and it did nothing and the pdmode is equal 66

Can you please help me with it ?

forms of the coordinates

0.155750 5.085015 -4.799957
0.152540 5.087977 -4.799713

[Coder]

(read (strcat "(" r ")")

Thanks , that looks interesting .

[Lee Mac]

Perhaps not the fastest program, but one that is already written 

http://lee-mac.com/ptmanager.html

I am sorry , your program did not work for me and I don't know why !!!!

I selected the txt file and selected point option from the list of your program and it did nothing and the pdmode is equal 66

Can you please help me with it ?

forms of the coordinates

0.155750 5.085015 -4.799957
0.152540 5.087977 -4.799713

Did you select the correct data delimiter (in your case 'Space')?

[Coder]

Perhaps not the fastest program, but one that is already written 

http://lee-mac.com/ptmanager.html

I am sorry , your program did not work for me and I don't know why !!!!

I selected the txt file and selected point option from the list of your program and it did nothing and the pdmode is equal 66

Can you please help me with it ?

forms of the coordinates

0.155750 5.085015 -4.799957
0.152540 5.087977 -4.799713

Did you select the correct data delimiter (in your case 'Space')?

Yes that's what I missed , and although the computer is hanging because the coordinates file is about 100 MB and I am almost sure the coordinates are more than 250.000 points .

How to run that huge file of coordinates ?? ( big question )

Thank you

[efernal]

using bPe suggestion:


(SETQ r "10.0 10.0 0.0")
(ENTMAKE (LIST (CONS 0 "POINT") (CONS 10 (READ (STRCAT "(" r ")")))))

[Lee Mac]

Using pBe's suggestion:

Code: [Select]

(defun c:test ( / f l )
    (if
        (and
            (setq f (getfiled "" "" "txt" 16))
            (setq f (open f "r"))
        )
        (progn
            (while (setq l (read-line f))
                (entmakex (list (cons 0 "POINT") (cons 10 (read (strcat "(" l ")")))))
            )
            (setq f (close f))
        )
    )
    (princ)
)

[Coder]

I saw lots of points drawn in the screen but and during that process I have received a message and it says ( Autocad exceeding system memory ) not
the exact message but the almost the same meaning , and before finishing prompt a message at the command line : unwinned and unexceptional error .

and at last Autocad report to exit .  :ugly:

OMG I could't succeed the trick .

Thanks for all the hard work and your time guys .

( I think there is no chance to do the trick with things with these kind huge coordinates file to points ( but hope somebody would break through it and do it )) 

[Lee Mac]

Try breaking down the file into multiple smaller files, then import each one.

100MB sounds absolutely mahoosive though 

[alanjt]

mahoosive

[Jeff_M]

Code: [Select]

                (entmakex (list (cons 0 "POINT") (cons 10 (read (strcat "(" l ")")))))


It's been quite a while since I've done any serious work in lisp, so maybe I've missed this discussion......why the use of ENTMAKEX instead of ENTMAKE? From the help file:

Warning Objects and entities without owners are not written out to DWG or DXF files. Be sure to set an owner at some point after using entmakex.

Could this be the cause of Acad failing with so many un-owned objects being added? (Probably not, just caught my eye as a possibility.)

[alanjt]

Code: [Select]

                (entmakex (list (cons 0 "POINT") (cons 10 (read (strcat "(" l ")")))))


It's been quite a while since I've done any serious work in lisp, so maybe I've missed this discussion......why the use of ENTMAKEX instead of ENTMAKE? From the help file:

Warning Objects and entities without owners are not written out to DWG or DXF files. Be sure to set an owner at some point after using entmakex.

Could this be the cause of Acad failing with so many un-owned objects being added? (Probably not, just caught my eye as a possibility.)

To build on Jeff's comment, since you are already using strcat, you could also include your dxf 10 code in there, instead of using cons once the list is created.

Code: [Select]

(entmake (list '(0 . "POINT") (read (strcat "( 10 " l ")"))))

Code: [Select]

(benchmark '(
(entmakex (list (cons 0 "POINT") (cons 10 (read (strcat "(" l ")")))))
(entmakex (list '(0 . "POINT") (read (strcat "( 10 " l ")"))))
(entmake (list '(0 . "POINT") (read (strcat "( 10 " l ")"))))
            )
)

Code: [Select]

    (ENTMAKE (LIST (QUOTE (0 . "POINT"))...).....1061 / 1.16 <fastest>
    (ENTMAKEX (LIST (QUOTE (0 . "POINT")...).....1155 / 1.07
    (ENTMAKEX (LIST (CONS 0 "POINT") (CO...).....1232 / 1 <slowest>

[Lee Mac]

The code was hastily written, but that's no excuse - good point guys, great suggestions.

[Coder]

I guess I have an idea but without the ability to write codes for it .

Can I divide the txt file into three or four files by using lisp which is going to select a txt file and cut a 50.000 lines of strings of the file and paste them ( I mean write them to a txt file with another name and remove them from the source file )

Otherwise should I look for Autodesk program that could do the easily (3D civil or MAP or etc ....) I have no idea which is the right one .

Hope you bear with me guys you've been so kind with me ...

Thank you so much

[dgorsman]

Anything more than a few thousand (literally, not tens/hundreds of thousands) points are better handled in dedicated point cloud software.  They use all sorts of tricks to speed up processing and making useable.  Newer versions of AutoCAD and related products include the Faro engine, if that helps.

[JohnK]

<post removed>

[Lee Mac]

Hacked together to divide the file:

Code: [Select]

(defun c:dividefile ( / fn i j l lines rf wf )

    (setq lines 50000) ;; Number of Lines for each file
   
    (if
        (and
            (setq fn (getfiled "" "" "txt" 16))
            (setq rf (open fn "r"))
            (setq fn (fnsplitl fn)
                  fn (strcat (car fn) (cadr fn))
            )
        )
        (progn
            (setq i 0)
            (while (setq l (read-line rf))
                (if (setq wf (open (strcat fn "_" (itoa (setq i (1+ i))) ".txt") "w"))
                    (progn
                        (setq j 1)
                        (write-line l wf)
                        (while (and (<= (setq j (1+ j)) lines) (setq l (read-line rf)))
                            (write-line l wf)
                        )
                        (close wf)
                    )
                )
            )
            (close rf)
        )
    )
    (princ)
)

[Lee Mac]

Not sure if Daniel made an AutoCAD version of this?

[Lee Mac]

To update my earlier code to incorporate Alanjt & Jeff_M's suggestions:

Code: [Select]

(defun c:test ( / f l )
    (if
        (and
            (setq f (getfiled "" "" "txt" 16))
            (setq f (open f "r"))
        )
        (progn
            (while (setq l (read-line f))
                (entmake (list '(0 . "POINT") (read (strcat "( 10 " l ")"))))
            )
            (setq f (close f))
        )
    )
    (princ)
)

[JohnK]

<post removed>

[Coder]

I just wrote a program (in C++) to add a paren before and after the coordinates and tested it out on a file with 10,000 lines in it.

process-coordinate-file.exe -i input_me.txt -o out.txt
This operation took: 0.0065025 seconds.

Not sure if that would help you at all but you are more than welcome to have it if you want it.

You're too kind , of course I would like ti have it and I may in need to know how load it and run it as well .

many thanks.

[JohnK]

<post removed>

[Coder]

I did not get what you mean that well with paren , Does it mean that I have to divide the txt file that contains the coordinates or so ( I did not get it ).

Won't you program import the coordinates directly from file to Autocad as points ?

I am sorry for not being able to understand your point .

Many thanks

[JohnK]

<post removed>

[Coder]

That's alright.

No, you guys were having a hard time reading and processing that many items in the file so I asked you if you would like me to write you a program in C++ to do that processing for you...so all you had to do was to read the file into your program (saving the processing time/steps).  I suggested this because I can process a huge amount of data in milliseconds with C++ (get the data ready for you to read with lisp).

If you don't want the program that's okay too.

That sounds great , and I do looking forward to see my problem disappear with long txt file with coordinates with Autocad .

Waiting for your kind reply and application .

Thank you

[JohnK]

<post removed>

[Coder]

According to what I understood from you that you would divide the txt file into another separate more files to reduce the quantity of coordinates into each one , but I think this is not the issue at the moment .

To speed up the process of importing coordinates and insert them as points at every coordinate of it , is what I am looking for .

Dividing file into pieces is not that important since Lee supported us with excellent one .

Your zip file hold only one file (index.php) only and there is not anything else .

Thank you so much for your time and for interests .

[JohnK]

<post removed>

[JohnK]

<post removed>

[Lee Mac]

Dividing file into pieces is not that important since Lee supported us with excellent one .

Did dividing the file work for you?

[Coder]

Dividing file into pieces is not that important since Lee supported us with excellent one .

Did dividing the file work for you?

Worked perfectly without a piece of doubt .

Thank you so much Lee , you made my day .

[pBe]

Hacked together to divide the file:

Code: [Select]

(defun c:dividefile ( / fn i j l lines rf wf )

   .....)

Nice Lee.

Question: Which approach eats up more memory,
Reading the file collect the list THEN entamke the point as opposed to entmake the point while reading the file?

Code: [Select]

(defun c:test ( / f l )
    (if
        (and
    (setq f (findfile "d:\\My Documents\\pts.txt"))
            (setq f (open f "r"))
        )
        (progn
            (while (setq l (read-line f))
                (entmake (list '(0 . "POINT") (read (strcat "( 10 " l ")"))))
            )
            (setq f (close f))
        )
    )
    (princ)
)

Code: [Select]

(defun c:test2 ( / f l pt)
    (if
        (and
    (setq f (findfile "d:\\My Documents\\pts.txt"))
            (setq f (open f "r"))
        )
        (foreach pt_
(progn
            (while (setq l (read-line f))
                (setq pt (cons (read (strcat "( 10 " l ")")) pt))
            )
            (setq f (close f))
  pt
  )

      (entmake (list '(0 . "POINT") pt_)))
        )
    (princ)
)
I'm not talking about speed but memory usage (or is it one and the same?)
Does the longer the file stays open the more memory usage?

[Lee Mac]

Dividing file into pieces is not that important since Lee supported us with excellent one .

Did dividing the file work for you?

Worked perfectly without a piece of doubt .

Thank you so much Lee , you made my day .

Excellent 

Hacked together to divide the file:

Code: [Select]

(defun c:dividefile ( / fn i j l lines rf wf )

   .....)

Nice Lee.

It was only quickly written, but cheers pBe 

Question: Which approach eats up more memory,
Reading the file collect the list THEN entamke the point as opposed to entmake the point while reading the file?

Without any testing, I would have said that creating the points whilst iterating through the file would be both quicker and use less memory. Quicker for obvious reasons since the list of points is being iterated once instead of twice; and using less memory since no memory needs to be allocated to store the vast list of points in a variable; also, I should imagine there is a lot of memory usage as the list is being created since the list is constantly expanded and so more memory must be allocated.

Does the longer the file stays open the more memory usage?

I don't believe the 'length of time' will affect the memory usage of reading the file; I suspect you were referring to the number of processes that are occurring between reading each line of the file - in one code you are using memory to creating a list variable, in the other you are using memory to create Point Entities; neither will affect the memory allocated for reading the file.



/Speaking from a non-computer-science background.