Topic: [Help Me]How to deal with these data？

[myloveflyer]

Because the job needs, wrote a modification of data files, don't know why output data format is not right after, each data corresponding position change, how to write can do and the original data format exactly the same?
Please help me,Thank!

Code: [Select]

(defun c:test()
(arxload "geomcal.arx")
(setq date_list nil)
(princ (if (setq f (open "d:/old.DAT" "r")) "Data file open success!\n" "Data files can't open!\n"))
(while  (setq str_line (read-line f))
(setq str_line (strcat "(" str_line ")"))
(setq list_line (read str_line))
(if (= (length list_line) 10)
(setq list_line (append (list (nth 0 list_line) (nth 1 list_line) "  ") (cddr list_line)))
)
(setq date_list (append date_list (list list_line)))
)
(close f)
(setq n (length date_list))
(setq wt_list (nth (- n 1) date_list))
(setq old_wt (nth 2 wt_list))
(setq new_wt (* old_wt 1.05))
(setq wt old_wt)
(setq min_wt (* old_wt 0.001))
(setq i 1)
(while (> (abs (- wt new_wt)) min_wt)
(setq minus_list (nth i date_list))
(setq minus_n (nth 5 minus_list))
(if (> minus_n 1)
(progn
(setq add_list (nth (+ i 1) date_list))
(setq minus_wt (nth 9 minus_list))
(setq add_wt (nth 9 add_list))
(setq wt (cal "wt - minus_wt + add_wt"))
(setq minus_list
(list
(nth 0 minus_list)
(nth 1 minus_list)
(nth 2 minus_list)
(nth 3 minus_list)
(nth 4 minus_list)
(- minus_n 1)
(nth 6 minus_list)
(nth 7 minus_list)
(nth 8 minus_list)
minus_wt
(- (nth 10 minus_list) minus_wt)
)
)
(setq date_list (subst minus_list (nth i date_list) date_list))
(setq add_list
(list
(nth 0 add_list)
(nth 1 add_list)
(nth 2 add_list)
(nth 3 add_list)
(nth 4 add_list)
(+ (nth 5 add_list) 1)
(nth 6 add_list)
(nth 7 add_list)
(nth 8 add_list)
add_wt
(+ (nth 10 add_list) add_wt)
)
)
(setq date_list (subst add_list (nth (+ i 1) date_list) date_list))
)
)
(setq i (+ i 1))
(if (= i (- n 2))
(setq i 1)
)
)
(setq date_list (subst (subst wt old_wt wt_list) wt_list date_list))
(princ (if (setq f (open "d:/new.DAT" "w")) "Data file open success!\n" "Data files can't open!\n"))
(foreach line date_list
(progn
(foreach num line
(progn
(princ num f)
(princ "\t" f)
)
)
(princ "\n" f)
)
)
(close f)
(princ)
)



[myloveflyer]

My aim is to make the output file ( new.dat ) format and the original document ( old.dat ) format is the same as。

[Keith™]

The problem you have is that the original file ( old.dat ) has spaces between the columns and the new file ( new.dat ) has tabs. Also, the original data is mixed justification. some columns are right justified, while others are left justified.

The old.dat file contains eleven fields. Each field is separated by two spaces and the data in the field is padded with spaces depending upon justification.

Characters per field (in order): 4-4-12-4-4-4-3-5-10-6-7

The record is 83 characters long.

Justification per field (in order): r-r-l-l-l-r-l-l-r-r-r

So, when you output each field to the new file, do this:

Pad spaces on the left for right aligned text and pad spaces on the right for left aligned text, then put an additional two spaces between each output.

This is a simple padding function to make the text align and fit the field as needed. I am sure it could be tweaked, but it works as advertised.

Code: [Select]

;;;By: K. Blackie 02/05/2012
;;;Function: Pad
;;;Notes:     Pad text with specified character(s) until the text is as long as the passed length.
;;;              Justification is handled by the alignment parameter. acAlignmentLeft, acAlignmentRight
;;;              and acAlignmentMiddle are the options, otherwise the function returns the passed value
;;;              unchanged. The bit parameter is used only for middle justification. If T then the first
;;;              padded character is appended, if nil the first character is prepended.
;;;
;;;Usage:    (pad "text" integer "character" alignment nil)
;;;Examples:
;;;              (pad "73.453" 8 "0" acAlignmentLeft nil) returns "0073.453"
;;;              (pad "73.453" 8 "0" acAlignmentRight nil) returns "73.45300"
;;;              (pad "73.453" 8 "0" acAlignmentMiddle nil) returns "073.4530"


(defun pad (value len char alignment bit)
  (if (< (strlen value) len)
    (cond
      ((= alignment acAlignmentLeft)
       (setq value (pad (strcat char value) len char alignment nil))
      )
      ((= alignment acAlignmentRight)
       (setq value (pad (strcat value char) len char alignment nil))
      )
      ((= alignment acAlignmentMiddle)
       (if (= bit t)
(setq value (pad (strcat value char) len char alignment nil))
(setq value (pad (strcat char value) len char alignment t))
       )
      )
    )
  )
  value
)

[BlackBox]

Building on Keith's post, I would test for the ARX application being found in (arx) prior to subsequent Arxloads, which is not necessary, and may cause issues.

[myloveflyer]

Keith, thank you, can you help me to revise my program？
Due to the presence of large amounts of data file is to be processed, so the trouble you help me deal or help me to arrange a program to complete the data processing.

[myloveflyer]

Because the job needs, wrote a modification of data files, don't know why output data format is not right after, each data corresponding position change, how to write can do and the original data format exactly the same?
Please help me,Thank!

Code: [Select]

(defun c:test()
(arxload "geomcal.arx")
(setq date_list nil)
(princ (if (setq f (open "d:/old.DAT" "r")) "Data file open success!\n" "Data files can't open!\n"))
(while  (setq str_line (read-line f))
(setq str_line (strcat "(" str_line ")"))
(setq list_line (read str_line))
(if (= (length list_line) 10)
(setq list_line (append (list (nth 0 list_line) (nth 1 list_line) "  ") (cddr list_line)))
)
(setq date_list (append date_list (list list_line)))
)
(close f)
(setq n (length date_list))
(setq wt_list (nth (- n 1) date_list))
(setq old_wt (nth 2 wt_list))
(setq new_wt (* old_wt 1.05))
(setq wt old_wt)
(setq min_wt (* old_wt 0.001))
(setq i 1)
(while (> (abs (- wt new_wt)) min_wt)
(setq minus_list (nth i date_list))
(setq minus_n (nth 5 minus_list))
(if (> minus_n 1)
(progn
(setq add_list (nth (+ i 1) date_list))
(setq minus_wt (nth 9 minus_list))
(setq add_wt (nth 9 add_list))
(setq wt (cal "wt - minus_wt + add_wt"))
(setq minus_list
(list
(nth 0 minus_list)
(nth 1 minus_list)
(nth 2 minus_list)
(nth 3 minus_list)
(nth 4 minus_list)
(- minus_n 1)
(nth 6 minus_list)
(nth 7 minus_list)
(nth 8 minus_list)
minus_wt
(- (nth 10 minus_list) minus_wt)
)
)
(setq date_list (subst minus_list (nth i date_list) date_list))
(setq add_list
(list
(nth 0 add_list)
(nth 1 add_list)
(nth 2 add_list)
(nth 3 add_list)
(nth 4 add_list)
(+ (nth 5 add_list) 1)
(nth 6 add_list)
(nth 7 add_list)
(nth 8 add_list)
add_wt
(+ (nth 10 add_list) add_wt)
)
)
(setq date_list (subst add_list (nth (+ i 1) date_list) date_list))
)
)
(setq i (+ i 1))
(if (= i (- n 2))
(setq i 1)
)
)
(setq date_list (subst (subst wt old_wt wt_list) wt_list date_list))
(princ (if (setq f (open "d:/new.DAT" "w")) "Data file open success!\n" "Data files can't open!\n"))
(foreach line date_list
(progn
(foreach num line
(progn
(princ num f)
(princ "\t" f)
)
)
(princ "\n" f)
)
)
(close f)
(princ)
)



Who helps me to look at, thank you！

[CAB]

See if this will work for you.

Code: [Select]

(defun c:cnvdata (/ alllines fn oldfname newfname ln x newlines)
  (setq oldfname "d:/old.DAT")
  (setq newfname "d:/new.DAT")

  (defun sparsers (str delim / ptr lst) ; CAB
    (while (setq ptr (vl-string-search delim str))
      (setq lst (cons (substr str 1 ptr) lst))
      (setq str (substr str (+ ptr 2)))
    )
    (reverse (cons str lst))
  )

  ;;  Combine strings with , seperator - CAB
  (defun Str-Make (lst del / str)
    (mapcar (function (lambda(s) (setq str (if str (strcat str del s) s)))) lst)
    str
  )

  (if (and (setq fn (findfile oldfname))
           (setq fn (open fn "r"))
      )
    (progn
      (princ "\nData file open success!\n")
      (while (setq ln (read-line fn)) ;  read all the lines in the file
        (setq alllines (cons ln alllines))
      )
      (close fn) ; close the open file handle

      (foreach ln alllines
        (setq lst (vl-remove "" (sparsers ln " ")))
        (cond
          ((= (length lst) 10)
           (setq ln (str-make (append (list (car lst)(cadr lst) "  ") (cddr lst)) (chr 9)))
           (setq newlines (cons ln newlines))
          )
          (t
           (setq ln (str-make lst (chr 9)))
           (setq newlines (cons ln newlines))
          )
        )
      )

      (setq fn (open newfname "w")) ; open file for writing
      (foreach ln Newlines
        (write-line ln fn)
      )
      (close fn) ; close the open file handle


    ) ; progn
    (princ "\nData files can't open!\n")
  ) ; Endif
) ; end defun

[myloveflyer]

CAB，Thank you!
Output file format and the original file format is different, and my program on the original document sixth columns were processed, you write a program like no processing.

[CAB]

I'm out of the office until 12:00 here. I'll look at it again.
Yes I only showed how the formatting could be converted.

Could you explain how the data is being changed? Otherwise I will have to dig into the code when I get back.
You have columns 1 through 11 and you change column 3 when not blank.
When column 3 is blank you alter column 6 using column 11 and maybe column 9.

I'll have to deal with this when I return.

BTW, the variable add_list is a protected symbol and should not be used as a variable.

[myloveflyer]

Thank you, CAB, thank you in his busy schedule to help me solve the problem.

My idea is: the whole data in a total of11 columns, changes in the column is sixth column and the eleventh column, let old data in the sixth column of the number of random changes, then the random changes after the number is multiplied by tenth row number, finally get the eleventh row number, finally let eleventh row number all together get the last line of the last digit, sixth columns of the sum of the number of total should and the original final line second the same number, finally is the new get the last line of the last digit should be larger than the original data of the last line of the last digit ( this is larger than the range can be set a digital ), my English is not very good, if I express is not very clear, you can run down my program, I think may be of help to you.

CAB, actually these in my program has been reflected, the only problem is that my program in the output format problems, you can also modify my program, join your output function, estimation can reach my goal.

[CAB]

Maybe this?

Code: [Select]

(defun c:cnvdata (/ data_list new_list fn oldfname newfname ln x newlist
                  ADDLIST ADD_WT DEL I LST MINUS_LIST MINUS_N MINUS_WT MIN_WT
                  N NEW_WT OLD_WT RAW_LIST STR WT WT_LIST X)
  (setq oldfname "d:/old.DAT")
  (setq newfname "d:/new.DAT")


  (if (and (setq fn (findfile oldfname))
           (setq fn (open fn "r"))
      )
    (progn
      (princ "\nData file open success!\n")
      (while (setq ln (read-line fn)) ;  read all the lines in the file
        (setq raw_list (cons ln raw_list))
      )
      (close fn) ; close the open file handle


  (foreach str_line raw_list
    (setq lst (read (strcat "(" str_line ")")))
    (if (= (length lst) 10)
      (setq lst (append (list (car lst) (cadr lst) "  ") (cddr lst)))
    )
    (setq data_list (cons lst data_list))
  )


      ;;====================================================
      ;;  Process the data
 
  (setq n (length data_list))
  (setq wt_list (nth (- n 1) data_list))
  (setq old_wt (nth 2 wt_list))
  (setq new_wt (* old_wt 1.05))
  (setq wt old_wt)
  (setq min_wt (* old_wt 0.001))
  (setq i 1)

 
  (while (> (abs (- wt new_wt)) min_wt)
    (setq minus_list (nth i data_list))
    (setq minus_n (nth 5 minus_list))
    (if (> minus_n 1)
      (progn
        (setq addlist (nth (+ i 1) data_list))
        (setq minus_wt (nth 9 minus_list))
        (setq add_wt (nth 9 addlist))
        (setq wt (cal "wt - minus_wt + add_wt"))
        (setq minus_list
               (list
                 (nth 0 minus_list)
                 (nth 1 minus_list)
                 (nth 2 minus_list)
                 (nth 3 minus_list)
                 (nth 4 minus_list)
                 (- minus_n 1)
                 (nth 6 minus_list)
                 (nth 7 minus_list)
                 (nth 8 minus_list)
                 minus_wt
                 (- (nth 10 minus_list) minus_wt)
               )
        )
        (setq data_list (subst minus_list (nth i data_list) data_list))
        (setq addlist
               (list
                 (nth 0 addlist)
                 (nth 1 addlist)
                 (nth 2 addlist)
                 (nth 3 addlist)
                 (nth 4 addlist)
                 (+ (nth 5 addlist) 1)
                 (nth 6 addlist)
                 (nth 7 addlist)
                 (nth 8 addlist)
                 add_wt
                 (+ (nth 10 addlist) add_wt)
               )
        )
        (setq data_list (subst addlist (nth (+ i 1) data_list) data_list))
      )
    )
    (setq i (+ i 1))
    (if (= i (- n 2))
      (setq i 1)
    )
  )
  (setq data_list (subst (subst wt old_wt wt_list) wt_list data_list))
      ;;====================================================

     
      (setq fn (open newfname "w")) ; open file for writing
      (foreach ln data_list ; (reverse Newlist)
        (mapcar'(lambda(x)(princ x fn)(princ "\t" fn)) ln)
        (princ "\n" fn)
      )
      (close fn) ; close the open file handle

    ) ; progn
    (princ "\nData files can't open!\n")
  ) ; Endif
) ; end defun

[myloveflyer]

CAB，Program output still exist, estimates are text alignment 's sake, there is my original procedures in the output file ( for example:48x3.5, output becomes 48x3, why decimal output have a problem? ) This procedure, you and I made the same mistake, because the program output file formats must be and the original format, so there is every column of text alignment problem, hope there is a way to solve this problem, thank you！

[CAB]

Another try.

Code: [Select]

(defun c:cnvdata (/ alllines fn oldfname newfname ln x newlines
                  ADDLIST ADD_WT DATA_LIST F I LST MINUS_LIST MINUS_N
                  MINUS_WT MIN_WT N NEW_WT OLD_WT WT WT_LIST)
  (setq oldfname "d:/old.DAT")
  (setq newfname "d:/new.DAT")

  (defun sparsers (str delim / ptr lst) ; CAB
    (while (setq ptr (vl-string-search delim str))
      (setq lst (cons (substr str 1 ptr) lst))
      (setq str (substr str (+ ptr 2)))
    )
    (reverse (cons str lst))
  )

  (defun MakeNum (lst) ; convert items 5, 9, 10 to numbers
    (list
      (car lst)
      (cadr lst)
      (caddr lst)
      (cadddr lst)
      (nth 4 lst)
      (atoi (nth 5 lst))
      (nth 6 lst)
      (nth 7 lst)
      (nth 8 lst)
      (atof (nth 9 lst))
      (atof (nth 10 lst))
    )
  )

 
  (if (and (setq fn (findfile oldfname))
           (setq fn (open fn "r"))
      )
    (progn
      (princ "\nData file open success!\n")
      (while (setq ln (read-line fn)) ;  read all the lines in the file
        (setq alllines (cons ln alllines))
      )
      (close fn) ; close the open file handle

      (foreach ln alllines
        (setq lst (vl-remove "" (sparsers ln " ")))
        (cond
          ((= (length lst) 10) ; add 3rd item
           (setq data_list (cons (makeNum (append (list (car lst)(cadr lst) "  ") (cddr lst))) data_list))
          )
          ((= (length lst) 3)(setq data_list (cons lst data_list)))
          ((setq data_list (cons (makeNum lst) data_list)))
        )
      )


     
  (setq n (length data_list))
  (setq wt_list (last data_list)) ; get last record
  (setq old_wt (atof (nth 2 wt_list)))   ; get 3rd item
  (setq new_wt (* old_wt 1.05))
  (setq wt old_wt)
  (setq min_wt (* old_wt 0.001))
  (setq i 1)

 
  (while (> (abs (- wt new_wt)) min_wt)
    (setq minus_list (nth i data_list))
    (setq minus_n (nth 5 minus_list))
    (if (> minus_n 1)
      (progn
        (setq addlist (nth (1+ i) data_list))
        (setq minus_wt (nth 9 minus_list))
        (setq add_wt (nth 9 addlist))
        (setq wt (+ (- wt minus_wt) add_wt))  ; no cal function needed
        (setq minus_list
               (list
                 (nth 0 minus_list)
                 (nth 1 minus_list)
                 (nth 2 minus_list)
                 (nth 3 minus_list)
                 (nth 4 minus_list)
                 (- minus_n 1)
                 (nth 6 minus_list)
                 (nth 7 minus_list)
                 (nth 8 minus_list)
                 minus_wt
                 (- (nth 10 minus_list) minus_wt)
               )
        )
        (setq data_list (subst minus_list (nth i data_list) data_list))
        (setq addlist
               (list
                 (nth 0 addlist)
                 (nth 1 addlist)
                 (nth 2 addlist)
                 (nth 3 addlist)
                 (nth 4 addlist)
                 (1+ (nth 5 addlist))
                 (nth 6 addlist)
                 (nth 7 addlist)
                 (nth 8 addlist)
                 add_wt
                 (+ (nth 10 addlist) add_wt)
               )
        )
        (setq data_list (subst addlist (nth (1+ i) data_list) data_list))
      )
    )
    (setq i (1+ i))
    (if (= i (- n 2))
      (setq i 1)
    )
  )
  (setq data_list (subst (subst wt old_wt wt_list) wt_list data_list))


     
  (princ (if (setq f (open "d:/new.DAT" "w"))
           "Data file open success!\n"
           "Data files can't open!\n"
         )
  )
  (foreach line data_list
    (foreach num line
      (princ num f)(princ "\t" f)
    )
    (princ "\n" f)
  )
  (close f)

    ) ; progn
    (princ "\nData files can't open!\n")
  ) ; Endif
) ; end defun

[myloveflyer]

CAB, program output also has problem, see my screenshots, and there is no space in front of data first column, the output of data the last line of the last digit of the total amount wrong (should be better than the original data total amount big), I think it should write a character aligned son function, according to the original data file each character of the length of the output, do not know such output data and original treatment is consistent.

[CAB]

I installed your conversion code exactly to correct data error.
I added new output routine for formatting.

Code: [Select]

(defun c:cnvdata (/ alllines fn oldfname newfname ln x newlines
                  ADDLIST ADD_WT DATA_LIST F I LST MINUS_LIST MINUS_N
                  MINUS_WT MIN_WT N NEW_WT OLD_WT WT WT_LIST)
  (setq oldfname "d:/old.DAT")
  (setq newfname "d:/new.DAT")

  (defun sparsers (str delim / ptr lst) ; CAB
    (while (setq ptr (vl-string-search delim str))
      (setq lst (cons (substr str 1 ptr) lst))
      (setq str (substr str (+ ptr 2)))
    )
    (reverse (cons str lst))
  )

 
  ;;  CAB 02/09/06
  (defun pad (word len / spaces)
    (repeat (- len (strlen word)) (setq spaces (cons 32 spaces)))
    (strcat word (vl-list->string spaces))
  )

  ;;  CAB 02/09/06
  (defun padL (word len / spaces)
    (repeat (- len (strlen word)) (setq spaces (cons 32 spaces)))
    (strcat (vl-list->string spaces) word)
  )
 
  (defun MakeNum (lst) ; convert items 5, 9, 10 to numbers
    (list
      (car lst)
      (cadr lst)
      (caddr lst)
      (cadddr lst)
      (nth 4 lst)
      (atoi (nth 5 lst))
      (nth 6 lst)
      (nth 7 lst)
      (nth 8 lst)
      (atof (nth 9 lst))
      (atof (nth 10 lst))
    )
  )

 
  (if (and (setq fn (findfile oldfname))
           (setq fn (open fn "r"))
      )
    (progn
      (princ "\nData file open success!\n")
      (while (setq ln (read-line fn)) ;  read all the lines in the file
        (setq alllines (cons ln alllines))
      )
      (close fn) ; close the open file handle

      (foreach ln alllines
        (setq lst (vl-remove "" (sparsers ln " ")))
        (cond
          ((= (length lst) 10) ; add 3rd item
           (setq data_list (cons (makeNum (append (list (car lst)(cadr lst) "  ") (cddr lst))) data_list))
          )
          ((= (length lst) 3)(setq data_list (cons lst data_list)))
          ((setq data_list (cons (makeNum lst) data_list)))
        )
      )



 (setq n (length data_list))
  (setq wt_list (nth (- n 1) data_list))
  (setq old_wt (atof (nth 2 wt_list)))   ; get 3rd item
  (setq new_wt (* old_wt 1.05))
  (setq wt old_wt)
  (setq min_wt (* old_wt 0.001))
  (setq i 1)

 
  (while (> (abs (- wt new_wt)) min_wt)
    (setq minus_list (nth i data_list))
    (setq minus_n (nth 5 minus_list))
    (if (> minus_n 1)
      (progn
        (setq addlist (nth (+ i 1) data_list))
        (setq minus_wt (nth 9 minus_list))
        (setq add_wt (nth 9 addlist))
        (setq wt (cal "wt - minus_wt + add_wt"))
        (setq minus_list
               (list
                 (nth 0 minus_list)
                 (nth 1 minus_list)
                 (nth 2 minus_list)
                 (nth 3 minus_list)
                 (nth 4 minus_list)
                 (- minus_n 1)
                 (nth 6 minus_list)
                 (nth 7 minus_list)
                 (nth 8 minus_list)
                 minus_wt
                 (- (nth 10 minus_list) minus_wt)
               )
        )
        (setq data_list (subst minus_list (nth i data_list) data_list))
        (setq addlist
               (list
                 (nth 0 addlist)
                 (nth 1 addlist)
                 (nth 2 addlist)
                 (nth 3 addlist)
                 (nth 4 addlist)
                 (+ (nth 5 addlist) 1)
                 (nth 6 addlist)
                 (nth 7 addlist)
                 (nth 8 addlist)
                 add_wt
                 (+ (nth 10 addlist) add_wt)
               )
        )
        (setq data_list (subst addlist (nth (+ i 1) data_list) data_list))
      )
    )
    (setq i (+ i 1))
    (if (= i (- n 2))
      (setq i 1)
    )
  )
  (setq data_list (subst (subst wt old_wt wt_list) wt_list data_list))
     
  (princ (if (setq fn (open "d:/new.DAT" "w"))
           "Data file open success!\n"
           "Data files can't open!\n"
         )
  )
  (foreach record data_list
    (cond
      ((= (length record) 11)
    (princ (padL (car record) 4) fn)
    (princ "    " fn)
    (princ (pad (cadr record) 7) fn)
    (princ (pad (caddr record) 11) fn)
    (princ (cadddr record) fn)
    (princ "  " fn)
    (princ (nth 4 record) fn)
    (princ (padL (itoa(nth 5 record)) 6) fn)
    (princ "  " fn)
    (princ (nth 6 record) fn)
    (princ "  " fn)
    (princ (nth 7 record) fn)
    (princ "    " fn)
    (princ (pad (nth 8 record) 10) fn)
    (princ (padL (rtos(nth 9 record)2 2) 7) fn)
    (princ (padL (rtos(nth 10 record)2 1) 9) fn)
    )
    ((princ (padL (car record) 2) fn)
     (princ (padL (cadr record) 40) fn)
     (princ (padL (caddr record) 41) fn)
     )
      )
    (princ "\n" fn)
  )
  (close fn)

    ) ; progn
    (princ "\nData files can't open!\n")
  ) ; Endif
) ; end defun