Universal read_data function



On 05/27/2011 02:38 PM, Edwin Woollett wrote:
> On May 25, 2011, I wrote:
> ----------------------------------------
> The use of the string processing functions from
> van Nek's stringproc.lisp allows construction of
> a "Universal" read_data function which not only
> doesn't care about line endings, but doesn't care
> whether data separators are spaces or commas,
> and can handle fractions, decimals, floating point
> and strings as data items.
> ----------------------------------------------------
> Well, I was wrong, since the proposed universal
> read_data function chokes if a string contains
> spaces, such as "my mistake".
>
> To read in string data containing spaces, using
> the stringproc.lisp methods, requires (I think)
> the use of a comma separated data file.
>
> Hence a re-designed read_data function with
> the syntax:
>         read_data (filename, data_sep)
> in which the second arg is
>  1. optional, but if not present, space sep is assumed,
>  2. if present, must be either
>      "comma"  or "space" .
>
> ---------------------------------------
> Maxima 5.24.0 http://maxima.sourceforge.net
> using Lisp GNU Common Lisp (GCL) GCL 2.6.8 (a.k.a. GCL)
> Distributed under the GNU Public License. See the file COPYING.
> Dedicated to the memory of William Schelter.
> The function bug_report() provides bug reporting information.
>                                  2011-05-27
>
> (%i1) load(readwrite);
> (%o1) "c:/work2/readwrite.mac"
>
> (%i2) printfile("ndata1.dat")$
> 2.3e9 "Abc"
>
> (%i3) file_info ("ndata1.dat");
> (%o3) ["WINDOWS",13]
>
> (%i4) read_data ("ndata1.dat");
> (%o4) [[2.3E+9,"Abc"]]
>
> (%i5) read_data ("ndata1.dat","space");
> (%o5) [[2.3E+9,"Abc"]]
>
> /* note that the following data file has a string with
>    spaces:  5, "! = ", 120
>    there is a space before and after the equal sign */
>
> (%i6) printfile("tmp12.out")$
> 5, "! = ", 120
>
> (%i7) file_info ("tmp12.out");
> (%o7) ["UNIX",16]
>
> (%i8) read_data ("tmp12.out","comma");
> (%o8) [[5,"! = ",120]]
>
> (%i9) fundef (read_data);
>
> (%o9) read_data ([%v]) :=
>   block ([%s,%r,%l,%filename,%dsep],
>             %filename:part(%v,1),
>             if not stringp(%filename)
>                  then (disp(" filename must be a string 
> "),return(false)),
>             if not file_search(%filename)
>                  then (disp(" file not found "),return(false)),
>
>             if length(%v) = 1 then %dsep:"space" else %dsep:part(%v,2),
>
>             if lfreeof(["space","comma"],%dsep) then
>  (disp(" The allowed data-separators are the strings ``space'' or 
> ``comma''. "),
>                 disp(" The default file type is space separated data. "),
>                          return(false)),
>            %s:openr(%filename),
>           %r:[],
>
>            while (%l:readline(%s)) # false do
>                  if %dsep = "space"
>                        then %r:cons(map(parse_string,split(%l)),%r)
>                        else %r:cons(map(parse_string,split(%l,",")),%r),
>            close(%s),reverse(%r))
>
> -------------------------------------------------
>
> Ted Woollett
>
> _______________________________________________
> Maxima mailing list
> Maxima at math.utexas.edu
> http://www.math.utexas.edu/mailman/listinfo/maxima
>
Hi again Ted:

Would this be of any use to you?

read_data2(%filename) := block([%s, %r, %l, %lines, %chr],
   if not file_search(%filename) then
      (disp(" file not found "), return(false)),
   %s : openr(%filename),
   %l : "",
   %lines : [],
   while ( %r : ?read\-char(%s,false) ) #false do
   (
    %chr : cint(cunlisp(%r)),
    if %chr # 13 then
    (
      if %chr = 10 then
      (
        %lines : append(%lines, cons( strim(" ",%l) ,[]) ),
        %l : ""
      ) else
      %l : concat(%l,  %r)
    )
   ),
   disp(%lines),
   close(%s)
);

Data file contains (with embedded spaces, carriage returns, and newlines):
2.3e9 "Abc"
10.3
14.4
read_data1(%filename):=block([%s,%r],if notfile_search(%filename) then 
(disp(" file not found "),return(false)) 
,%s:openr(%filename),%r:[],while (l:readline(%s))#false do 
%r:cons(map(parse_string,split(l)),%r),reverse(%r))
12.6
14.5

Output from:
   read_data2("/home/pfb/ndata1.dat");
was:
["2.3e9 "Abc"","10.3","14.4",
"read_data1(%filename):=block([%s,%r],if notfile_search(%filename) then 
(disp(" file not found "),return(false)) 
,%s:openr(%filename),%r:[],while (l:readline(%s))#false do 
%r:cons(map(parse_string,split(l)),%r),reverse(%r))"
,"12.6","14.5"]

You may need to do more to work with the strings in the list because I 
don't know the specifics of your needs.

Paul