foreign language patch for build-index+cl-ppcre branch

On 2/24/11 7:26 PM, Leo Butler wrote:
>
> On Thu, 24 Feb 2011, Raymond Toy wrote:
>
> < On 2/24/11 4:10 PM, Leo Butler wrote:
> < >  I've pushed a patch that fixes this issue for 
> < >  sbcl, clisp and cmucl (debian testing). All the
> < >  maxima-index.lisp files are made, and I don't see
> < >  any non-printable characters.
> < >
> < Mostly works for me with cmucl, but I had to apply the following patch,
> < basically adding with-standard-io-syntax, which is a good idea for all
> < Lisps.  For some reason I have non-NIL values for *print-length* and
> < *print-level*, so maxima-index.lisp had ... characters in there.
>
> Ok, this is good to know. I will patch the slurp function, too.
Possibly.  I have not looked at the slurp function.
> < 
> < One other thing that needs to be solved for cmucl is that when I used
> < LANG=de_DE, maxima somehow decides to use utf8 encoding.  That's fine,
> < but when the documentation is printed, cmucl uses it's default encoding
> < of iso8859-1, so ? is used in quite a few places.  For example, ?
> < additive gives question marks in various places.
>
> This is built from doc/info/de/maxima.info, correct? Could you check
> what this file says on the line
>
> coding: ...
>
> near the bottom? The coding should be iso-8859-1 and it should be
> picked up by Maxima (it is for me).
It does say iso-8859-1.
> I have chosen the default external format to be utf-8, but the default
> could be set at build time, too.
Ah, but default for whom?  There are many defaults.  I have not studied
your code, so I don't know what default external format you are talking
about.  I'll need to see how you write out the documentation and get
back to you on this.  I was just hacking something to make the output
look correct.
> < 
> < I'll need to look around to see where to hook  these together.
> < 
> < If it still matters, I don't think gcl has any kind of external format
> < support.  
> < 
> < In many ways, it would be good to treat the info files as binary files
> < of octets and use octet offsets.  Then when we read and display the
> < documentation, we don't need any special support.  Just output the
> < octets assuming the user has done the right magic so that the
> < terminal/wxmaxima/etc. have the correct encoding.  (The terminal setting
> < has to be set correctly no matter whether we use octets or characters.)
>  
>  Yes, this was my preconception, too. We do not need to use an encoding
>  to read, categorise or search the info files. *But* when it comes to
>  to outputting the stuff, I think you are wrong: the de.utf8 online
>  help is displayed correctly in emacs when output as a utf8 encoded
>  string, but not without the encoding.
Are you using maxima in a shell window in emacs?   There are many places
where the octets can be misinterpreted, anywhere from maxima reading the
info file to emacs reading the octets.
>  
>  
>  I can patch build-index.lisp so
>  that you can compare the results with encodings vs. octets.
Only if you think this is a worthwhile choice.  It could solve the issue
of everyone's favorite lisp using different ways of specifying the
external format.

Ray