Characters in an ncurses app showing correctly only in tmux

Question

I have ported and cross-compiled the calcurse app to Blackberry 10 as part of the Berrymuch project (https://github.com/BerryFarm/berrymuch/tree/master/ports-wip/calcurse) however I am facing display issues with pseudo-graphical characters. When launched in the term48 terminal app (TERM=xterm-256color) the separators are displayed as letters. broken output

However if the same app is started in tmux the separators appear fine. ok output

I've tried running with TERM=screen in the terminal itself but that didn't solve the issue.

Does anyone have a theory where it might be going wrong?

I suspect either a) Buggy ncurses on bb10 b) Broken ncurses feature detection in the app c) Broken terminal feature detection d) locale information missing in the system

LC_ALL is set to en_US.UTF-8. Setting NCURSES_NO_UTF8_ACS=1 doesn't help either. ldd shows that calcurse is linked against libncursesw.so.1

Answer 1

I think you are on the right track with NCURSES_NO_UTF8_ACS - tmux will use UTF-8 for drawing the ACS by default if it thinks the terminal supports it (because your LANG contains UTF-8), so it sounds like your terminal supports UTF-8 but does not support ACS properly. Does the following work outside tmux (it should show symbols rather than ASCII):

tput enacs; tput smacs; tput acsc; tput rmacs; echo

Modern ncurses should also use UTF-8, I don't know why it is not. You might want to check what ncurses version you are using and perhaps try a newer one, there is also the U8 terminfo(5) capability, you could try a custom entry with that set to 0 or 1.

Answer 2

"term48" is a particular terminal emulator, which (see source code) sets TERM to "xterm-256color" (as usual, not a good idea).

In this case, you can see in the source-code that it does not implement line-drawing:

• SI/SO: Shift-In and Shift-Out (VT100)

/*
SO - SHIFT-OUT
Notation: (C0)
Representation: 00/14
SO is used for code extension purposes. It causes the meanings of the bit
combinations following it in the data stream to be changed.
The use of SO is defined in Standard ECMA-35.
NOTE SO is used in 7-bit environments only; in 8-bit environments LOCKING-SHIFT
ONE (LS1) is used instead.
*/
void ecma48_SO(){
  ecma48_NOT_IMPLEMENTED("SO");
}

/*
SI - SHIFT-IN
Notation: (C0)
Representation: 00/15
SI is used for code extension purposes. It causes the meanings of the bit
combinations following it in the data stream to be changed.
The use of SI is defined in Standard ECMA-35.
NOTE SI is used in 7-bit environments only; in 8-bit environments LOCKING-SHIFT
ZERO (LS0) is used instead.
*/
void ecma48_SI(){
  ecma48_NOT_IMPLEMENTED("SI");
}

• SCS (VT220 and later)

      case ECMA48_STATE_ANSI_SCS:                                               
        switch(tbuf[i]){                                                        
          default: ecma48_NOT_IMPLEMENTED("SCS");                               
        }; break;

Without those features, terminal applications will have only UTF-8 to work with for line-drawing.

ncurses does have a workaround, but it relies upon the application calling setlocale. The berrymuch port disables that feature (see build.sh):

CONFIGURE_CMD="./configure
                --host=$PBHOSTARCH
                --build=$PBBUILDARCH
                --target=$PBTARGETARCH
                --prefix=$PREFIX
                --disable-nls
                CC=$PBTARGETARCH-gcc"

And because of that, it provides no way for using UTF-8 in ncurses, either.