Why does iconv behave differently on php8.0 than php8.1?

113 Views Asked by At

We have the problem that on PHP8.1 iconv does not transliterate the long dash ("–") to a minus ("-").

This is how I am reproducing it:

$ cat iconv.php 
<?php
phpinfo();
echo iconv('UTF-8', 'ASCII//TRANSLIT', '–');

$ php8.0 iconv.php > /tmp/8.0.out 2>&1
$ php8.1 iconv.php > /tmp/8.1.out 2>&1
$ diff /tmp/8.0.out /tmp/8.1.out 
2c2
< PHP Version => 8.0.28
---
> PHP Version => 8.1.18
5c5
< Build Date => Feb 14 2023 18:32:25
---
> Build Date => Apr 14 2023 04:39:07
9,52c9,52
< Configuration File (php.ini) Path => /etc/php/8.0/cli
< Loaded Configuration File => /etc/php/8.0/cli/php.ini
< Scan this dir for additional .ini files => /etc/php/8.0/cli/conf.d
< Additional .ini files parsed => /etc/php/8.0/cli/conf.d/10-mysqlnd.ini,
< /etc/php/8.0/cli/conf.d/10-opcache.ini,
< /etc/php/8.0/cli/conf.d/10-pdo.ini,
< /etc/php/8.0/cli/conf.d/15-xml.ini,
< /etc/php/8.0/cli/conf.d/20-bcmath.ini,
< /etc/php/8.0/cli/conf.d/20-bz2.ini,
< /etc/php/8.0/cli/conf.d/20-calendar.ini,
< /etc/php/8.0/cli/conf.d/20-ctype.ini,
< /etc/php/8.0/cli/conf.d/20-curl.ini,
< /etc/php/8.0/cli/conf.d/20-dom.ini,
< /etc/php/8.0/cli/conf.d/20-exif.ini,
< /etc/php/8.0/cli/conf.d/20-ffi.ini,
< /etc/php/8.0/cli/conf.d/20-fileinfo.ini,
< /etc/php/8.0/cli/conf.d/20-ftp.ini,
< /etc/php/8.0/cli/conf.d/20-gd.ini,
< /etc/php/8.0/cli/conf.d/20-gettext.ini,
< /etc/php/8.0/cli/conf.d/20-iconv.ini,
< /etc/php/8.0/cli/conf.d/20-mbstring.ini,
< /etc/php/8.0/cli/conf.d/20-mysqli.ini,
< /etc/php/8.0/cli/conf.d/20-pdo_mysql.ini,
< /etc/php/8.0/cli/conf.d/20-phar.ini,
< /etc/php/8.0/cli/conf.d/20-posix.ini,
< /etc/php/8.0/cli/conf.d/20-readline.ini,
< /etc/php/8.0/cli/conf.d/20-shmop.ini,
< /etc/php/8.0/cli/conf.d/20-simplexml.ini,
< /etc/php/8.0/cli/conf.d/20-sockets.ini,
< /etc/php/8.0/cli/conf.d/20-sysvmsg.ini,
< /etc/php/8.0/cli/conf.d/20-sysvsem.ini,
< /etc/php/8.0/cli/conf.d/20-sysvshm.ini,
< /etc/php/8.0/cli/conf.d/20-tokenizer.ini,
< /etc/php/8.0/cli/conf.d/20-xmlreader.ini,
< /etc/php/8.0/cli/conf.d/20-xmlwriter.ini,
< /etc/php/8.0/cli/conf.d/20-xsl.ini,
< /etc/php/8.0/cli/conf.d/20-zip.ini,
< /etc/php/8.0/cli/conf.d/90-blackfire.ini
< 
< PHP API => 20200930
< PHP Extension => 20200930
< Zend Extension => 420200930
< Zend Extension Build => API420200930,NTS
< PHP Extension Build => API20200930,NTS
---
> Configuration File (php.ini) Path => /etc/php/8.1/cli
> Loaded Configuration File => /etc/php/8.1/cli/php.ini
> Scan this dir for additional .ini files => /etc/php/8.1/cli/conf.d
> Additional .ini files parsed => /etc/php/8.1/cli/conf.d/10-mysqlnd.ini,
> /etc/php/8.1/cli/conf.d/10-opcache.ini,
> /etc/php/8.1/cli/conf.d/10-pdo.ini,
> /etc/php/8.1/cli/conf.d/15-xml.ini,
> /etc/php/8.1/cli/conf.d/20-bcmath.ini,
> /etc/php/8.1/cli/conf.d/20-bz2.ini,
> /etc/php/8.1/cli/conf.d/20-calendar.ini,
> /etc/php/8.1/cli/conf.d/20-ctype.ini,
> /etc/php/8.1/cli/conf.d/20-curl.ini,
> /etc/php/8.1/cli/conf.d/20-dom.ini,
> /etc/php/8.1/cli/conf.d/20-exif.ini,
> /etc/php/8.1/cli/conf.d/20-ffi.ini,
> /etc/php/8.1/cli/conf.d/20-fileinfo.ini,
> /etc/php/8.1/cli/conf.d/20-ftp.ini,
> /etc/php/8.1/cli/conf.d/20-gd.ini,
> /etc/php/8.1/cli/conf.d/20-gettext.ini,
> /etc/php/8.1/cli/conf.d/20-iconv.ini,
> /etc/php/8.1/cli/conf.d/20-mbstring.ini,
> /etc/php/8.1/cli/conf.d/20-mysqli.ini,
> /etc/php/8.1/cli/conf.d/20-pdo_mysql.ini,
> /etc/php/8.1/cli/conf.d/20-phar.ini,
> /etc/php/8.1/cli/conf.d/20-posix.ini,
> /etc/php/8.1/cli/conf.d/20-readline.ini,
> /etc/php/8.1/cli/conf.d/20-shmop.ini,
> /etc/php/8.1/cli/conf.d/20-simplexml.ini,
> /etc/php/8.1/cli/conf.d/20-sockets.ini,
> /etc/php/8.1/cli/conf.d/20-sysvmsg.ini,
> /etc/php/8.1/cli/conf.d/20-sysvsem.ini,
> /etc/php/8.1/cli/conf.d/20-sysvshm.ini,
> /etc/php/8.1/cli/conf.d/20-tokenizer.ini,
> /etc/php/8.1/cli/conf.d/20-xmlreader.ini,
> /etc/php/8.1/cli/conf.d/20-xmlwriter.ini,
> /etc/php/8.1/cli/conf.d/20-xsl.ini,
> /etc/php/8.1/cli/conf.d/20-zip.ini,
> /etc/php/8.1/cli/conf.d/90-blackfire.ini
> 
> PHP API => 20210902
> PHP Extension => 20210902
> Zend Extension => 420210902
> Zend Extension Build => API420210902,NTS
> PHP Extension Build => API20210902,NTS
57a58
> Zend Max Execution Timers => disabled
66,68c67,69
< Zend Engine v4.0.28, Copyright (c) Zend Technologies
<     with Zend OPcache v8.0.28, Copyright (c), by Zend Technologies
<     with blackfire v1.86.8~linux-x64-non_zts80, https://blackfire.io, by Blackfire
---
> Zend Engine v4.1.18, Copyright (c) Zend Technologies
>     with Zend OPcache v8.1.18, Copyright (c), by Zend Technologies
>     with blackfire v1.86.8~linux-x64-non_zts81, https://blackfire.io, by Blackfire
86c87
< Blackfire => 1.86.8~linux-x64-non_zts80
---
> Blackfire => 1.86.8~linux-x64-non_zts81
137c138
< PHP Version => 8.0.28
---
> PHP Version => 8.1.18
164c165,166
< extension_dir => /usr/lib/php/20200930 => /usr/lib/php/20200930
---
> extension_dir => /usr/lib/php/20210902 => /usr/lib/php/20210902
> fiber.stack_size => no value => no value
181d182
< log_errors_max_len => 1024 => 1024
276c277
< timelib version => 2020.03
---
> timelib version => 2021.19
326c327
< libmagic => 539
---
> libmagic => 540
356a358
> AVIF Support => enabled
369c371
< Hashing Engines => md2 md4 md5 sha1 sha224 sha256 sha384 sha512/224 sha512/256 sha512 sha3-224 sha3-256 sha3-384 sha3-512 ripemd128 ripemd160 ripemd256 ripemd320 whirlpool tiger128,3 tiger160,3 tiger192,3 tiger128,4 tiger160,4 tiger192,4 snefru snefru256 gost gost-crypto adler32 crc32 crc32b crc32c fnv132 fnv1a32 fnv164 fnv1a64 joaat haval128,3 haval160,3 haval192,3 haval224,3 haval256,3 haval128,4 haval160,4 haval192,4 haval224,4 haval256,4 haval128,5 haval160,5 haval192,5 haval224,5 haval256,5 
---
> Hashing Engines => md2 md4 md5 sha1 sha224 sha256 sha384 sha512/224 sha512/256 sha512 sha3-224 sha3-256 sha3-384 sha3-512 ripemd128 ripemd160 ripemd256 ripemd320 whirlpool tiger128,3 tiger160,3 tiger192,3 tiger128,4 tiger160,4 tiger192,4 snefru snefru256 gost gost-crypto adler32 crc32 crc32b crc32c fnv132 fnv1a32 fnv164 fnv1a64 joaat murmur3a murmur3c murmur3f xxh32 xxh64 xxh3 xxh128 haval128,3 haval160,3 haval192,3 haval224,3 haval256,3 haval128,4 haval160,4 haval192,4 haval224,4 haval256,4 haval128,5 haval160,5 haval192,5 haval224,5 haval256,5 
424c426
< Client API library version => mysqlnd 8.0.28
---
> Client API library version => mysqlnd 8.1.18
436a439
> mysqli.local_infile_directory => no value => no value
445c448
< Version => mysqlnd 8.0.28
---
> Version => mysqlnd 8.1.18
494c497
< Client API version => mysqlnd 8.0.28
---
> Client API version => mysqlnd 8.1.18
784c787
< _ => /usr/bin/php8.0
---
> _ => /usr/bin/php8.1
820c823
< $_SERVER['_'] => /usr/bin/php8.0
---
> $_SERVER['_'] => /usr/bin/php8.1
827,828c830,831
< $_SERVER['REQUEST_TIME_FLOAT'] => 1686385683.8084
< $_SERVER['REQUEST_TIME'] => 1686385683
---
> $_SERVER['REQUEST_TIME_FLOAT'] => 1686385688.4512
> $_SERVER['REQUEST_TIME'] => 1686385688
969c972
< -
\ No newline at end of file
---
> PHP Notice:  iconv(): Detected an illegal character in input string in /var/www/html/web/public/iconv.php on line 3
$ diff /etc/php/8.0/cli/conf.d/20-iconv.ini /etc/php/8.1/cli/conf.d/20-iconv.ini
$ cat /etc/php/8.0/cli/conf.d/20-iconv.ini
; configuration for php common module
; priority=20
extension=iconv.so

So PHP Info is pretty much the same, I expect a "-" to be print, but instead on PHP8.1 it prints:

iconv(): Detected an illegal character in input string in /var/www/html/web/public/iconv.php on line 3

It is not seem to be a general problem of PHP8.1, as verified, but what can be different here? It's the same system / user and so on.

EDIT: It's on a Ubuntu system, packages are (apt show)

8.0.29-1+ubuntu18.04.1+deb.sury.org+1

and

8.1.20-1+ubuntu18.04.1+deb.sury.org+1

EDIT: that are not the installed packages but the last available ones!

The ini files were not changed, I compared them like this:

diff -r /etc/php/8.0/cli/ /etc/php/8.1/cli/|grep -v " ;"

And the only difference seems

524,528d528
< log_errors_max_len = 1024

Iconv settings are same-same:

Registered Stream Filters => zlib.*, string.rot13, string.toupper, string.tolower, convert.*, consumed, dechunk, bzip2.*, convert.iconv.*
iconv
iconv support => enabled
iconv implementation => glibc
iconv library version => 2.27
iconv.input_encoding => no value => no value
iconv.internal_encoding => no value => no value
iconv.output_encoding => no value => no value

EDIT: I upgraded to the latest versions 8.0.29 / 8.1.20, the problem persists on a the Ubuntu 18.04 system, but does not appear on Ubuntu 22.04 with sury packages.

0

There are 0 best solutions below