Encode invalid/unencoded characters in a JSON

Question

Sometimes I get JSON from a backend that contains unencoded/invalid characters. For example, they might contain an (unencoded) character with ASCII code 31, which means that when I try to do jsonDecode, it fails. Don't ask why the JSON-encoding on the backend is broken, it just is.

Anyway, I was wondering if there is a way to replace all unencoded characters with their encoded representations so I will be able to decode the JSON?

Answer 1

The solution I came up with is this. It basically checks every character in the JSON and if it is outside the "safe range" of ASCII 32-255, it JSON-encodes the character, strips any surrounding "" and adds the JSON-encoded representation instead of the unencoded character.

It could probably be improved, but it seems to do the job at least.

/// Replaces all characters that can not exist unencoded in a JSON 
/// with their JSON-encoded representations.
String replaceInvalidJsonCharacters(String json) {
  var charCodes = <int>[];

  for (final int codeUnit in json.codeUnits) {    
    if( codeUnit >= 32 && codeUnit <=255 ){ 
      // ASCII 32...255 are guaranteed to be valid in a JSON
      charCodes.add(codeUnit);
    } 
    else {
      // Json-encode the character and add the encoded version.
      // For characters that are valid in a JSON, the encoded version is the same
      // as the original (possibly surrounded by "").
      String encoded = jsonEncode(String.fromCharCode(codeUnit));
      if( encoded.length > 1 ){
        if( encoded.startsWith('"') ) encoded = encoded.substring(1,encoded.length);
        if( encoded.endsWith('"') ) encoded = encoded.substring(0,encoded.length-1);
      }
      charCodes.addAll(encoded.codeUnits);
    }
  }

  return String.fromCharCodes(charCodes);
}