java - 来自 byte[] 和 UTF-8 的字符串在 Android 上与在 Windows JVM 上给出不同的结果

标签 java android utf-8 jvm

我正在尝试使用以下代码将字节数组转换为 Java 中的字符串:

byte[] myArray = {25, -50, -86, 81, 47, 44, 97, -5, 69, -4, 87, -114, -47, 62, -113, -64, 58, -32, -121, -102, 53, -89, -122, 12, -2, -23, -127, 111, -100, 53, -87, -23, -44, -28, 4, -21, -42, 75, 87, -112, -38, 118, 54, 92, -116, 4, -118, 110, -87, 7, -13, 3, -72, -63, -69, 123, 92, 94, 56, 61, 120, -52, 98, -17, 5, 41, 101, -3, 121, 81, -90, 12, -35, -21, -24, 112, -94, 123, 62, 8, 27, 54, 107, -77, 64, 8, -102, -99, -1, 119, 127, 43, 12, -31, -1, 51, -15, 83, -4, -68, -30, 91, -104, 84, 18, -122, -120, 66, 116, -17, -101, -24, 105, -112, -116, -64, -108, 112, -35, 61, 66, 100, 5, -24, -26, -44, 81, -84}; // Bytes from Byte.MIN_VALUE to Byte.MAX_VALUE
String result = new String(myArray, StandardCharsets.UTF_8);

问题是,如果我在 Windows(JVM 1.8.0_112)中运行代码与在我的 android 设备(在 android 5.1 和 6.0 中测试)中运行代码,我会得到不同的结果。我正在使用长度为 128 的字节数组进行测试,在 android 中我得到一个长度为 120 的字符串,而在 Windows 中我得到一个长度为 125 的字符串。我猜它与一些无效的 utf 字节有关-8 个字符,但根据平台得到不同的结果仍然很奇怪。

如果我将编码更改为 US-ASCII,我会在两个平台上得到与预期相同的结果:
String result = new String(myArray, StandardCharsets.US_ASCII);

编辑:很抱歉造成困惑。我不是每次都随机生成它。我只是说这些字节没有有意义的 UTF-8 值。这是我用来测试的字节数组:
System.out.println(Arrays.toString(myArray)): [25, -50, -86, 81, 47, 44, 97, -5, 69, -4, 87, -114, -47, 62, -113, -64, 58, -32, -121, -102, 53, -89, -122, 12, -2, -23, -127, 111, -100, 53, -87, -23, -44, -28, 4, -21, -42, 75, 87, -112, -38, 118, 54, 92, -116, 4, -118, 110, -87, 7, -13, 3, -72, -63, -69, 123, 92, 94, 56, 61, 120, -52, 98, -17, 5, 41, 101, -3, 121, 81, -90, 12, -35, -21, -24, 112, -94, 123, 62, 8, 27, 54, 107, -77, 64, 8, -102, -99, -1, 119, 127, 43, 12, -31, -1, 51, -15, 83, -4, -68, -30, 91, -104, 84, 18, -122, -120, 66, 116, -17, -101, -24, 105, -112, -116, -64, -108, 112, -35, 61, 66, 100, 5, -24, -26, -44, 81, -84]

编辑2:
窗口结果:
System.out.println(String(myArray, StandardCharsets.UTF_8)).length: 125
System.out.println(String(myArray, StandardCharsets.UTF_8)): ΪQ/,a�E�W��>��:���5����o�5������KW��v6\��n�����{\^8=x�b�)e�yQ����p�{6k����w+��3�S���[�T��Bt��i����p�=Bd���Q�
System.out.println(toUnicode(String(myArray, StandardCharsets.UTF_8))): \u0019\u03aa\u0051\u002f\u002c\u0061\ufffd\u0045\ufffd\u0057\ufffd\ufffd\u003e\ufffd\ufffd\u003a\ufffd\ufffd\ufffd\u0035\ufffd\ufffd\u000c\ufffd\ufffd\u006f\ufffd\u0035\ufffd\ufffd\ufffd\ufffd\u0004\ufffd\ufffd\u004b\u0057\ufffd\ufffd\u0076\u0036\u005c\ufffd\u0004\ufffd\u006e\ufffd\u0007\ufffd\u0003\ufffd\ufffd\ufffd\u007b\u005c\u005e\u0038\u003d\u0078\ufffd\u0062\ufffd\u0005\u0029\u0065\ufffd\u0079\u0051\ufffd\u000c\ufffd\ufffd\ufffd\u0070\ufffd\u007b\u003e\u0008\u001b\u0036\u006b\ufffd\u0040\u0008\ufffd\ufffd\ufffd\u0077\u007f\u002b\u000c\ufffd\ufffd\u0033\ufffd\u0053\ufffd\ufffd\ufffd\u005b\ufffd\u0054\u0012\ufffd\ufffd\u0042\u0074\ufffd\ufffd\u0069\ufffd\ufffd\ufffd\ufffd\u0070\ufffd\u003d\u0042\u0064\u0005\ufffd\ufffd\ufffd\u0051\ufffd

安卓结果:
System.out.println(String(myArray, StandardCharsets.UTF_8)).length: 120
System.out.println(String(myArray, StandardCharsets.UTF_8)): ΪQ/,a�E�W��>��:ǚ5����o�5������KW��v6\��n���{{\^8=x�b�)e�yQ����p�{>6k�@���w+�
System.out.println(toUnicode(String(myArray, StandardCharsets.UTF_8))): \u0019\u03aa\u0051\u002f\u002c\u0061\ufffd\u0045\ufffd\u0057\ufffd\ufffd\u003e\ufffd\ufffd\u003a\u01da\u0035\ufffd\ufffd\u000c\ufffd\ufffd\u006f\ufffd\u0035\ufffd\ufffd\ufffd\ufffd\u0004\ufffd\ufffd\u004b\u0057\ufffd\ufffd\u0076\u0036\u005c\ufffd\u0004\ufffd\u006e\ufffd\u0007\ufffd\u0003\ufffd\u007b\u007b\u005c\u005e\u0038\u003d\u0078\ufffd\u0062\ufffd\u0005\u0029\u0065\ufffd\u0079\u0051\ufffd\u000c\ufffd\ufffd\ufffd\u0070\ufffd\u007b\u003e\u0008\u001b\u0036\u006b\ufffd\u0040\u0008\ufffd\ufffd\ufffd\u0077\u007f\u002b\u000c\ufffd\ufffd\u0033\ufffd\u0053\ufffd\ufffd\u005b\ufffd\u0054\u0012\ufffd\ufffd\u0042\u0074\ufffd\ufffd\u0069\ufffd\ufffd\u0014\u0070\ufffd\u003d\u0042\u0064\u0005\ufffd\ufffd\ufffd\u0051\ufffd

编辑 3:添加了正确的 UTF-16 字符串

编辑 4:将代码更改为工作示例

最佳答案

看来,Android 在解释 UTF-8 序列时有点草率。标准的相关部分在D92chapter 3, “Conformance” :

Before the Unicode Standard, Version 3.1, the problematic “non-shortest form” byte sequences in UTF-8 were those where BMP characters could be represented in more than one way. These sequences are ill-formed, because they are not allowed by Table 3-7.



您的输入具有这样的“非最短形式”序列,例如-32, -121, -102-63, -69 .虽然 Android 将这些序列中的每一个解释为单个字符,但 Java 正确地拒绝了这些序列并将格式错误的输入的每个字节转换为单个替换字符,从而产生更长的字符串。

您可以使用解释“Modified UTF-8”的解析器在 Java 中演示它:
byte[][] samples = {
    { -32, -121, -102 },
    { -63, -69 }
};
for(byte[] array: samples) {
    System.out.println("source: "+Arrays.toString(array));
    String string = new String(array, StandardCharsets.UTF_8);
    System.out.println("strictly interpreted: "+string);
    System.out.println("length: "+string.length());
    ByteBuffer bb = ByteBuffer.allocate(array.length+2);
    bb.putShort((short)array.length).put(array);
    ByteArrayInputStream bis = new ByteArrayInputStream(bb.array());
    DataInputStream dis = new DataInputStream(bis);
    string = dis.readUTF();
    System.out.println("sloppily interpreted: "+string);
    System.out.println("length: "+string.length());
    byte[] actual = string.getBytes(StandardCharsets.UTF_8);
    System.out.println("correct sequence: "+Arrays.toString(actual));
    System.out.println();
}

这将打印

source: [-32, -121, -102]
strictly interpreted: ���
length: 3
sloppily interpreted: ǚ
length: 1
correct sequence: [-57, -102]

source: [-63, -69]
strictly interpreted: ��
length: 2
sloppily interpreted: {
length: 1
correct sequence: [123]

它还显示了字符的正确“最短形式”序列。

关于java - 来自 byte[] 和 UTF-8 的字符串在 Android 上与在 Windows JVM 上给出不同的结果,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/43004496/

相关文章:

java - 从多个类访问未同步的 map

java - 使用 Hibernate 对 Java 类进行建模的最佳实践是什么?

android - org.gradle.execution.TaskSelectionException : Task 'wrapper' not found in project ':app'

python - 如何在 Python 中从 SQL 数据库中打印 UTF-8 字符?

jsf - 在 JSF 1.2 中使用错误编码的 POST 参数

python - 为什么所有这些 un​​icode 命令在 Python 中都能正常工作?无论我做什么,他们都正确地打印了我的角色

java - 具有组计数 gremlin 查询的子句

java - Android 社​​交 SDK : Unable to execute dex: Multiple dex files define

Android工具栏没有做我想要的

java - 当用户在Android中关闭应用程序时如何重新启动应用程序