Příklad

public static void main(String[] args){
java.util.StringTokenizer st = new java.util.StringTokenizer("this is \ud800\udc00 a test", "\ud800"); //$NON-NLS-1$ //$NON-NLS-2$
int segment=0;
while (st.hasMoreTokens()) {
System.out.println(st.nextToken());
segment=segment+1;
}
}
Řešení
Použijte ICU 2.6.1com.ibm.icu.util.StringTokenizer.

public static void main(String[] args){
com.ibm.icu.util.StringTokenizer st = new com.ibm.icuutil.StringTokenizer("this is \ud800\udc00 a test", "\ud800"); //$NON-NLS-1$ //$NON-NLS-2$
int segment=0;
while (st.hasMoreTokens()) {
System.out.println(st.nextToken());
segment=segment+1;
}
}