Példa

public static void main( String[] args ){
java.util.StringTokenizer st = new java.util.StringTokenizer("this is \ud800\udc00 a test", "\ud800"); //$NON-NLS-1$ //$NON-NLS-2$
int segment=0;
while (st.hasMoreTokens()) {
System.out.println(st.nextToken());
segment=segment+1;
}
}
Megoldás
Használjon ICU 2.6.1-et (com.ibm.icu.util.StringTokenizer).

public static void main( String[] args ){
com.ibm.icu.util.StringTokenizer st = new com.ibm.icuutil.StringTokenizer("this is \ud800\udc00 a test", "\ud800"); //$NON-NLS-1$ //$NON-NLS-2$
int segment=0;
while (st.hasMoreTokens()) {
System.out.println(st.nextToken());
segment=segment+1;
}
}