UTF-8 Transformation Details  (Character Sets, Part 3:  Unicode Encoding Environment)

The UTF-8 transformation of a Unicode scalar value into an octet sequence is accomplished by reallocating its bits into octets that begin with bit sequences identifying the function of the octet.

The following patterns show how bits of the scalar value are allocated to UTF-8 octets.

NOTE:  x,y,z, and u are used to show how bits get distributed among UTF-8 octets; the final two octets of a four-octet sequence, not shown here, are identical to the final two of the three-octet sequence.  Observe that the first hex digit of a four-octet sequence will always be F because the initial octet begins with bits 11110.  Similarly, the first hex digit of a three-octet sequence will always be E, and the first one of a two-octet sequence will be C or D.  A second or subsequent octet will begin with hex 8,9,A, or B.

Example of three encodings expressed in binary and hexadecimal notation:

No example of a four-octet sequence is shown, but the preceding table gives the pattern. Characters beyond FFFF (hex) are very rarely encountered in MARC 21.

To return, select:

Part 3:  Unicode Encoding Environment

Character Sets and Encoding Options