Character encoding
A character encoding defines how text characters or code points are represented as byte sequences and decoded back.
Definition
A character encoding specifies how textual units are converted to numbers or bytes for storage and transmission, and how those values are decoded back into text. The sender and receiver must use the same encoding.
ASCII, UTF-8, UTF-16, and legacy code pages are character encodings or encoding schemes. Unicode defines a shared repertoire and code points; UTF-8 and UTF-16 define concrete byte or code-unit representations of those points.
How it works
An encoder maps characters to a valid byte sequence. A decoder applies the inverse rules. In UTF-8, A is the byte 41, while € is E2 82 AC. Variable-width encodings use different numbers of bytes for different code points.
Metadata such as an HTTP charset, a file format declaration, or an agreed protocol tells software which decoder to use. Without that context, the same bytes can produce different text.
Practical example
If the UTF-8 bytes E2 82 AC are decoded as UTF-8, they produce €. Interpreting them with an unrelated single-byte encoding produces other characters instead. This visible corruption is often called mojibake.
Converting text between encodings requires decoding the source bytes correctly and then encoding the resulting characters in the destination encoding; relabeling bytes is not conversion.
Errors and pitfalls
A decoder may reject invalid byte sequences or replace them with U+FFFD, the replacement character. A destination encoding may also be unable to represent some input characters, causing an error, substitution, or data loss.
Byte order marks, endianness, Unicode normalization, and line endings are related but distinct concerns. Character encoding is reversible representation, not encryption.
No. Unicode assigns code points and properties to characters; UTF-8 is one encoding that represents those code points as bytes.
The bytes may have been decoded with the wrong encoding, or the original characters may have been replaced because the chosen destination encoding could not represent them.
No. Some byte patterns fit several encodings, especially short ASCII-only text. Protocol metadata, a byte-order mark, or external knowledge is often needed.