Data types the hardware understands

A data type has hardware support when some instruction expects data in that format. Software can invent any representation it likes — a 256-bit integer, a date, a rational number — but only the hardware-supported ones have instructions that work on them directly, in one step. Everything else is built in software out of those. This chapter goes through the formats CPUs actually support: integers, decimal, floating point, characters, bits and pointers.

The variables chapter looked at these types from C: sizes, signedness, byte order. Here the question is what the instructions themselves assume.

Integers: one set of bits, two readings

Every mainstream ISA supports integers of 8, 16, 32 and 64 bits, unsigned and signed, with signed numbers in two's complement: to negate a number, invert all its bits and add 1. An 8-bit byte holds 0 to 255 unsigned, or −128 to 127 signed. 1111 1111 is 255 or −1, depending on which you mean.

Two's complement won over the alternatives because addition doesn't care. Adding the bit patterns gives the right answer whether you read them as signed or unsigned, so one adder and one add instruction serve both, and subtraction is addition of the negation. Older machines used sign-magnitude (a sign bit plus the absolute value) or one's complement (negation just inverts the bits); both have two zeros, +0 and −0, and need separate hardware for signed arithmetic. The C23 standard finally dropped them: C integers are now required to be two's complement.

Since the bits don't say which reading is meant, the instruction does. The operations where signed and unsigned differ come in pairs:

movsx fills ebx with copies of the top bit, 0xFFFFFFFF, which is −1. movzx fills with zeros, 255. The first add sets the carry flag, the unsigned overflow. The second sets the overflow flag instead: 200 fits in a byte as an unsigned number, but as a signed one, 100 + 100 has wrapped around to −56. Each add sets both flags; the program decides which one to look at, as the flags chapter showed.

Two's complement isn't the only integer encoding still in use, though. Excess-k (or biased) notation stores a value v as v + k: with k = 127, the byte 0111 1111 means 0. Its advantage is that bigger values have bigger bit patterns even when negative, so they compare like unsigned numbers. That's exactly how floating-point exponents are stored, as we'll see below.

Decimal: BCD

Binary-coded decimal stores one decimal digit per 4 bits, two per byte: 42 is 0100 0010, or 0x42. It's wasteful — a byte holds 0 to 99 instead of 0 to 255 — but converting to and from text is trivial, and decimal fractions like 0.10 are exact, which accountants and COBOL programs care about. Binary addition gives wrong results on BCD digits (0x09 + 0x01 is 0x0A, not 0x10), so machines that support BCD have decimal-adjust instructions that fix up the result, using the carry out of bit 3 — the auxiliary-carry flag that x86 still computes on every addition.

Tanenbaum lists BCD among the Core i7's data types. It is, but only in 32-bit mode. x86's decimal-adjust instructions (DAA, DAS, AAA, AAS, AAM, AAD) were removed from 64-bit mode, and their opcodes reassigned: nasm assembles daa as the byte 0x27 in 32-bit mode, and in 64-bit mode refuses with instruction not supported in 64-bit mode. What survives is the x87 floating-point unit's FBLD and FBSTP, which load and store an 18-digit packed BCD number. ARM and RISC-V never had BCD. The machines that still take decimal seriously are IBM's mainframes and POWER processors, which implement decimal floating point — the IEEE 754 decimal formats — in hardware.

Floating point

For numbers that aren't integers, CPUs use floating point: a number written as ± significand × 2^exponent, the binary equivalent of scientific notation. Since 1985 practically every CPU has followed the IEEE 754 standard, which fixes the formats bit for bit, and fixes the rounding — so a computation gives the same result on any conforming machine.

A binary floating-point number has three fields:

• a sign bit;
• an exponent, stored in excess notation, with bias 127 for single precision;
• a fraction: the significand's bits after the binary point. For normal numbers the significand is 1.fraction: the leading 1 isn't stored, since a normalized binary number always starts with 1. That's the hidden bit, one bit of precision for free.

The explorer below stores a number in each format. Type a value, or click any bit to flip it:

0.1 has no finite binary representation: like 1/3 in decimal, its binary digits repeat forever (0.000110011001100…). The format keeps the first 24 significant bits and rounds, so what's actually stored as a float is 0.100000001490116119384765625. Switch to fp64 and the error shrinks to about 5.6 × 10⁻¹⁸, but it doesn't disappear. That's why, in double precision, 0.1 + 0.2 is 0.30000000000000004 and not 0.3: both inputs are already rounded, and so is their sum. Try 0.1+0.2 in the explorer.

The same rounding limits whole numbers: a float has 24 bits of significand, so above 2²⁴ = 16,777,216 not every integer fits. Type 16777217 and it's stored as 16777216. A double has 53 bits, which is why JavaScript's largest exactly-representable integer is 2⁵³ − 1.

The standard reserves the extreme exponents for special values:

• exponent all zeros, fraction zero: ±0 — there is a negative zero, and it compares equal to +0;
• exponent all zeros, fraction non-zero: subnormal numbers, which have no hidden 1 and fill the gap between the smallest normal number and zero, losing precision gradually (try 1e-40 in fp32);
• exponent all ones, fraction zero: ±∞, the result of overflow or of dividing a nonzero number by zero;
• exponent all ones, fraction non-zero: NaN, not a number, the result of 0/0 or √−1. A NaN compares unequal to everything, itself included.

The formats

The book lists floating-point lengths of 32, 64 and 128 bits. Since then, machine learning has pushed the other way: half precision (fp16) and Google's bfloat16 are supported in hardware by recent x86 (AVX-512 FP16 and BF16), ARM64 and GPUs, and AI accelerators go down to 8 and even 4 bits. bfloat16 is simply a float with the bottom 16 fraction bits cut off: the same 8-bit exponent and range, much less precision, which is what training neural networks turns out to need. Switch the explorer between fp16 and bf16 on 65504 or 0.1 to see the trade-off. At the other end, 128-bit quad precision is mostly done in software; x86 also still has the x87's 80-bit extended format.

Floating point also gets its own registers on most ISAs: xmm0–xmm15 on x86-64 (the old x87 had an 8-entry register stack), v0–v31 on ARM64, f0–f31 on RISC-V with its F and D extensions.

Characters

A character is stored as a number, its code point. ASCII assigns 128 characters to 7 bits. Tanenbaum describes Unicode as 16-bit, which was the original design. It has been out of date since 1996, when Unicode grew past 65,536 characters: code points now go up to U+10FFFF, a 21-bit space of 1,114,112 values. It's stored in one of several encodings:

UTF-8 uses 1 to 4 bytes per character and is identical to ASCII for the first 128, which is why it has taken over files and the web. UTF-16, used inside Windows, Java and JavaScript, needs two 16-bit units, a surrogate pair, for characters beyond U+FFFF.

At the ISA level, characters are just bytes: no mainstream CPU knows about UTF-8. What x86 does have is string instructions that work on runs of bytes: rep movsb copies a block, repne scasb searches for a byte, and SSE4.2's pcmpistri compares 16 bytes against a set of characters in one instruction. ARM and RISC-V have none of these, and leave strings to ordinary loads, stores and vector instructions.

Bits and pointers

A Boolean needs one bit, but a single bit has no address, so languages store each Boolean in a byte. When there are many — a bitmap of free disk blocks, the set of pages a program has touched — they're packed 64 to a word, and the ISA has instructions for them: x86's bt/bts test and set bit n, popcnt counts the 1 bits in a word, and tzcnt/lzcnt find the first or last one. ARM64 and RISC-V have equivalents.

A pointer is an unsigned integer as wide as an address. The hardware treats it like any other integer until it's used to address memory. Since 64-bit CPUs use only 48 to 57 bits of address, recent ISAs have started using the rest: ARM64's top-byte ignore lets software store a tag in the top 8 bits, and pointer authentication stores a cryptographic signature there, checked before the pointer is used, which defeats many memory-corruption attacks. x86 is adding a similar feature, linear address masking.

Who supports what

The ATmega168 column comes from the book: a microcontroller whose only real data type is the byte. The ARM column differs from the book's 32-bit OMAP4430 in one more way: the book says all its operands must be aligned, but ordinary 32-bit ARM loads and stores have accepted misaligned addresses since ARMv6, as the previous chapter's measurements on ARM64 showed.

Takeaways

• A type has hardware support when instructions assume its format; anything else is built in software.
• Integers are two's complement: one adder for signed and unsigned. The bits don't say which; the instruction does — movsx/movzx, jl/jb, sar/shr, imul/mul, OF/CF.
• BCD stores decimal digits in 4 bits. x86 dropped its decimal-adjust instructions in 64-bit mode; IBM machines keep decimal floating point in hardware.
• IEEE 754 floating point: sign, biased exponent, fraction with a hidden 1; special values ±0, subnormals, ±∞, NaN. 0.1 is always rounded, and a float can't hold 16,777,217.
• fp16 and bfloat16 are now in hardware for machine learning; bfloat16 keeps a float's range with far less precision.
• Unicode is a 21-bit code space, not 16-bit; UTF-8 encodes it in 1 to 4 bytes. CPUs see only bytes, with some x86 string instructions.
• Pointers are address-sized integers; ARM64 puts tags and signatures in their unused top bits.