On the side, I’ve been wrapping up some improvements to the classic Unix stdio libraries in illumos. stdio contains the classic functions like fopen(), printf(), and the security nightmare gets(). While working on support for fmemopen() and friends I got to reacquaint myself with some of the joys of the stdio ABI and its history from 7th Edition Unix. With that in mind, let’s dive into this, history, and some mistakes not to repeat. While this is written from the perspective of the C programming language, aspects of it apply to many other languages.

APIs and ABIs

Before we dive into a discussion of the stdio ABI, let’s first talk about APIs and ABIs. In programming, many people talk about application programming interfaces (APIs). APIs define how a program can call into another piece of functionality. In C, APIs are often contained in header files and these are documented in manual pages. For example, here are some API declarations in C:

extern int fstat(int struct stat *);
extern void *(size_t);
extern int fputc(int, FILE *);

The API names a function and describes the types of the parameters and the return value of the function. These declarations (and dependent headers) are all that one needs to write a C program. While the API is enough to write a program, when the compiler and link-editor get to work and you actually run your program, you need to rely on something else entirely: the ABI.

The application binary interface (ABI) describes a lot of aspects of the program that are required to have it run. For example, when you call into libc, where are the arguments found? Are they found on the stack? Are they found in registers? The ABI also describes certain things like how many bytes comprise an int and how should one lay out a structure.

Now, a large amount of the ABI is standardized in different documents for a given platform. For example, many Unix-based systems follow the System V ABI. This ABI defined many aspects of systems such as ELF (executable and linkable format) and dynamic linking. Portions of the ABI were relegated to processor-specific parts. These describe the use of registers, the alignment of types, and more. Some of the more interesting documents include:

These documents don’t describe everything that one needs. For example, while Linux, illumos, and the various BSDs all use the same amd64 calling conventions and ELF-based binaries and libraries, there are many things that each system uniquely defines. For example, let’s compare the struct stat used in illumos and OpenBSD. I’ve placed the two side by side, though I’ve trimmed out a bunch of pre-processor macros:

illumos                                         | OpenBSD
struct stat {                                   | struct stat {
        dev_t           st_dev;                 |         mode_t    st_mode;
        ino_t           st_ino;                 |         dev_t     st_dev;
        mode_t          st_mode;                |         ino_t     st_ino;
        nlink_t         st_nlink;               |         nlink_t   st_nlink;
        uid_t           st_uid;                 |         uid_t     st_uid;
        gid_t           st_gid;                 |         gid_t     st_gid;
        dev_t           st_rdev;                |         dev_t     st_rdev;
        off_t           st_size;                |         struct  timespec st_atim;
        timestruc_t     st_atim;                |         struct  timespec st_mtim;
        timestruc_t     st_mtim;                |         struct  timespec st_ctim;
        timestruc_t     st_ctim;                |         off_t     st_size;
        blksize_t       st_blksize;             |         blkcnt_t  st_blocks;
        blkcnt_t        st_blocks;              |         blksize_t st_blksize;
        char            st_fstype[_ST_FSTYPSZ]; |         u_int32_t st_flags;
};                                              |         u_int32_t st_gen;
                                                |         struct  timespec __st_birthtim;
                                                | };

Here, while there are a number of members which have the same types, the layout of these such as the order of the members is rather different. There’s nothing wrong with that; however, it’s pieces like this that are part of each system’s ABI. While these differences may seem a bit mundane at first, when you start to go further afield from a Unix-family system and say contrast with Windows, the differences are much larger. For example, the long data-type in 64-bit Windows is a 32-bit value, whereas it’s a 64-bit value in Unix-family systems.

Backwards Compatibility

You might reasonably ask, why do we care about these ABIs? Well, one of the areas where ABIs are quite important is in backwards compatibility. Backwards compatibility refers to the ability for today’s systems to run software that was built in the past. If that future system can run the older program, it is considered backwards compatibility. A common example is how you could play Gamecube games on the Nintendo Wii.

Different systems value backwards compatibility in different ways. Windows is famed (and cursed) for its backwards compatibility. One can run old software that targeted Windows 95 on Windows 10. Apple takes a different view entirely, and they generally drop compatibility after two or three major releases. Linus Torvalds long instilled the rule of "not breaking user space" in the Linux kernel, which is a similar edict about backwards compatibility for user applications, though the Linux kernel doesn’t have a stable API or ABI for drivers.

In illumos, we generally value maintaining a stable ABI over time so that way older software will continue to work. Maintaining backwards compatibility does come with some cost and sometimes requires that you put forth more effort to deal with it. However, if every major update meant you had to rebuild all of the software for your operating system, that can also be frustrating. illumos isn’t the only system with this property. A lot of other software is also known for valuing backwards compatibility. A notable example is glibc.

You might ask what does all of this talk of backwards compatibility have to do with the ABI. The two are pretty closely related. Take the struct stat example above. When a C program is compiled, the offsets of the members of a structure become party of the generated binary. If a new member were inserted into the middle of the structure that would change the offsets of all subsequent members and would mean that older programs would access the wrong thing.

The struct stat also shows another challenge with the ABI. If you look at the way that a program often uses it, it looks something like:

foo(const char *path)
	struct stat st;

	if (stat(path, &st) == 0)
		printf("%x\n", st.st_mode);

Here, the application has declared the stat structure on the stack. This means that the size of it is known at compile time and used. If we increased the size of the structure, but ran a program compiled using the older, smaller size of the structure, the stat() call would write beyond the space allocated to the structure on the stack and wreak havoc.

With all this in mind, there’s a lot we can learn from stdio and in particular what not to do.

The History of stdio

Standard I/O (stdio) was introduced in 7th Edition Unix. Many aspects of this original version of stdio are the same today. The FILE *, fopen(), getc(), putc(), and even stdin, stdout, and stderr were all there. The now-common header file /usr/include/stdio.h was introduced as well.

From there, it soon entered into the land of BSD. The second BSD release in 1979 actually contains its own implementation of a small part of stdio. However, by the time of BSD 2.9, if not earlier, it ended up with a version of stdio that looked pretty similar to the one in 7th edition. Eventually though, the folks at Berkeley wrote their own version of it. You can see that most clearly in the 4.4BSD version.

illumos, due to its heritage from Solaris, was a combination of BSD and System V Release 4 (SVR4). An interesting side effect of this is that we can track most of the implementation back to the release of SVR4. In fact, you can even see all the original AT&T copyrights in the files. Many parts of the code look similar to the corresponding version in 7th Edition. Many of the file names are even the same!

As a result of this particular bit of history, a lot of the ABI decisions that had been made in 7th edition were carried into SVR4 and made their way into Solaris. If we look at some of the decisions with a modern eye, they’re all things that make it much harder to extend standard I/O without breaking the ABI.

The FILE Structure

At the heart of stdio is the FILE structure. We’ll look at a portion of stdio.h from 7th Edition. This comes from The Unix Tree:

#define	BUFSIZ	512
#define	_NFILE	20
# ifndef FILE
extern	struct	_iobuf {
	char	*_ptr;
	int	_cnt;
	char	*_base;
	char	_flag;
	char	_file;
} _iob[_NFILE];
# endif


#define	NULL	0
#define	FILE	struct _iobuf
#define	EOF	(-1)


Well, here we go. This struct _iobuf is what everyone knows and loves as the FILE structure. A lot of this structure is still with us today. If you look at different versions of the structure, you’ll see the same members. The _ptr, _cnt, and _base members are used to help implement the buffering policies. The _flag member is how the structure kept track of knowing if it was open for read, write, or had hit EOF or an error.

The _file member is particularly interesting. This represented the file descriptor that was backed by the FILE structure. If you consider a char was either a signed or unsigned byte value, this limited you from file descriptors 0 to 127 or 255. Most folks could quite reasonably have more than 256 file descriptors open. If you’re writing a tool like grep, this might not be true, but if you’re writing a network application and there’s a file descriptor per connection, having more than 256 connections is quite common.

Back in 7th Edition, the FILE structures were statically allocated. There was no dynamic allocation of them and if we look at the bit above there, we’ll see that all you could have were 20 different FILE structures. In that world, having a file descriptor limit probably made a lot more sense and when this was first written it was before there was much networking present.

Unfortunately, the fact that these structures and the array were declared this way leaked into many different applications. Let’s take a look at the 4.4BSD version of the structure:

typedef	struct __sFILE {
	unsigned char *_p;	/* current position in (some) buffer */
	int	_r;		/* read space left for getc() */
	int	_w;		/* write space left for putc() */
	short	_flags;		/* flags, below; this FILE is free if 0 */
	short	_file;		/* fileno, if Unix descriptor, else -1 */
	struct	__sbuf _bf;	/* the buffer (at least 1 byte, if !NULL) */
	int	_lbfsize;	/* 0 or -_bf._size, for inline putc */

	/* operations */
	void	*_cookie;	/* cookie passed to io functions */
	int	(*_close) __P((void *));
	int	(*_read)  __P((void *, char *, int));
	fpos_t	(*_seek)  __P((void *, fpos_t, int));
	int	(*_write) __P((void *, const char *, int));

	/* separate buffer for long sequences of ungetc() */
	struct	__sbuf _ub;	/* ungetc buffer */
	unsigned char *_up;	/* saved _p when _p is doing ungetc data */
	int	_ur;		/* saved _r when _r is counting ungetc data */

	/* tricks to meet minimum requirements even when malloc() fails */
	unsigned char _ubuf[3];	/* guarantee an ungetc() buffer */
	unsigned char _nbuf[1];	/* guarantee a getc() buffer */

	/* separate buffer for fgetline() when line crosses buffer boundary */
	struct	__sbuf _lb;	/* buffer for fgetline() */

	/* Unix stdio files get aligned to block boundaries on fseek() */
	int	_blksize;	/* stat.st_blksize (may be != _bf._size) */
	fpos_t	_offset;	/* current lseek offset (see WARNING) */

While it’s different, many things are still the same. The flags and file descriptor grew to a short, but everything is still public, which means the layout of the structure is in the ABI. This all leads to a few critical issues that are all intertwined.

Lack of Opacity

If we look at the APIs that exist around stdio, all of them take a FILE *. This is a pointer to the FILE structure. This means that the use of the APIs doesn’t actually require someone to know the size. Even from the beginning, the APIs that gave you access to stdio, fopen(), and fdopen() returned a FILE * and stdin, stdout, and stderr all referred to a FILE * pointer.

All of this tells us that a consumer didn’t actually need to know the layout of the structure. If you have a consumer where you don’t actually know the layout of a structure, then we call that an opaque structure. There are a couple implications of this. Most notably it means that an application cannot allocate the structure itself, it must ask a library to, and that you need to use functions to access the members of the structure.

While in the early days of Unix, things like binary compatibility weren’t top of mind, by the 90s, some systems started caring about backwards compatibility and it was a bit too late. An existing body of software was using those fields directly.

Many of the things that we know of as functions that were standardized in C89 such as getc() or fileno(), actually were macros and just dereferenced the structure members. Here’s an example of another part of V7’s stdio.h:

#define getc(p)         (--(p)->_cnt>=0? *(p)->_ptr++&0377:_filbuf(p))
#define getchar()       getc(stdin)
#define putc(x,p) (--(p)->_cnt>=0? ((int)(*(p)->_ptr++=(unsigned)(x))):_flsbuf((unsigned)(x),p))
#define putchar(x)      putc(x,stdout)
#define feof(p)         (((p)->_flag&_IOEOF)!=0)
#define ferror(p)       (((p)->_flag&_IOERR)!=0)
#define fileno(p)       p->_file

What we think of today as functions were actually all macros dereferenced the FILE structure directly. Every consumer had to know the implementation. Take fileno() for example. It returns the corresponding file descriptor for a FILE *. Here, it just dereferenced the field, which means that programs that called fileno() encoded the actual size and offset of the _file member in their programs. The same is true of all the other members referenced

If we turn to modern implementations, the stdio functions aren’t actually implemented as macros for the most part. Folks will just pay the cost of a function for that opacity. However, that doesn’t actually mean it’s safe to modify and change the FILE structure around. The problem isn’t actually just software that people compile on their own that could have encoded it in the past. No, some modern software refuses to let go of the encoding and accept that these structures may now be opaque.

A great example of this is Gnulib which is a portability library. Gnulib has actually gone back and encoded the now-opaque structures that exist into itself! It will happily reach into the structures and modify them. Here’s a header file that encodes all of the structures for a number of different operating systems from Windows, to Android, to OS X, and even Minix. At this point, an operating system maintainer can’t do too much, without risking arbitrary corruption in user programs. While it’s tempting to say who cares about someone that encodes the private interface of a once-public structure, when software breaks we all lose. Users generally don’t care about who was responsible for breakage, just that it broke and that’s not necessarily a bad thing.

For better or for worse, the stdio ship has sailed here. It doesn’t matter if Android made their version private or if the Solaris 64-bit structure was never even in a public header, because stdio once visible, some software will still use it and encode the private implementation. Even when functions were added to get and set these private members.

An important take away here is that if you are concerned about backwards compatibility for a long time, a structure that users allocate themselves is probably not the right answer. Opaque structures do give library authors the ability to really change this over time. While not all software has the same constraints, when building a new interface, think twice about whether or not the structure should actually be exposed.

Member Sizing and Padding

A related challenge with the FILE structure is the size of its original members. Several implementations stuck with using a char for both the flags and the file descriptor (or in some cases a short). In the context of 7th Edition, this is a reasonable choice. However, as we consider what the impacts of sizing here are on the ABI, it makes things more difficult.

Because of these choices, there was a fundamental limit on the file descriptors that can be used. While 32-bit Solaris and illumos inherited a limit of 255 (!), FreeBSD, NetBSD, and OpenBSD today are all still limited to a short, which on current mainstay processors is a signed 16-bit quantity or 32767. Other systems such as musl, DragonFlyBSD, and 64-bit illumos don’t currently have the same constraints. In some of these cases that’s due to a useful application of hindsight or a lot of careful engineering.

There is always a trade-off and tension between how large to size something and the corresponding memory impact. If you’re going to implement a structure whose implementation you need to maintain in a backwards compatible fashion for a while, you’ll want to think through what those sizes should be. The size that was right in the 1980s may not be right in the 2020s and the size that’s right today maybe seen as small in the future as well.

A related thing to think is structure padding. Padding is a way to add extra members that aren’t used today but may be in the future. API designers often add a bunch of arbitrary bytes to the end of a structure so they can change it in the future without having to break ABIs. This is a technique that’s used beyond operating systems. Most structures in the NVMe specification use this technique as well. Padding is also used to make sure that members in a structure are aligned at a certain granularity such as a cache line. This is done to avoid false sharing.

The nice thing about structure padding is that software will always allocate a structure of the right size, but will ignore the unused fields. This lets older software still work with newer versions of the structure, even if they don’t understand what all the fields mean.

Arrays and ABIs

A related challenge to structure opacity is the use of arrays in ABIs. Wait, where have we really used ABIs? While systems have managed to get around the _iob[_NFILE] declaration with fopen(), there’s actually a more interesting case: the definitions of stdin, stdout, and stderr. Let’s look at a couple examples side by side including 7th Edition and 4.4BSD (which can still be found in some of the BSDs today):

7th Edition                | 4.4BSD
#define	stdin	(&_iob[0]) | #define	stdin	(&__sF[0])
#define	stdout	(&_iob[1]) | #define	stdout	(&__sF[1])
#define	stderr	(&_iob[2]) | #define	stderr	(&__sF[2])

Here, we can see that we’ve made an array an explicit part of the ABI. This is another way that enforces the size of the existing structures. Using this technique forces the structures actually to have a known size, even if the actual member layout is private. This can be seen on various systems, such as 64-bit illumos.

If one were to do this today, they’d instead define these macros as pointers to a symbol that existed in libc. Instead the program has a base reference to the _iob or __sF symbol and then knows the memory offset to find the next values at. If we were to increase the structure size, suddenly a program that used to refer to stdout would actually be finding a bit of the memory at the end of stdin! Trust me, having made the mistake in development, that this doesn’t end well at all.

If you’re designing something new, think twice before an array becomes part of an ABI. And if not for this, then because of copy relocations.

ABI Lessons

To summarize, if you’re working on something where backwards compatibility is important, here are some lessons we can take from stdio:

  • Keep your structures opaque if you can! Folks can and will encode public things even if you later make them private.

  • If your structure will be public, think carefully about the sizing of members.

  • Make sure to add room for expansion in public structures.

  • Think twice before exposing an array!

Further Reading

If you want to learn more about the implementation of stdio in illumos, as part of my recent work I wrote up a design document. Parts of this will generalize to other systems.

Here are historical versions of different implementations of stdio:




7th Edition












System III


pdp11 and vax

Final Thoughts

Reading this may lead to some despair: 'Ugh, parts of this ABI are terrible.', 'Really, some things are limited to 256 file structures?', 'Growing the structure requires jumping through hoops?', 'Why do you still live with it?'.

Trying to answer the question of when you should break an API or an ABI is a tough one. The answers will vary depending on the project and what its values are. In some areas of software, the API and ABI are broken regularly. When colleagues of mine have experienced that in different frameworks, it was often quite frustrating and demotivating. Not everything has the same trade offs as an operating system and there are times where breaking the API and ABI are important and necessary.

While the constraints of working within an existing API or ABI can occasionally be infuriating, figuring out how to enable new functionality without breaking existing functionality is an important part of engineering. Studying and dealing with episodes like the legacy of stdio helps teach us what not to do. Hopefully that means that when we look ten or twenty years in the future, some of the things that we’re putting together today will have stood the test of time.

I’m sure that this won’t be the last time that I’m dealing with the design decisions whose history stretches back to 7th Edition Unix. While older code bases with 40+ years of history have a lot of rocks you may not want to turn over, there’s a lot that you can learn from them and a large number of interesting problems as well. And, if we don’t learn from them, people in 4o years will view us the same way.

Still, I find it helpful to remember what Roger Faulkner, a man who had to suffer with more of this than anyone else, was fond of saying: 'That code came from New Jersey'.

Previous Entry: Joining Oxide| All Entries | Next Entry: None