An application binary interface (ABI) specifies the interface between two binary programs in machine code, i.e., which registers to pass arguments to, and which registers the return value is placed. Adhering to an ABI is usually the job of a compiler, operating system, or library author.

Programming language calling conventions are examples of important ABIs. For example, the C ABI is stack-based, i.e., any function calls allocate a block of memory on the stack (a stack frame) to store local variables, parameters, and the return address.

Instruction set architectures

Many instruction set architectures also have ABI specifications. ABIs encompass calling conventions, data type sizes and alignments, memory layout, object file format, syscall interfaces, and symbol naming conventions. See x86-64 for an example.

Calling conventions define what registers are used for what purpose: passing parameters to subroutines, return values, how registers are used, and stack manipulation. Two key terms re: how registers are used:

  • Caller-saved registers mean the data stored are the responsibility of the caller subroutine. They must be saved on the stack by the caller if it wants to preserve them, i.e., we save, then call a subroutine, then the caller restores them when returned.
  • Callee-saved registers are the responsibility of the callee subroutine. If it wants to use these registers, their contents must be saved before being changed, and restored before returning.

for aarch64 Linux: syscalls don’t have specific addresses but are instead called via interrupts for the OS to handle:

  • x8: syscall number
  • x0 to x5: arguments
  • And the svc is a syscall instruction