Computer graphics

Computer graphics is the study of creating, manipulating, and using visual images.¹

Basics

A frame buffer is an area in memory that holds a bitmap of an image. Software will write the picture to be drawn into this memory, and the display controller will read from that memory to transmit information to the screen to render the picture. Pixels are individual squares in an image, made up of three elements: red, green, blue — this allows it to make many colours.

Pipeline

The graphics software pipeline generally has these components:²

• Application — user-space code.
• API runtime — the libraries that are called by the application. This API keeps track of the current application state, validates parameters, does error/consistency checking, manages some resources. It batches work and sends it over to the graphics driver.
• User-mode driver (UMD) — a great-deal of the pipeline occurs here. Often as much work as possible is done here since kernel-mode transitions are costly. For example:
  • Shader compilation, including optimisations (register allocation, loop unrolling). Much of the compilation is done in the API runtime (like Direct3D) to an IR, but machine-specific optimisations are done here.
  • Hand-optimised shaders are substituted in (especially in well-known games).
  • Legacy shader and API compatibility.
  • Memory allocation of blocks within KMD-allocated blocks.
  • Schedules transfers between physical memory and GPU memory.
  • Writes to DMA buffers once the KMD has allocated and passed them over.
• Scheduler — for the GPU functions like the OS scheduler. It partitions access to the GPU among multiple concurrently running processes.
• Kernel-mode driver (KMD) — there’s a single KMD that deals with the hardware.
  • Memory allocator and mapping (i.e., for virtual memory), hardware initialisation, setting configuration (display, power modes), manage the hardware mouse cursor, respond to interrupts.
  • Some DRM-related functionality.
  • Manages the actual hardware DMA buffer.
• Hardware — past this point is mostly hardware-related (probably barring the firmware on the GPU). The OS communicates with the GPU, usually with PCIe.

Sub-topics

• APIs
  • DirectX
  • Direct3D
  • Vulkan
  • OpenGL
• Hardware
  • GPU
  • Display

Resources

• A trip through the Graphics Pipeline 2011, by Fabien Giesen

See also

• CSC317 — Computer Graphics

Footnotes

1. From CS4620 — Introduction to Computer Graphics at Cornell. ↩

2. From A trip through the Graphics Pipeline 2011, part 1. ↩