Definition
A graphics processing unit is a throughput-oriented processor that executes many operations in parallel. Neural-network workloads map well to GPUs because training and inference rely heavily on large matrix multiplications, convolutions, and other tensor operations that can be divided across many threads.
Usable performance depends on more than advertised floating-point operations. Device memory limits the model, activations, optimizer state, and batch that fit at once. Memory bandwidth constrains how quickly operands reach compute units. Numerical precision changes speed, memory use, and accuracy. Kernel quality, batching, and communication between devices determine how much of the hardware is occupied.
NVIDIA's CUDA guide documents the single-instruction, multiple-thread execution model used by its GPUs. The 2012 AlexNet paper supplied an influential deep-learning example: the authors split a large convolutional network across two GPUs to make its training practical. That paper did not invent general-purpose GPU computing, but it helped demonstrate its value for large neural networks.
Distinguish it from nearby terms
A model is architecture plus learned state; a GPU is hardware that executes operations. A GPU is not automatically faster than a CPU for small, serial, branch-heavy, or data-transfer-bound work. An accelerator-hours figure also says little without the device type and utilization.
Check your understanding
A model's arithmetic fits easily within a GPU's peak FLOPS, but execution remains slow and utilization is low. Which evidence would help distinguish a memory-bandwidth bottleneck, an undersized batch, and host-to-device transfer overhead?