Tensor
In deep-learning software, a tensor is an array of values organized along zero or more dimensions. Scalars, vectors, matrices, images and batches can all be represented as tensors, with a shape and a data type.
Model ArchitectureAI Infrastructure
A batch of 32 red-green-blue images can have shape [32, 3, 224, 224]: batch, color channel, height and width. That is one layout, not a universal image convention; some libraries put channels last. A shape error often means an operation expected a different arrangement or number of values.
Three properties help when debugging: shape describes dimensions, data type describes how each value is represented, and device describes where the data is stored, such as processor or accelerator memory. Operations may require compatible shapes, types and devices. PyTorch’s tensor tutorial demonstrates these properties and the operations on them.
This matters for memory as well as correctness. A dense float32 tensor in that example holds 4,816,896 values, taking 19,267,584 bytes for the raw values, before other training or framework allocations. Learned weights, activations and embeddings can all be tensors. The term describes their representation, not whether their values were learned.
Sources
- PyTorch: Tensors tutorial — Demonstrates tensors, shape, data type, device and tensor operations; the image-memory calculation here is illustrative.
Go deeper
- NumPy: Broadcasting docs
Work through shape compatibility and array operations, a useful foundation for tensor debugging.