LESSON p.3 · Lab · 150 min
From NumPy arrays to PyTorch tensors
From NumPy arrays to PyTorch tensors. Learn ndarray, shape, axis, broadcasting, views/copies, dtype, and tensors and complete: Vectorize bigram counts and cross-check NumPy against
Learning objectives
- Describe arrays with ndim, shape, axis, size, and dtype.
- Predict whether broadcasting is valid and write the output shape.
- Distinguish a slicing view from an explicit copy.
- Match NumPy and PyTorch results on the same tiny input.
Core concepts
ndarray and homogeneous data
An ndarray has a fixed dtype and rectangular shape. Those constraints enable compact storage and batch operations; nested-list behavior cannot be assumed to transfer.
shape, axis, and reduction
Shape gives each axis length; axis selects which dimension is reduced. Write input and output shapes before sum, mean, or softmax.
Broadcasting
Compare axes from the end: sizes must match or one must be 1. Broadcasting is a logical expansion and does not necessarily materialize a full copy.
Views and copies
Basic NumPy slices usually share data. Cross-library conversions may also share memory; use copy or clone when independent storage is required.
NumPy to PyTorch
The libraries share many shape and broadcasting rules, while PyTorch adds device, requires_grad, and a computation graph. Establish numeric parity before gradients.
Build and verify
Vectorize bigram counts and compare NumPy with PyTorch
- Hand-build a 3×3 count matrix from four token IDs.
- Implement a loop, np.add.at, and torch.index_put_.
- Assert shape, dtype, total count, and elementwise equality.
- Change vocabulary size, repeated pairs, and empty inputs.