Data Loaders
Image data in 2. AREAS/Tech/deep learning/Deep Learning is memory-intensive due to large image sizes, batch processing, intermediate activations, real-time data augmentation, and limited GPU memory for weights and activations.
Problem with Loading All Images into RAM
This method works well for small datasets or low-resolution images, like MNIST (28×28, 70,000 images) or CIFAR10 (32×32, 60,000 images).
However, for larger datasets (e.g., 100,000+ images at 224×224), memory usage skyrockets. For instance, 80,000 images at 400×400 can consume over 24 GB of RAM, requiring high-end systems with 32–64 GB RAM.
This is impractical for most beginners or those experimenting with deep learning, as such hardware is uncommon.