This deep learning dataset is designed for image classification and segmentation of bulky waste. It contains 22,659 patches with dimensions of 50 × 50 × 717 px. The dataset provides both patch-wise and pixel-wise annotations, with labels categorized into two main classes and 16 subclasses. The data was acquired using a multi-sensor imaging system comprising a high-resolution VIS/RGB camera, a hyperspectral NIR camera, a thermographic camera, and a THz scanner.