Multisensory integration does not only recruit higher-level association cortex, but also primary sensory cortices like A1 (auditory), S1 (somatosensory), and V1 (visual). The underlying anatomical pathways, which might preferentially serve short-latency integration processes, include direct thalamocortical and corticocortical connections across the senses. We investigated how these multisensory connections develop over the individuals lifespan and how early sensory deprivation alters them. Using tracer injections into A1, S1, and V1 of a rodent model (Mongolian gerbil) we could show that multisensory thalamocortical connections emerge before corticocortical connections but mostly disappear towards the end of the critical sensory period. Early auditory, somatosensory, or visual deprivation increases multisensory connections via axonal reorganization processes mediated by non-lemniscal thalamic nuclei and the primary areas themselves. Functional imaging reveals a mostly reduced stimulus-induced activity but a higher functional connectivity specifically between primary areas in deprived animals. In adult animals, primary sensory cortices receive substantial inputs from thalamic nuclei and cortical areas of non-matched sensory modalities. In very old animals, these multisensory connections strongly decrease in number or vanish entirely. This is likely due to a retraction of the projection neuron axonal branches and is accompanied by changes in anatomical correlates of inhibition and excitation in the sensory thalamus and cortex. Together, we show that during early development, intracortical multisensory connections are formed as a consequence of sensory driven multisensory thalamocortical activity and that during aging, multisensory processing is probably shifted from primary cortices towards other sensory brain areas.