Long-range inter-chromosomal interactions in the interphase nucleus subsume critical genome-level regulatory functions such as transcription and gene expression. To decipher the physical basis of diverse pan-nuclear patterns of chromosomal arrangement that facilitates these processes, we investigate the scaling effects within disparate genomes and compared their total number of genes with chromosome size. First, we derived the pan-nuclear average fractal dimension of inter-chromosomal arrangement in interphase nuclei of different species and corroborated our predictions with independently reported results. Then, we described the different patterns across disparate unicellular and multicellular eukaryotes. We report that, unicellular lower eukaryotes have inter-chromosomal fractal dimension ≈ 1 at the pan-nuclear scales, which is analogous to the multi-polymer crumpled globule model. Multi-fractal dimensions, corresponding to different inter-chromosomal arrangements emerged from multicellular eukaryotes, such that closely related species have relatively similar patterns. Using this theoretical approach, we could distinguish fractal patterns from human acrocentric versus metacentric chromosomes, implying that the multi-fractal nature of inter-chromosomal geometry facilitates viable large-scale chromosomal aberrations, such as Robertsonian translocations. We report that the nature of such an average multi-fractal dimension for nocturnal mammals is very different in diurnal mammals, which suggests a greatly enhanced plasticity in arrangement across different cell types, for example retinal versus dermal fibroblasts. Altogether, our results substantiate that genome-level constraints have also co-evolved with the average pan-nuclear fractal dimension of inter-chromosomal folding during eukaryotic evolution.