Histone modifications alone or in combination are thought to modulate chromatin structure and function; a concept termed histone code. By combining evidence from several studies, we investigated if the histone code can play a role in higher-order folding of chromatin. Firstly using genomic data, we analyzed associations between histone modifications at the nucleosome level. We could dissect the composition of individual nucleosomes into five predicted clusters of histone modifications. Secondly, by assembling the raw reads of histone modifications at various length scales, we noticed that the histone mark relationships that exist at nucleosome level tend to be maintained at the higher orders of chromatin folding. Recently, a high-resolution imaging study showed that histone marks belonging to three of the five predicted clusters show structurally distinct and anti-correlated chromatin domains at the level of chromosomes. This made us think that the histone code can have a significant impact in the overall compaction of DNA: at the level of nucleosomes, at the level of genes, and finally at the level of chromosomes. As a result, in this article, we put forward a theory where the histone code drives not only the functionality but also the higher-order folding and compaction of chromatin.