Structural Maintenance of Chromosomes (SMC) protein complexes are key determinants of chromosome conformation. Using Hi-C and polymer modelling, we study how cohesin and condensin, two deeply-conserved SMC complexes, organize chromosomes in budding yeast. The canonical role of cohesins is to co-align sister chromatids whilst condensins generally compact mitotic chromosomes. We find strikingly different roles in budding yeast mitosis. First, cohesin is responsible for compacting mitotic chromosomes arms, independent of and in addition to its role in sister-chromatid cohesion. Cohesin dependent mitotic chromosome compaction can be fully accounted for through cis-looping of chromatin by loop extrusion. Second, condensin is dispensable for compaction along chromosomal arms and instead plays a specialized role, structuring rDNA and peri-centromeric regions. Our results argue that the conserved mechanism of SMC complexes is to form chromatin loops and that SMC-dependent looping is readily deployed in a range of contexts to functionally organize chromosomes.