Dualities are mathematical mappings that reveal links between apparently unrelated systems in virtually every branch of physics. Systems mapped onto themselves by a duality transformation are called self-dual and exhibit remarkable properties, as exemplified by the scale invariance of an Ising magnet at the critical point. In this talk, we show how dualities can enhance the symmetries of a dynamical matrix (or Hamiltonian), enabling the design of metamaterials with emergent properties that escape a standard group theory analysis. As an illustration, we consider twisted Kagome lattices, reconfigurable mechanical structures that change shape via a collapse mechanism. Pairs of distinct configurations along the mechanism exhibit the same vibrational spectrum and related elastic moduli. We show that these puzzling properties arise from a duality between pairs of configurations on either side of a mechanical critical point. The normal modes of the self-dual system exhibit non-Abelian geometric phases that affect the semi-classical propagation of wave packets, leading to non-commuting mechanical responses. Our results hold promise for holonomic computation and mechanical spintronics by allowing on-the-fly manipulation of synthetic spins carried by phonons.