Moduli spaces of Gieseker semistable sheaves on curves and surfaces carry natural theta divisors, which are higher-rank analogues of the classical theta divisors on the Picard varieties of curves. If one has two complementary setups each consisting of a moduli space of sheaves together with a theta bundle on it, it is often the case that the spaces of sections of the two theta bundles, on the different moduli spaces, are naturally dual. I will describe this phenomenon in a series of examples. [This is joint work with Dragos Oprea.]