We consider the question in the subtitle, and the closely related question "When is a semigroup embeddable in a group?". Although the latter question has a history going back 70 years, it appears that no-one has previously considered its categorical analogue. We show that the categorical viewpoint enables one to give simpler and more perspicuous proofs of the results of Mal'cev, Lambek, Bush and others, as well as strengthening them in some respects. The central ingredient in our proofs is a simple combinatorial structure which we call a quadrangle club, but the proofs also have a homotopy-theoretic flavour.