To each Lie bialgebra, Lie bialgebroid and twilled Lie-Rinehart algebra there is associated a pair of Gerstenhaber algebras. We show how various weakenings of the respective definitions give rise in each case to a quasi-Gerstenhaber algebra, the simplest example of a homotopy Gerstenhaber algebra.