Generalizing the concept of Lie triple systems, which describe symmetric spaces algebraically, I will discuss the algebraic structure pertaining to affine homogeneous spaces, i.e. homogeneous spaces allowing a left invariant connection of their tangent bundle. An unexpected characteristic of the deformation theory of this algebraic structure is that infinitely many polynomial equations have to be satisfied in contrast to the single quadratic Jacobi identity for Lie algebras or the quadratic relation characterising Lie triple systems. This tower of relations gives rise to a filtration on the space of formal deformations, whose successive quotients can be calculated using a suitable cohomology theory.