We will discuss several decompositions related to the work of Connes and Kreimer on renormalization of perturbative quantum field theory. We show that their algebraic Birkhoff decomposition giving the counter term and renormalization of a regularized Feynman rule coincides with a factorization in a complete Rota-Baxter algebra. In this context, a similar decomposition also gives an extension of multiple zeta values for non-positive integers. Other decompositions will also be considered.