We obtain a compactness result for various classes of Riemannian metrics in dimension four; in particular our method applies to anti-self-dual metrics, Kahler metrics with constant scalar curvature, and metrics with harmonic curvature. With certain geometric assumptions, the moduli space can be compactified by adding metrics with orbifold singularities. Similar results were obtained previously for Einstein metrics, but our analysis differs substantially from the Einstein case in that we do not assume any pointwise Ricci curvature bound. This is joint work with Gang Tian.