In this talk, I will explain how to compute the trace of a power of the Frobenius endomorphism on the intersection cohomology of the Baily-Borel compactification of a Siegel modular variety, or rather how to reduce the problem to Kottwitz's calculation of the number of points of PEL Shimura varieties over finite fields. The main tools are : - a theorem of Pink about the direct image in the Baily-Borel compactification of a local system on a Shimura variety; - a new construction of the intermediate extension of a pure perverse sheaf as a weight truncation of the full direct image.