Optical cavities provide high sensitivity to dispersion since their resonance frequencies depend on the index of refraction. We present a direct, broadband, and accurate measurement of the modes of a high finesse cavity using an optical frequency comb and a mechanical Fourier transform spectrometer with a kHz-level resolution. We characterize 16000 longitudinal cavity modes spanning 16 THz of bandwidth in terms of center frequency, linewidth, and amplitude. Using the center frequencies we retrieve the group delay dispersion of the cavity mirror coatings and pure N2 with 0.1 fs2 precision and 1 fs2 accuracy, as well as the refractivity of the 3v1 + v3 absorption band of CO2 with 5 × 10-12 precision. This opens up for broadband refractive index metrology and calibration-free spectroscopy of entire molecular bands.