In many low temperature plasmas, the OH radical and temperature represent important properties of plasma reactivity. However, experimentally measuring OH radicals in weakly ionized low temperature plasmas is often difficult because of its low concentration, posing a significant challenge to diagnostics. To address such issue, this work combines cavity enhanced absorption spectroscopy (CEAS) with femtosecond (fs) laser to simultaneously measure OH concentration and temperature in-situ in low temperature plasmas. This combination has two major benefits. First, the CEAS technique featuring a low detection limit can be used to measure low-concentration OH radicals in the weakly ionized low temperature plasma environment. Second, the broadband fs laser further provides CEAS with many absorption features as constraints, such that simultaneous measurements of OH concentration and temperature can be enabled with improved accuracy. In this work, the fs CEAS was demonstrated on OH both numerically and experimentally in plasmas generated by a nanosecond-pulsed dielectric barrier discharge. The results showed that, at OH concentration down to 0.05 ppm, the fs CEAS could achieve a 0.5% measurement error for OH concentration and a 0.2% error for temperature. This illustrated the benefits of low detection limit and high-accuracy simultaneous measurements brought about by combining CEAS with fs laser.