This paper focuses on the design of medium access control protocols for cognitive radio networks. The scenario in which a single cognitive user wishes to opportunistically exploit the availability of empty frequency bands within parts of the radio spectrum having multiple bands is first considered. In this scenario, the availability probability of each channel is unknown a priori to the cognitive user. Hence efficient medium access strategies must strike a balance between exploring (learning) the availability probability of the channels and exploiting the knowledge of the availability probability identified thus far. For this scenario, an optimal medium access strategy is derived and its underlying recursive structure is illustrated via examples. To avoid the prohibitive computational complexity of this optimal strategy, a low complexity asymptotically optimal strategy is developed. Next, the multi-cognitive user scenario is considered and low complexity medium access protocols, which strike an optimal balance between exploration and exploitation in such competitive environments, are developed.