We consider the problem of classifying an unknown concept into one of two subclasses of concepts. Specifically, if C is a concept class and C0 and C1 are two disjoint subsets of C, given an unknown c ∈ C0 in union with C1 we wish to decide whether c ∈ C0 or c ∈ C1 based on a set of random examples. We consider both uniform and non-uniform probably correct classification for which the number of samples is or is not required to be independent of c, respectively. For both cases, we obtain necessary and sufficient conditions on C0 and C1 that allow probably correct classification. The conditions obtained are in terms of separability and/or coverability conditions on the classes C0 and C1. Furthermore, in the non-uniform case we show that this is equivalent to classification in the limit. Several examples of the applicability of our results are also provided.