The Gemara in Masechet Kiddushin tells that Rav Kahana would accept gifts in lieu of the Pidyon Ha’ben payment. The clear assumption underlying this account is that Rav Kahana was a Kohen. Tosafot, however, note that we know from other sources that Rav Kahana was not, in fact, a Kohen, thus giving rise to the question of why he was accepting payments for Pidyon Ha’ben. The first answer suggested by Tosafot is that there were two Rabbis named Rav Kahana, one of whom was a Kohen and one of whom wasn’t. Tosafot’s second answer is that Rav Kahana was not a Kohen, but he was married to a Kohenet (the daughter of a Kohen), and he was accepting these gifts on his wife’s behalf for the Misva of Pidyon Ha’ben.
It clearly emerges from Tosafot that in their view, the Pidyon Ha’ben payment may be given even to a Kohenet, and does not have to be given specifically to a male Kohen.
The Shulhan Aruch (Yoreh De’a 305:2), however, based on the implication of the Rambam’s ruling in Hilchot Bikkurim (1), writes that the money may not be given a Kohenet, and must be given specifically to a male Kohen. This is the accepted Halacha. The Hatam Sofer (Rav Moshe Sofer of Pressburg, 1762-1839), in one of his responsa (Yoreh De’a 301), writes that if no male Kohen is available, one may perform the Pidyon by giving the money to a Kohenet, and in such a case the transaction should be made on the condition that a male Kohen is not found. Needless to say, this is not very common in the vast majority of Jewish communities in the world today.
A minority view among the Poskim (the Yabetz) maintained that nowadays, since we cannot be certain that one who is assumed to be a Kohen is truly a Kohen, a Kohen should always return the Pidyon Ha’ben payment after receiving it. Given the possibility that he is not actually a Kohen, and thus does not have rights to this money, he should return it to the father. Others, however, disagree. The Aruch Ha’shulhan (Rav Yechiel Michel Epstein of Nevarduk, 1829-1908) vehemently disputes this position, and writes that we rely on the Hazaka (presumption) that those with a family tradition of descending from Kohanim are, indeed, Kohanim, and it is improper to call this Hazaka into question. Therefore, a Kohen does not have to return the money.
Summary: A father must give the Pidyon Ha’ben money to a Kohen, and not a Kohenet (daughter of a Kohen). The money is given to someone who is presumed to be a Kohen despite the fact that we cannot definitively ascertain whether all those who are presumed to be Kohanim are, in fact, Kohanim.
