The Shulhan Aruch, in Siman 345, based on the Gemara, delineates the four different Reshuyot, or domains, that are relevant to the laws of Shabbat. These are Reshut Ha’yahid (a private domain) and a Reshut Ha’rabim (public domain), and the two categories of Karmelit and Mekom Petur which were instituted by Hazal. The Shulhan Aruch writes that a private domain is defined as a place with an area of at least 4 X 4 Tefahim (handbreadths) surrounded by walls that are at least ten Tefahim high. A furrow or mound with these dimensions would also qualify as a Reshut Ha’yahid.
Later (in Halacha 4), the Shulhan Aruch addresses the situation where there is a hole in a wall of a Reshut Ha’yahid, and he rules that the hole is regarded as part of the Reshut Ha’yahid. The Mishna Berura (commentary by Rav Yisrael Meir Kagan of Radin, 1839-1933) explains that although the hole is open to both the public domain and to the private domain, it is nevertheless considered as part of the private domain. Therefore, if an object is situated in the hole, it would be forbidden to take it from the hole outside on Shabbat, as this would amount to carrying an object from a private domain to a public domain on Shabbat. It would likewise be forbidden to insert an object into the hole from the public domain. The Mishna Berura notes that although Tosafot (Medieval French and German Talmudists) consider such a hole a Reshut Ha’yahid only if contains an area of 4 X 4 Tefahim, others (Rashba, Meiri) disagree, and the Shulhan Aruch follows this stringent view.
A modern application of this Halacha is placing a key in the keyhole of a front door on Shabbat. Many people circumvent the prohibition of carrying in a public domain by hiding the key somewhere near the door, or wearing it on their belt or tie such that it becomes part of their attire and they are thus not actually “carrying” it. However, even if they resolve the problem of carrying the key through the Reshut Ha’rabim, another issue arises concerning placing the key in the keyhole to open the door. If the keyhole is open on both sides of the front door, it would seem to have the status of a hole in the wall of a Reshut Ha’yahid, and thus one who stands outside and places the key into the keyhole would be in violation of the prohibition against transferring objects from a public domain to a private domain on Shabbat. This would certainly be the case if the front door opens directly to the public domain. But even if there is a porch in front of the door, placing the key in the keyhole may still be prohibited. Depending on its height and dimensions, a porch can be considered either a Reshut Ha’yahid or a Karmelit. If it is considered a Reshut Ha’yahid, then of course it is entirely permissible to place a key into the keyhole, since one is not transferring the key from a public domain to a public domain. A Karmelit, however, is regarded as a Reshut Ha’rabim on the level of Rabbinic enactment, and thus if the porch constitutes a Karmelit, it would be forbidden to place a key into the keyhole.
Therefore, the Shemirat Shabbat Ke’hilchatah (Rav Yehoshua Neubert, contemporary Halachic authority in Israel) rules that if one’s front door opens into the public domain or into a Karmelit, one must open the lock without detaching the key from his body. Meaning, he should continue wearing the key on his tie or belt and turn the lock while the key is still attached, such that the key is considered part of his attire and is thus not actually transferred to a different domain. A different solution is proposed by the Minhat Yishak (Rav Yishak Weiss, 1902-1989), who advised placing a covering over the lock from the inside of the house. Once the hole is not open all the way through the door, it does not have the status of a Reshut Ha’yahid, and it is then permissible to place a key in the keyhole from outside.
Summary: If a keyhole in the front door to a building is open all the way from one side of the door to the other, then it is forbidden on Shabbat to place a key in the keyhole while standing outside in a public domain. (If there is a porch around the front door, it may be considered a public domain, depending on its dimensions.) One should therefore either insert the key while it is attached to his garment, or ensure to place a covering over the lock from the inside, so that the hole does not go all the way through the door.