The Shulchan Aruch writes (27:2) that the "Yod" knot of the Tefillin Shel Yad must touch the box of the Tefillin, and not move away from the box. The Mishna Berura (commentary to the Shulchan Aruch by Rabbi Yisrael Meir Kagan, the "Chafetz Chayim," Lithuania, 1835-1933) adds that the Zohar (Parashat Pinchas) expresses particular vehemence in this regard, affording great importance to the requirement to keep the "Yod" knot attached to the box of one's Tefillin.
The Sha'arei Teshuva (work on the Shulchan Aruch by Rabbi Chayim Mordechai Margoliyot, Poland, 18th-early 19th century) cites a work entitled "Shalmei Tzibur" by Mahari Tzemach as requiring that the knot be attached to the box at all times, even when the Tefillin is not being worn. Even while the Tefillin sits in its bag, the knot must be firmly attached to the box of the Tefillin Shel Yad.
Since the knot tends to move away from the Tefillin Shel Yad, the custom developed to have a Sofer (scribe) wrap a "Gid" (special kind of string) around the knot to hold it in place. The Sha'arei Teshuva cites a number of authorities who objected to this practice, because the "Gid" would often be wrapped underneath the box of the Tefillin Shel Yad, and thus constituted a "Hatzitza" (interruption) between the Tefillin and one's skin. Nowadays, however, Sofrim wrap the "Gid" around the box, and not underneath it, such that it does not come between the Tefillin and one's skin. Accordingly, the Kaf Ha'chayim (by Rabbi Chayim Palachi, Turkey, 1788-1868), as well as Chacham Bentzion Abba Shaul (Jerusalem, 1924-1998) advise using a "Gid" to hold the knot firmly in place alongside the box of the Tefillin.
Summary: The "Yod" knot of the Tefillin Shel Yad must be attached to the box of the Tefillin; therefore, one must either have a Sofer wrap a special string around the knot to hold in place, or occasionally check his Tefillin to ensure that the knot has not moved out of place.
