The famous rule of "Tadir Ve’she’eno Tadir, Tadir Kodem" establishes that when we have two Misvot to perform, we first observe the more frequent of the two Misvot, and we observe the less frequent Misva afterward. A classic example of this rule is the recitation of Birkat Ha’mazon when Shabbat and Rosh Hodesh coincide. We insert two additions in the third Beracha of Birkat Ha’mazon, the Beracha of "Rahem" – we add "Reseh" for Shabbat, and we add "Ya’aleh Ve’yabo" for Rosh Hodesh. Since Shabbat occurs far more frequently than Rosh Hodesh – every week, as opposed to every month – the recitation of "Reseh" is a more frequent Misva than the recitation of "Ya’aleh Ve’yabo," and so we first recite "Reseh" and then "Ya’aleh Ve’yabo."
Nevertheless, if one mistakenly recited "Ya’aleh Ve’yabo" before "Reseh," he has satisfied his obligation, and does not need to repeat "Ya’aleh Ve’yabo" after "Reseh."
The Bet Yosef (work by Maran, Rav Yosef Karo, author of the Shulhan Aruch) notes that an exception to this rule is made on Hanukah. On Shabbat Hanukah, we must recite both "Al Ha’nissim" for Hanukah and "Reseh" for Shabbat. However, "Al Ha’nissim" is recited at an earlier point in Birkat Ha’mazon – during the second Beracha, the Beracha of "Nodeh," when we thank Hashem for all that He has given us. This is the appropriate place to recite "Al Ha’nissim," thanking Hashem for the great miracle He performed. It thus turns out that although Shabbat is more frequent than Hanukah, nevertheless, "Al Ha’nissim" is recited earlier in Birkat Ha’mazon than "Reseh," since the appropriate place for "Al Ha’nissim" is the second Beracha of Birkat Ha’mazon, whereas "Reseh" is recited in the third Beracha. "Reseh" and "Ya’aleh Ve’yabo," however, are both recited in the same Beracha, and so we can apply the rule of "Tadir" and recite "Reseh" first, since it is more frequent.
Another exception to this rule is noted by Rav Yaakov Kastro (Egypt, 1525-1610). When Shabbat and Yom Tob coincide, we recite the Amida of Yom Tob, adding special insertions for Shabbat ("Et Yom Ha’Shabbat Ha’zeh"; "Reseh Na Ve’hahalisenu"; etc.). Instead of reciting the standard Amida for Shabbat, and adding special insertions for Yom Tob, we instead recite the standard Amida for Yom Tob, and add special insertions for Shabbat. Seemingly, Rav Yaakov Kastro pointed out, the law of "Tadir," which instructs that precedence should be given to the more frequent Misva, should require us to give precedence to Shabbat over Yom Tob, such that we should recite the Amida for Shabbat. And yet, we do just the opposite, reciting the Amida for Yom Tob. Rabbi Yaakov Kastro explained that on Shabbat and Yom Tob, the Sages instituted an Amida with only seven blessings – the three standard blessings at the beginning, the three standard blessings at the end, and one in the middle. Therefore, we cannot recite separate Berachot for Shabbat and Yom Tob, such that we could then recite the blessing for Shabbat first. Instead, we must choose which text to recite, and therefore, due to the fact that Yom Tob is more "Habib" ("beloved," or "dear"), as it is observed more rarely, we prefer reciting the Amida of Yom Tob. This is much different from Birkat Ha’mazon, when we recite two separate paragraphs – one for Shabbat and one for Yom Tob – and so we arrange them according to the principle of "Tadir." In the case of the Amida, we give precedence to the factor of "Habib," over the factor of "Tadir," marking an interesting exception to this rule.
This discussion reminds us that not only are the words of the prayer service significant, but even the sequence and arrangement of the prayer text are laden with meaning and significance. The sequence in which we recite the various prayers is no less important than the text itself, and we must therefore ensure to conduct our prayers in precise accordance with Halacha and our ancient traditions.
Summary: When both "Reseh" and "Ya’aleh Ve’yabo" must be recited in Birkat Ha’mazon, we recite "Reseh" before "Ya’aleh Ve’yabo." Nevertheless, one who mistakenly reversed the sequence, and recited "Ya’aleh Ve’yabo" before "Reseh," is not then required to repeat "Ya’aleh Ve’yabo" after "Reseh."