The Ben Ish Chai (Rav Yoseif Chaim ben Eliyahu 1835-1909) writes that the proper time for tashlich is on the first day of Rosh Hashana, after Mincha. So one would pray Mincha in the afternoon, like we pray Mincha Ketana, and then say tashlich. The question is why?
he Ben Ish Chai explains that this is the time that Adam Harishon sinned. Rosh Hashana commemorates the birthday of Adam Harishon who was created on Rosh Hashana, and then he sinned later that same day in the afternoon. So therefore the tashlich, which is a Tikun (repair), happens at the time of day when Adam made the sin. Also we know that Mincha is a time of Et Ratzon, as brought down that Eliyahu Hanavi was answered in the time of Mincha. And lastly we do it in Mincha because, since Akedat Yitzchak was on Rosh Hashana, and the prayer that Yitzchak Avinu instituted was Tefilat Mincha, so therefore to commemorate the Akeda, we do tashlich at Mincha time.
However, the Ben Ish Chai writes that in the event that we donít do tashlich on the first day of Rosh Hashana, for whatever reason, one would then do it on the second day of Rosh Hashana. However, the Ben Ish Chai holds that on the second day it should be done after Musaf. So there is a difference between day one and day two. Day one is after Mincha, and day two is after Musaf. It should be pointed out that Rav Chida (Rav Chaim Yoseif David Azulai, 1724-1806) argues with this and says it should always be said after Mincha. The Halacha as brought down by Chacham Ovadia is that if it is being done on the second day, it would be more proper to do it after Musaf.
The question is now, when the first day of Rosh Hashana falls out on Shabbat, is tashlich done on the first day, or on the second day Rav Chidaís opinion is that it is not done on the first day in this case. He holds that the Kabalah says that when Rosh Hashana comes out on Shabbat, it is not proper to do it on Shabbat. There is a concern in a place where there is no Erub, that the people might come to carry the Sidur. So therefore he says not to gather the people because itís likely that theyíre going to commit a sin of carrying and defeat the purpose as they came to throw away their sins. So therefore Rav Chida holds not to do tashlich when the first day comes out on Shabbat.
However, thatís not the opinion of the Ben Ish Chai, or of Rav Chayim Palachi, or of the Rashash (Rabbi Shalom Shar'aby Mizrahi), and it was not the custom in Bet El in Jerusalem. They used to make it on Shabbat, and they held that itís proper to do it on Shabbat. The Ben Ish Chai has a whole Teshuva on this in Torah Lishma. He says that when the first day of Rosh Hashana falls on Shabbat, that that is the best day to do tashlich. They too were concerned about the Erub issues, but stated that if you have an Erub at the place where you are making tashlich, and therefore since there wouldnít be a problem of people carrying, it would then be proper to make tashlich on the first day of Rosh Hashana occurring on Shabbat.
It should be pointed however, it seems the custom in our community, according to Chacham Baruch Ben Haim ZíL, is that they donít make tashlich in the year when it comes out on Shabbat because of the issues of carrying. However, this is leveled in Machloket (argument). Many Rabbis held that you are supposed to make tashlich. In conclusion one should folloe the Minhag of his community and consult his Rabbi.