It is the custom of the Sepharadim to conduct the "Birkat Kohanim" service in the synagogue each morning during the Hazan’s repetition of the Amida during Shaharit. The Kohanim go to the front of the synagogue, and, after the Hazan concludes the Beracha of "Ha’tob Shimcha U’l’cha Na’eh Le’hodot," the Kohanim bless the congregation with the Birkat Kohanim, repeating word-for-word after the Hazan.
The Shulhan Aruch writes (Orah Haim 128) that the congregation should answer "Amen" after each of the three verses of Birkat Kohanim. Meaning, they should respond "Amen" after the Kohanim recite the words "Ve’yishmerecha," "Vi’huneka," and "Shalom." Rav Moshe Feinstein (Russia-New York, 1895-1986) writes in his Iggerot Moshe that if there is no one in the congregation to answer "Amen," then Birkat Kohanim should not be recited at all. Rav Moshe cites a comment of the Mabit (Rabbi Moshe of Trani, 1505-1580) who claimed that just as the Torah obligates the Kohanim to bless the people, similarly, there is a Torah obligation for the listeners to respond "Amen" to the blessing. Therefore, everyone in the congregation must ensure to pay attention to Birkat Kohanim and answer "Amen" after each verse.
There is a debate among the Halachic authorities as to whether or not one should answer "Baruch Hu U’baruch Shemo" after the Kohanim recite God’s Name in Birkat Kohanim. God’s Name appears in each of the three verses of the Beracha ("Yevarechecha Hashem"; "Ya’er Hashem"; "Yisa Hashem"), and the question thus arises as to whether one should answer "Baruch Hu U’baruch Shemo" just as we do when we hear Hashem’s Name recited in an ordinary Beracha. The custom of the Ashkenazim is not to answer "Baruch Hu U’baruch Shemo" when hearing God’s Name in Birkat Kohanim. They declare this response only when they hear God’s Name mentioned in an ordinary Beracha. However, Rav Haim Palachi (Turkey, 1788-1869), in his Kaf Ha’haim, writes that the custom of the Sepharadim is to answer "Baruch Hu U’baruch Shemo" even during Birkat Kohanim. This is, indeed, the proper practice for Sepharadim to follow. However, Hacham Ovadia Yosef writes that this response should preferably be recited in a whisper. If everyone in the congregation responds "Baruch Hu U’baruch Shemo" out loud, this might cause confusion for the Hazan and the Kohanim, and this response should therefore be made in a soft whisper. The response of "Amen," by contrast, should be said in a loud voice.
Summary: The congregation must answer "Amen" after each verse of Birkat Kohanim. The custom of the Sepharadim is to answer "Baruch Hu U’baruch Shemo" upon hearing God’s Name in Birkat Kohanim, though this response should be made in a whisper, and not aloud.
