The Shulhan Aruch (Orah Haim 214) emphasizes the importance of answering "Amen" whenever one hears any Beracha, or even when hearing an informal blessing, such as a "Mi’sheberach" prayer. Even if one hears a Beracha recited by a child, and the child is at least six years of age and understands that he recites a Beracha to G-d, he must answer "Amen" to the Beracha.
Rav Mordechai Yaffe (1530-1612), author of a classic, ten-volume set of halachic works known as the "Lebush," tells a famous story in his introduction to the first volume that demonstrates the vital importance of answering "Amen" to each and every Beracha which one hears. He writes that he was invited to serve as a Rabbi in a certain city, but he declined the offer because he felt that before accepting such a post, he should first study the "Sod Ha’ibur" – the topic of Jewish leap-years. The only one whom he felt could properly and thoroughly teach him this subject was the great Sephardic sage, Rav Yishak Abuhab, and so he went to Rav Abuhab and became his close student.
One day, a child entered the room, took out a fruit, and recited the Beracha of "Boreh Peri Ha’etz" over the fruit. Rav Yaffe did not answer "Amen," probably because he was preoccupied. Rav Abuhab immediately declared "Nidui" (excommunication") upon Rav Yaffe and sent him away. In accordance with the laws of "Nidui," Rav Yaffe was not permitted to see Rav Abuhab for one month. After the month passed, Rav Yaffe – who did not even know what he had done to deserve "Nidui" – returned to Rav Abuhab and, with tears in his eyes, asked what he had done wrong. Rav Abuhab explained that he had neglected to answer "Amen" to a child’s Beracha, and because of this, a death sentence was issued against him at that time. He told Rav Yaffe that he could be forgiven if he would commit to always answer "Amen" to every Beracha he hears, and that whenever he delivered a lecture, he would mention to the audience the importance of answering "Amen" to every Beracha. Rav Yaffe agreed, and so wherever he went, he included in his lectures a warning about the importance of reciting "Amen."
Rav Abuhab also told Rav Yaffe a frightening story of an outstanding, pious individual who once neglected to answer "Amen" to a child’s Beracha, on account of which he endured great suffering, Heaven forbid.
Answering "Amen" is so important that Rav Abuhab felt it appropriate to cancel his learning with Rav Yaffe for an entire month, and to demand a lifelong commitment to rectify the mistake, when Rav Yaffe missed a single "Amen" response.
Hacham Ovadia Yosef mentions this story in his Yabia Omer (vol. 2), and notes that this child must have been at least six years of age, because it is only from this age that a child’s Beracha requires an "Amen" response.
Due to the severity of neglecting to answer "Amen," some Poskim advise that if one who is reciting a Beracha finds himself among people who he knows will not answer "Amen," it is preferable to recite the Beracha quietly, so they will not be guilty of this infraction. Otherwise, one should specifically make a point of reciting Berachot aloud in order to offer the people around him the opportunity to earn the great merits of answering "Amen."
Summary: It is very important to answer "Amen" to every Beracha which one hears, and it is considered sinful to neglect to answer "Amen." Even if one hears a Beracha recited by a child, and the child is at least six years of age and understands that he recites a Beracha to G-d, he must answer "Amen" to the Beracha. Therefore, if one is reciting a Beracha and finds himself around people who are not likely to answer "Amen," he should preferably recite the Beracha quietly, so the people around him will not be guilty of neglecting to answer "Amen." In general, however, one should specifically make a point of reciting Berachot audibly, in order to offer the people around him the precious opportunity to earn the great merit of answering "Amen" to a Beracha.
