The days of Hol Ha'mo'ed the period between the first and last days of Pesach, and between the first day of Sukkot and Shemini Atzeret are referred to in the Torah as "Mikra Kodesh" holy occasions. We describe them with this term in the Musaf prayer on Hol Ha'mo'ed, as well. Although they are not the same as Yom Tov, they are nevertheless designated as special, sacred days and must be observed as such.
For this reason, a number of Halachot apply regarding the proper observance of Hol Ha'mo'ed. As Chacham Ovadia Yosef writes (Chazon Ovadia Laws of Pesach, p. 158; listen to audio for precise citation), on Hol Ha'mo'ed one should increase the amount of time spent studying Torah. The Torah obligates us to rejoice on the festivals, and Torah learning brings a person a special kind of joy. Furthermore, certain restrictions apply with regard to working on Hol Ha'mo'ed. It is also proper to wear one's Yom Tov clothing on Hol Ha'mo'ed (as Chacham Ovadia cites from the Sefer Yerei'im and Shibolei Ha'leket), and to eat special meals with bread each day and night of Hol Ha'mo'ed. The special obligation of Simcha (joy) requires that men partake of men and wine and women wear special festive clothing and jewelry.
The Talmud (Sanhedrin 109) speaks very harshly about those who "disparage the festivals." Rabbi Ovadia of Bartenura (1445-1524), in his commentary to the Mishna, explains this passage as referring to those who treat Hol Ha'mo'ed as ordinary weekdays, rather than celebrating them as special days of joy and Torah study, as discussed. One must therefore ensure to properly observe Hol Ha'mo'ed through the practices described above.
A very important comment in the Talmud Yerushalmi (Mo'ed Katan 2:3) puts into proper perspective the desired nature of the Hol Ha'mo'ed observance. Rabbi Abba Bar Mamal is cited as saying that if he had the support of his colleagues, he would suspend the prohibition against working on Hol Ha'mo'ed. This prohibition was enacted solely for the purpose of allowing people to spend Hol Ha'mo'ed enjoying festive meals and studying Torah. But instead, Rabbi Abba observed, people use the free time for frivolous and meaningless activities. If this is how people spend Hol Ha'mo'ed, Rabbi Abba laments, it would be better for them to go to work and involve themselves in constructive activity.
Chacham Ovadia Yosef infers from this Gemara that frivolous behavior on Hol Ha'mo'ed is worse than working on Hol Ha'mo'ed. Rabbi Abba was prepared to allow people to work in order to prevent them from engaging in non-constructive, foolish activities, and he thus evidently saw frivolity as a worse infringement upon the honor of Hol Ha'mo'ed than work.
It is both ironic and unfortunate that in the Jewish world today Hol Ha'moed has become a time for just that frivolous entertainment and recreation. Rather than observing Hol Ha'mo'ed as a time for eating special meals and engaging in Torah, many Jews instead go on recreational trips and to all kinds of events. We must remember that Hol Ha'mo'ed is a "Mo'ed Katan," a quasi-Yom Tov, and should therefore be observed with special meals, special clothing, and Torah study.
Summary: Hol Ha'mo'ed must be observed as a special occasion, similar to Yom Tov, with special meals, Shabbat clothes, and Torah study. It is improper to spend Hol Ha'mo'ed only for going on recreational trips and the like.