Open-pollinated, self-pollinated, heirloom, and hybrid are all terms you'll hear fairly often, especially if you're buying seeds for your garden. And if you're interested in saving seeds or maybe trying to breed your own vegetable varieties, then all four are terms that you'll need to be familiar with. There is a lot of confusion around these terms, so let's clear some of that up so that you can buy and save seeds confidently.
Open-pollinated plants are those that require pollination by the wind, insects, or the gardener to set fruit and produce seeds. In some cases, the plant will produce both male and female flowers on the same plant, and it's just a matter of moving the pollen from a male flower to the stigma of a female flower -- squash and pumpkin plants are good examples of this. Open-pollinated plants come true from seed. This means that if you have a 'Musque de Provence' squash plant, and you've ensured that there was no cross-pollination with other plants in your garden, that the resulting seeds that you save from the crop you grow this year will produce 'Musque de Provence' squash plants, and not something else.
Self-pollinated plants are those that have what are called "perfect" flowers. This means that instead of needing pollen from one flower to move to the stigma of another, both the pollen and stigma are present in the same flower.
Often, all that's required for pollination is the act of the flower opening, which will transfer pollen to stigma. There is some evidence that plants self-pollinate better with the help of wind or from the gardener merely giving the plant a shake now and then to help the process along, but, in general, self-pollinated plants manage pretty well on their own.
Self-pollinated plants are often referred to under the umbrella term "open-pollinated" -- simply meaning that they are not hybrid plants.
Cross-pollination, which is much more common for open-pollinated plants, can still happen with self-pollinators. If a bee visits one tomato plant, then another, for example, it could end up cross-pollinating. If you're interested in saving seed and keeping the seed pure (and ending up growing exactly what you started with) then it's still a good idea to isolate self-pollinators, just to be sure. Examples of self-pollinators include tomatoes and peppers.
Heirloom plants are always open-pollinated (or self-pollinated -- the two are often lumped together, resulting in confusion) but not all open-pollinated plants are heirlooms. The term "heirloom" signifies a plant that has been around for at least 50 years (according to Seed Savers Exchange) and is open-pollinated or self-pollinated. So you'll see terms like "new heirloom!" in garden catalogs, and the term is absolute nonsense. How can something new be an heirloom? It can't. What this most often means is that the plant in question is a newly-introduced open-pollinated (or self-pollinated) variety.
A hybrid plant is one that is bred from two different types of plant. For an example, let's take a hypothetical F-1 hybrid tomato plant. Several years back, a plant breeder found one open-pollinated tomato plant that was very vigorous, but the fruit was nothing to write home about. And he also found an open-pollinated tomato plant that produced terrific fruit, but it was kind of a wimpy, low-production plant. He spends a few years working with the plants, separately, growing and selecting seed from those plants that show the best of its chosen attributes (in this case, vigor or flavor). Once he's satisfied that he has seeds from the very best of each type of plant, he grows them out, then cross-pollinates them. The resulting seed is an F-1 hybrid. F-1 hybrid seed is only produced by crossing two pure lines of other plants.
When you plant an F-1 hybrid seed, you get a plant that exhibits the attributes that the breeder spent all of those years perfecting.
That being said, saving seeds from an F-1 hybrid is pointless, because they don't come true from seed; the resulting plants from saved F-1 hybrid seed would be some version of one of the two pure open-pollinated varieties used in the breeding of the hybrid.