Capsaicin (pronounced "cap-SAY-a-sin") is the chemical in chili peppers that makes them spicy. Specifically, capsaicin occurs in the fruits of plants in the Capsicum family, including bell peppers, jalapeño peppers, cayenne peppers and other chili peppers.
Capsaicin in chili peppers is measured on the Scoville Scale and expressed in terms of Scoville Heat Units. Bell peppers are the only member of the capsicum family that don't contain capsaicin, and thus register zero Scoville units.
Besides being the source of the heat, or pungency, in chili peppers, capsaicin will cause a burning sensation in any part of the skin or other tissues it contacts. Thus, when a cook is working with cut chili peppers, the capsaicin from their hands can burn their eyes if they should rub their eyes.
The white membranes inside a pepper contain the most capsaicin, and the actual flesh of the pepper contains less. The seeds of the pepper don't contain any capsaicin at all.
Capsaicin may also stimulate the production of endorphins, which is why some people report experiencing a sense of euphoria when eating spicy foods.
Capsaicin is an oil-like compound in the sense that it repels water. Therefore, drinking water to soothe the burning caused by eating chilis isn't particularly effective, other than the cooling effect if the water happens to be cold. Capsaicin is soluble in milk and alcohol, however.
So a sip of cold milk, or to a lesser extent, a cold alcoholic beverage, can soothe the burning feeling from capsaicin.
Interestingly, while all mammals are sensitive to capsaicin, making it unappealing to rabbits and other such garden pests, birds are immune to its effects.
Capsaicin has a number of non-culinary applications, including as a pain reliever and as the active ingredient in pepper spray.