Harbor seals may live a maximum of about 25 to 30 years. Males tend to have a shorter lifespan, possibly due to the stress of fighting during breeding seasons.

Pup mortality is about 21% in the first year; not unusually high compared to most species of animals in the wild. Pups may starve, be abandoned, wash away from pupping areas during high seas, or become ill or injured.

As a harbor seal ages, it periodically produces growth layer groups of dental material. Age can be estimated by examining a sliced section of a tooth and counting these layers.

Harbor seal adults and pups can be preyed upon by killer whales, sharks, polar bears, Steller sea lions, walruses, coyotes, and eagles.

In the 1900s, fur traders hunted harbor seal pups for the fine coats they have when they are less then four weeks old.

Harbor seals were hunted by salmon fishermen who viewed the seals as competitors for fish. Hunting was so extensive that many harbor seal populations abandoned traditional haul-out areas.

Marine debris is a threat to harbor seals. They can become entangled in nylon fishing nets or plastic packaging materials, causing severe injury or drowning. Harbor seals also ingest plastic debris, which can cause starvation or obstructions in the digestive tract.

Indigenous Arctic peoples legally hunt harbor seals for food, clothing, and other raw materials. For centuries, hunting harbor seals has been an important part of their culture and traditions.

In some areas, industrial run-off has resulted in high levels of toxic chemicals, especially polychlorinated biphenyls (PCBs) and dichlorodiphenyltrichloroethane (DDTs), to be present in harbor seals. High levels of these chemicals may be associated with immune dysfunction.

Harbor seals are susceptible to a number of diseases and parasites.