There is no consistent distinction between cormorants and shags. The names "cormorant" and "shag" were originally the common names of the two species of the family found in Great Britain, Phalacrocorax carbo (now referred to by ornithologists as the Great Cormorant) and P. aristotelis (the European Shag). "Shag" refers to the bird's crest, which the British forms of the Great Cormorant lack. As other species were discovered by English-speaking sailors and explorers elsewhere in the world, some were called cormorants and some shags, depending on whether they had crests or not. Sometimes the same species is called a cormorant in one part of the world and a shag in another, e.g., the Great Cormorant is called the Black Shag in New Zealand (the birds found in Australasia have a crest that is absent in European members of the species). Van Tets (1976) proposed to divide the family into two genera and attach the name "Cormorant" to one and "Shag" to the other, but this flies in the face of common usage and has not been widely adopted.

The scientific genus name is latinized Ancient Greek, from φαλακρός (phalakros, "bald") and κόραξ (korax, "raven"). This is often thought to refer to the creamy white patch on the cheeks of adult Great Cormorants, or the ornamental white head plumes prominent in Mediterranean birds of this species, but is certainly not a unifying characteristic of cormorants. "Cormorant" is a contraction derived either directly from Latin corvus marinus, "sea raven" or through Brythonic Celtic. Cormoran is the Cornish name of the sea giant in the tale of Jack the Giant Killer. Indeed, "sea raven" or analogous terms were the usual terms for cormorants in Germanic languages until after the Middle Ages. The French explorer André Thévet commented in 1558 that "...the beak is similar to that of a cormorant or other corvid," which demonstrates that the erroneous belief that the birds were related to ravens lasted at least to the 16th century.

The word cormorant is pronounced /ˈkɔrmərənt/, with the stress on the first syllable.