1. Length laws. North American truck length laws were changed to allow for longer trailers, including the 40 foot and plus shipping containers. With the old laws, COEs, with their shorter cab length could squeeze a few extra feet into the load carrying section of a semi-trailer truck combination. BBC or Bumper to Back of Cab dimensions were all important when the restrictive length laws were in place. Generally the shorter COEs also allowed room for a sleeper, which the longer wheelbase conventionals could not under the old laws., without sacrificing cargo capacity. Freightliner COEs were built of aluminum, and thus allowed for a greater payload. However weight laws changed too, so that edge was mostly lost.
2. Safety. With several feet of engine ahead of the cab, a conventional is inherently safer in a head on collision.
3. Comfort. COEs have what is called the dog house, which is in fact the engine compartment, projecting up into the middle of cab, leaving very little space for the driver or passenger. Also with the driver right over the front axle, the early COEs were very uncomfortable riding. However this was resolved in the later models that had excellent cab suspensions, and independently driver suspended seats. Still many COEs had a shorter wheelbase, and thus were less comfortable than the long wheelbase conventionals.
As to why Europe has COEs in abundance - they still have restrictive length laws. You'll note how close coupled the tractors and trailers were in my recent Germany photos. They have to squeeze as much truck as they can into the overall length, and so the shortest (i.e. smallest Bumper to Back of Cab dimension) means the longest trailer.
Safety takes the back seat to revenue, and head on collisions still take a tremendous toll in drivers lives in Europe.
So why the small resurgence of COEs in North America?
It seems to me to be mostly a matter of taste. Some drivers still prefer the COE for its better visibility, higher eye level (in some trucks) and in some cases, easier access to the sleeper.
Although perhaps a minor point, there is less sheet metal in a COE, and thus less weight. However with most conventionals these days wearing fiberglass tilt hoods, it may not be a factor.
Even with the new length laws, there are still length limitations. So steel haulers like CanAm and oversize carriers like Watson, can still find situations where a COE could get them in under a limitation which would otherwise require a special (costly) permit, or an escort car.