Who was the first to make black a sad colour?

Black people often have different shades of brown and dark brown and darker brown, but never really black. Why we call them black people then, I don't know. Even in Pakistan we call people with light brown to dark brown skins, "kalay bandi", while they never really are black.

In my family we have some people with different shades, varying from almost white to very dark brown, but nobody really black, yet some cousins, females and males, who have either light brown or dark brown skin are sometimes called "kala" or "kalee" too.

But I'm not talking about skin colour here, I'm talking about fabrics, clothes and all kinds of things with the colour black. Skin colour is a different topic.