From the link in my other comment, cultural differences would seem to explain the Czechs’ [lack of tolerance.](https://www.expats.cz/czech-news/article/deciphering-dairy)
Same with First Nations in the colonies. There's a good chance that the first viking settlers were attacked and driven out because they were gifted cheese and got so sick they thought they were poisoned.
But losing the enzymes to digest lactose tends to be an exposure issue. A lot of Asians are lactose intolerant for this reason.