You're right: Publishers do tend to assume that a vast social media following gives them a better shot at a decent ROI. And they're often wrong.
As you say, they just don't know a lot of the time. I wrote about this not long ago in a post called "How Social Media Can--And Can't--Help You Sell Books."
An example I gave was that of a recent book by Billie Eilish who had something like 97 million Instagram followers when her book sold. It should have been a surefire hit.
But--by today's standards--it sold only a modest number and vastly underperformed books by less famous names. So much of what makes a book sell is (and always has been) a lottery.