You've put your finger on an issue I didn't have room to deal with (except in my brief note at the beginning that reviewers may not have enough experience with the subject of a book).
You're right that can be harder to find a single trusted reviewer if you have eclectic tastes that range over many subjects. This seems to be a particular problem if you're interested in books about math or science; some critics just don't have the needed expertise.
Back when most big newspapers had book sections, their editors would look for (and pay) freelancers with expertise in certain areas. I was certainly glad to pay them. But now those sections are mostly gone, alas.