The wisdom of crowds comes not from the consensus decision of the group, but from the aggregation of the ideas/thoughts/decisions of each individual in the group.

At its simplest form, it means that if you take a bunch of people
and ask them (as individuals) to answer a question, the average of each
of those individual answers will likely be better than if the group works together to come up with a single answer. And he has a ton of real examples (but you’ll just have to read the book for them ; )

[Also] diversity increases the quality of the aggregated wisdom of the group.
If you have too many people who are alike, then no matter how smart
they all are, they may not come up with the same quality of answer than
if you have less smart folks who have a very different point of view. Diversity brings new information. And that new information is valuable.

In order for the crowd to have wisdom, the crowd has to be made up of individuals who argue! Or as he puts it in the book,

and independence are important because the best collective decisions
are the product of disagreement and contest, not consensus or
compromise. An intelligent group, especially when confronted with
cognition problems, does not ask its members to modify their positions
in order to let the group reach a decision everyone can be happy with.
Instead, it figures out how to use mechanisms–like market prices, or
intelligent voting systems–to aggregate and produce collective
judgements that represent now what any one person in the group thinks
but rather, in some sense, what they all think.”

And my favorite line that sums it up:

“Paradoxically, the best way for a group to be smart is for each person in it to think and act as independently as possible.”