For a while, Google gave you good boy points for including microformats, and they still offer tests and validators [0] to tell you what the crawlers get out of your page. Supposedly microformats would not just give you better SEO ranking but also help Google connect people (like the fediverse) to accounts, so that you could surface things relevant to person by searching for the person.
If you go back to the time when they were invented, many semantic elements, like article or footer, didn't exist in HTML. People tried to find conventions and efforts like microformats were an attempt to standardize those when the best solution (updating the HTML standard) was difficult. In terms of timing, it's worth looking at the arc of Firefox, WHATWG, the advent of Safari and Chrome, and table use for layout.
Google was a driver in practice. Accessibility and better web experiences were important to those involved. The reality was that people interested in this area were at the bleeding edge. Many people still held onto tables for site layout and Flash was still a default option for some in the period when microformats emerged.
ARIA and accessibility microformats were separate from the ones the fediverse was excited about (and thus the GP was talking about)—things like hCard for identifying people, places, and things. Accessibility is useful to many people, but hCard et al. were probably never really useful to anybody other than Google. Still, many of us back then were obsessive-compulsive about using them in the hope that one day computers would better be able to understand authoritative information about identities and relationships between identities. I still have microdata on my personal web page.
I don't think anyone desires fragmentation. It's just the reality of the space. People were exploring options but didn't have support from the key stakeholders who were the browser makers (IE was at its peak) and Google. Firefox and WHATWG advanced some of the ideas in time.
People always mention RDF when the semantic web comes up. It's really important to understand where W3C was in the early-2000s and that RDF was driven by those with an academic bent. No one working with microformats was interested in anything beyond the RDF basics because they were too impractical for use by web devs. Part of this was complexity (OWL, anyone?), but the main part was browser and tool support.
> People always mention RDF when the semantic web comes up.
There's nothing wrong with RDF itself, the modern plain-text and JSON serializations are very simple and elegant. Even things like OWL are being reworked now with efforts like SHACL and ShEx (see e.g. https://arxiv.org/abs/2108.06096 for a description of how these relate to the more logical/formal, OWL-centered point of view).
