tolkiens legendarium - Who in Middle-earth knows the Istari's origin?

In the Middle-earth universe, there are a handful of wizards who work in various ways to fight evil. Out of universe, we know these wizards to be Istari, Maia who are in Middle-earth working to defeat Sauron, and we know this thanks to The Silmarillion which explains the origins of Middle-earth.

In universe though, I'd expect this sort of knowledge to be pretty rare. Do we know how rare it is? Who in Middle-earth knows that these wizards are Istari, and the ramifications of this? I'm not looking for a list of specific individuals here, I seek more a list of groups of people.

For example, I'd expect the Noldor who remain in Middle-earth to know what's up with the Istari, since it's part of their personal past. But do any dwarves know? Do any humans? What about Rohan's royalty? The Stewards of Gondor?

Answer

The only people (and yes, it's individuals, not groups) who are recorded as having known were Círdan, Elrond and Galadriel; from Rings of Power:

Even as the first shadows were felt in Mirkwood there appeared in the west of Middle-earth the Istari, whom Men called the Wizards. None knew at that time whence they were, save Círdan of the Havens, and only to Elrond and to Galadriel did he reveal that they came over the Sea.

The Istari essay in Unfinished Tales establishes that it was quite some time before Círdan even told the other two:

For they came from over the Sea out of the Uttermost West; though this was for long known only to Círdan, Guardian of the Third Ring, master of the Grey Havens, who saw their landings upon the western shores.

The name "Olórin" was, however, well-known in Gondor, per Faramir's statement in the Two Towers:

'Mithrandir we called him in elf-fashion,' said Faramir, 'and he was content. Many are my names in many countries, he said. Mithrandir among the Elves; Tharkun to the Dwarves; Olórin I was in my youth in the West that is forgotten, in the South Incanus; in the North Gandalf; to the East I go not.'

Although Tolkien notes that '"The West" here plainly means the Far West beyond the Sea, not part of Middle-earth; the name Olórin is of High-Elven form', it is not established whether or not those in Gondor had made the inevitable connection.