What kept the Shadows from attacking Babylon 5?

According to the 3rd and 4th seasons the Shadow ideology was about bringing out the best of the younger races by provoking conflict between them (or something along those lines). But at the beginning of the series those younger races were a hair's breadth from clashing with each other, and even one of the Shadow ships could likely have cut B5 in two with a single shot (I'm basing this on how they were seen wiping out Narn battle stations and warships so easily with little, if any damage in return).

So what stopped them from doing it? Were they still recollecting their forces? Did they not want to tip their hand too soon? Were they just not informed of current events?

Answer

(Please pardon the spoiler blocks.)

Early on,

Sheridan was seen as a possible ally. During ‘Z'ha'dum’, when he asks straight–up why the Shadows hadn't killed him yet, Justin informs Sheridan that it would be unproductive to make him a martyr.
Until that time, during the first three seasons, the Shadows weren't interested in B5 — they wanted Sheridan. It was only then that they were preparing to attack B5 directly — until the battlecrabs were called to fall–back, in response to Sheridan's own attack on Z'ha'dum with the thermonuclear weapons carried by the first White Star.

So, there we have one explanation. What about later?

Soon after — during the beginning of the fourth season, — the Shadows and the Vorlons begin cleansing the galaxy of those who've shown allegiance to the other. The Vorlons are first targetting planets which have held, or are holding, contingents of Shadow forces.
The Shadows, meanwhile, are playing a game of scatter–and–entrench. Then, when they are ready to move, they begin first attacking a few worlds which were faithful to the Vorlons. By the time ‘Into the Fire’, episode 06, comes along, the Shadows and Vorlons are going at it direct, but only because they were lured there by the White Star fleet.
They do yet want the younger races to side with their beliefs: both the Shadows and the Vorlons only attack those who have already chosen the other side.

Now, what does Babylon 5 mean to either the Shadows or the Vorlons for those episodes of season four?

Babylon 5 did have a Vorlon ambassador for quite some time, but they themselves ousted him during ‘Falling Towards Apotheosis’.

The Vorlons also knew that the Shadows had been using ever since Kosh first encountered Morden there in the first season.
So, why didn't either of them take moves to eliminate Babylon 5 when they began cleansing the galaxy during season four?

It is possible that the Great Machine in Epsilon III served as a deterrent, but we never see any evidence that its defensive weapons were used against either the Shadows or the Vorlons. That may be simply because the situations were not dire enough so as to require the intervention. We certainly don't see evidence of it intervening when the Shadows took Garibaldi.
It is also possible that neither Shadows nor Vorlons knew of the Machine. This is unlikely, however, given at least the fact that Susan Ivanova seemingly alerted the Shadows of its presence, and her use of it, during ‘Voices of Authority’, season 3, episode 05. It isn't clear exactly what is occuring during that time, and whether she is traversing some transspacial network or simply probing the databanks of the Great Machine, but Ivanova does seem to believe that something noticed her.
It would seem reasonable to conclude that the Vorlons would've learned what it was too, but we can't be certain. Nevertheless, it also seems reasonable to surmise that the Great Machine, if it would've posed a threat to either the Vorlons or the Shadows, simply would've increased the necessary firepower from either for them to do defeat Babylon 5.

So, I would argue that the fact that neither Shadows nor Vorlons made a move on Babylon 5, prior to when Sheridan and Delenn prompted them to notice the fleet of Non-Aligned Worlds which had gathered in ‘Into the Fire’, signifies that it wasn't a priority target for either of them.
I know, that seems like circular reasoning. However, if Babylon 5 were a target for either of them, we'd expect that both the Shadows and the Vorlons would've made moves to meet there to prevent the other from taking it for their own. They did no such thing.

Furthermore, it doesn't seem like either the Shadows or the Vorlons regarded the humans of B5 as contenders for their own positions until Lorien asks them to follow him out beyond the rim. They were simply one of the younger races, albeit a rather vehement race. Both the Shadows and the Vorlons continued trying to win over the humans until the very end.
This occurs in episode 06, when we see the Vorlons as a woman encased in ice and the Shadows as doppelgängers. From then on, the Shadows and the Vorlons are gone away. They don't themselves meddle with Babylon 5 later.

Those are the reasons why neither the Shadows

nor the Vorlons

attacked the Babylon 5 station.