tolkiens legendarium - What does it mean that the elves were diminished?

The Wikipedia article on the Elves of Middle-earth says:

After the destruction of the One Ring, the power of the Three Rings of the Elves would also end and the Age of Men would begin. Elves that remained in Middle-earth were doomed to a slow decline until, in the words of Galadriel, they faded and became a "rustic folk of dell and cave," and were greatly diminished from their ancient power and nobility. While the power of the remaining Noldor would be immediately lessened, the "fading" of all Elvenkind was a phenomenon that would play out over hundreds and even thousands of years; until, in fact, our own times, when occasional glimpses of rustic Elves would fuel our folktales and fantasies.

I just can't seem to grasp why they were diminished, though. I have only seen the films and it seems like the elves are leaving Middle-earth because of their diminishing. I am hoping that if I understand what that means I can understand why they left.

(src: Irregular Webcomic No. 123 2003-05-28)

Answer

This is summarised in Letter 131, Tolkien's letter of 1951 to Milton Waldman of Collins, who were then interested in publishing Lord of the Rings (and the Silmarillion, a primary cause of dispute between Tolkien and Unwin at the time). The extract from this in Letters is also printed in the second edition Silmarillion, and so should be widely available. all quotes, unless otherwise stated, are from this letter.

First, and in brief, covering the Doom of the Elves:

The doom of the Elves is to be immortal, to love the beauty of the world, to bring it to full flower with their gifts of delicacy and perfection, to last while it lasts, never leaving it even when 'slain', but returning – and yet, when the Followers come, to teach them, and make way for them, to 'fade' as the Followers grow and absorb the life from which both proceed.

As for why they have this doom - it was assigned to them by Iluvatar following the Ainulindale. Following the War of Wrath and the pardon of the Valar:

We learn that the Exiled Elves were, if not commanded, at least sternly counselled to return into the West, and there be at peace. They were not to dwell permanently in Valinor again, but in the Lonely Isle of Eressea within sight of the Blessed Realm.

However, and later in the Second Age, there is a second Fall of the Elves (the first being the rebellion of Feanor) due to their becoming possessive of both the world and of their higher status in it (a common theme in Tolkien is that avoiding one's doom just makes things worse):

There was nothing wrong essentially in their lingering against counsel, still sadly with the mortal lands of their old heroic deeds. But they wanted to have their cake without eating it. They wanted the peace and bliss and perfect memory of 'The West', and yet to remain on the ordinary earth where their prestige as the highest people, above wild Elves, dwarves, and Men, was greater than at the bottom of the hierarchy of Valinor. They thus became obsessed with 'fading', the mode in which the changes of time (the law of the world under the sun) was perceived by them.

And so to the Third Age, which Tolkien views as being an age of passing from Myth to History...

...a Twilight Age, a Medium Aevum, the first of the broken and changed world; the last of the lingering dominion of visible fully incarnate Elves, and the last also in which Evil assumes a single dominant incarnate shape.

Tolkien notes carefully that both mortality and immortality infer a natural degree of longevity within the world, and while on the surface it seems that this note only relates to Men, it should be clear that this also applies to Elves seeking to avoid their own doom (i.e. to avoid making way for the Followers):

The view is taken (as clearly reappears later in the case of the Hobbits that have the Ring for a while) that each 'Kind' has a natural span, integral to its biological and spiritual nature. This cannot really be increased qualitatively or quantitatively; so that prolongation in time is like stretching a wire out ever tauter, or 'spreading butter ever thinner' – it becomes an intolerable torment.

"Qualitatively or quantitatively" is key here; this doesn't just apply to the length of a lifespan but also to one's fate during that lifespan. Attempting to change it, whether by Men seeking immortality, or Elves seeking preservation, is viewed as a Bad Thing in Tolkien.

So finally we come to the end of the Third Age, when the transition from Myth to History is completed, and the function of the Firstborn in the world is over. As Gandalf says following the coronation of Aragorn in Return of the King:

This is your realm, and the heart of the greater realm that shall be. The Third Age of the world is ended, and the new age is begun; and it is your task to order its beginning and to preserve what may be preserved. For though much has been saved, much must now pass away; and the power of the Three Rings also is ended. And all the lands that you see, and those that lie round about them, shall be dwellings of Men. For the time comes of the Dominion of Men, and the Elder Kindred shall fade or depart.

And so to the last words of the Silmarillion, following the destruction of the Ring and the passing of the Elves:

White was that ship and long was it a-building, and long it awaited the end of which Círdan had spoken. But when all these things were done, and the Heir of Isildur had taken up the lordship of Men, and the dominion of the West had passed to him, then it was made plain that the power of the Three Rings also was ended, and to the Firstborn the world grew old and grey. In that time the last of the Noldor set sail from the Havens and left Middle-earth for ever. And latest of all the Keepers of the Three Rings rode to the Sea, and Master Elrond took there the ship that Círdan had made ready. In the twilight of autumn it sailed out of Mithlond, until the seas of the Bent World fell away beneath it, and the winds of the round sky troubled it no more, and borne upon the high airs above the mists of the world it passed into the Ancient West, and an end was come for the Eldar of story and of song.