How did J.R.R. Tolkien learn about the events of his books? (In-Universe)

So, according to Tolkien, all his Middle-earth novels take place in the real world, about 6,000 years before they were released.

So my question is, did he ever state how he learned of the history of Middle-earth? I vaguely recall that the Hobbits told him but I don't know if that is true or not.

(I know that in real life, he just wrote the books himself, but I'm looking for in-universe answers.)

Answer

The full origin is revealed in the prologue to Fellowship of the Ring, where Tolkien discusses the Red Book of Westmarch. The Red Book is actually a collection of other works, containing all of the material for

• The Silmarillion (through Bilbo's Translations From the Elvish)


• The Hobbit, which is Bilbo's account of the adventure


• The Lord of the Rings


• The Appendices

Tolkien discovered a copy that had been made in Minas Tirith early in the Fourth Age (emphasis mine):

This account of the end of the Third Age is drawn mainly from the Red Book of Westmarch. That most important source for the history of the War of the Ring was so called because it was long preserved at Undertowers, the home of the Fairbairns, Wardens of the Westmarch. It was in origin Bilbo's private diary, which he took with him to Rivendell. Frodo brought it back to the Shire, together with many loose leaves of notes, and during S.R. 1420-1 he nearly filled its pages with his account of the War. But annexed to it and preserved with it, probably in a single red case, were the three large volumes, bound in red leather, that Bilbo gave to him as a parting gift. To these four volumes there was added in Westmarch a fifth containing commentaries, genealogies, and various other matter concerning the hobbit members of the Fellowship.

The original Red Book has not been preserved, but many copies were made, especially of the first volume, for the use of the descendants of the children of Master Samwise. The most important copy, however, has a different history. It was kept at Great Smials, but it was written in Gondor, probably at the request of the great-grandson of Peregrin, and completed in S.R. 1592 (F.A. 172). Its southern scribe appended this note: Findegil, King's Writer, finished this work in IV 172. It is an exact copy in all details of the Thain's Book in Minas Tirith. That book was a copy, made at the request of King Elessar, of the Red Book of the Periannath, and was brought to him by the Thain Peregrin when he retired to Gondor in IV 64.

The Thain's Book was thus the first copy made of the Red Book and contained much that was later omitted or lost. In Minas Tirith it received much annotation, and many corrections, especially of names, words, and quotations in the Elvish languages; and there was added to it an abbreviated version of those parts of The Tale of Aragorn and Arwen which lie outside the account of the War. The full tale is stated to have been written by Barahir, grandson of the Steward Faramir, some time after the passing of the King. But the chief importance of Findegil's copy is that it alone contains the whole of Bilbo's 'Translations from the Elvish'. These three volumes were found to be a work of great skill and learning in which, between 1403 and 1418, he had used all the sources available to him in Rivendell, both living and written. But since they were little used by Frodo, being almost entirely concerned with the Elder Days, no more is said of them here.

^{_{Fellowship of the Ring Prologue 5: Note on the Shire Records}}

This is further revealed in a message printed in Elvish characters on the inside cover of The Lord of the Rings books; as I discussed in another answer, that script can be translated as:

The Lord of the Rings translated from the Red Book of Westmarch by John Ronald Reuel Tolkien. Herein is set forth the history of the War of the Ring and the Return of the King as seen by the Hobbits.

There's an abbreviated version of this in Tolkien's Foreword to the first edition of the book; while he changed the foreword for later editions (because he felt it was confusing; I guess some people thought he was being serious?), a part of the original foreword is printed in The Peoples of Middle-earth, and the whole thing is included in Hammond & Scull's Reader's Companion:

This tale is drawn in large part from the memoirs of Bilbo and Frodo Baggins, preserved for the most part in the Great Red Book of Samwise. It has been written during many years for those who were interested in the account of the great Adventure of Bilbo, and especially for my friends the Inklings (in whose veins, I suspect, a good deal of hobbit blood still runs), and for my sons and daughter.

^{_{History of Middle-earth XII The Peoples of Middle-earth Chapter 2: "The Appendix on Languages"}}

What I'm not sure about is how Tolkien got his hands on the Red Book. Presumably he found some old jars in a cave somewhere in the English countryside.

Interestingly, Tolkien initially had a different origin in mind for The Silmarillion. For much of his life he thought of that work as having been compiled by Ælfwine, a 10th-century Englishman who accidentally got to Tol Eressëa and heard the story directly from the Eldar. Ælfwine's role is discussed extensively throughout the History of Middle-earth series, but Christopher Tolkien edited him out of the published Silmarillion.

Additional sources

In addition to the Red Book, which contributed the bulk of the narrative, Tolkien reveals in the foreword to the first edition (reprinted in The Peoples of Middle-earth) that he also drew from some surviving records of Gondor:

I have supplemented the account of the Red Book, in places, with information derived from the surviving records of Gondor, notably the Book of the Kings.

^{History of Middle-earth XII The Peoples of Middle-earth Chapter 2: "The Appendix on Languages"}

These records were presumably the source of much of Appendix A, and possibly Appendix B.