The Prayer of Manasseh
3 Ezra (= LXX 1 Esdras)
4 Ezra (= 2 Esdras in some English versions of the apocrypha)
The Epistle to the Laodiceans
But, of course, these books did not constitute an "appendix" to the Vulgate in any ancient or medieval manuscripts of the Latin Bible. As far as I know, there was no such thing as an "appendix" to Latin biblical manuscripts. So, when someone talks about 3 Ezra as occupying a place in the appendix to the Vulgate, this could give quite a misleading impression. Such a statement is accurate only if the term "Vulgate" refers to a modern printed edition and not to anything that, for instance, Jerome would have recognized.
Where did the appendix come from? Well, I'm not completely sure, but I think that the Sixto-Clementine edition from 1592 was the first to include an appendix.
The Sistine edition from 1590 did not include an appendix. It's available here, all three volumes. Vol. 1 ends with Job, vol. 2 ends with 2 Maccabees, and vol. 3 contains the NT and ends with Revelation. No appendix.
The Sixto-Clementine edition is available here. After the NT, there's an appendix with the Prayer of Manasseh and 3–4 Ezra. And there's a preface to the Appendix:
Oratio Manassa, necnon Libri duo, qui sub libri Tertii & Quarti Esdrae nomine circumferuntur, hoc in loco, extra scilicet seriem canonicorum Librorum, quos sancta Tridentina Synodus suscepit, & pro Canonicis suscipiendos decreuit, sepositi sunt, ne prorsus interirent, quippe qui a nonnullis sanctis Patribus interdum citantur, & in aliquibus Bibliis Latinis tam manuscriptis quam impressis reperiuntur.
The Prayer of Manasseh, as well as two books, which circulate under the name of the Third and Fourth Book of Ezra, are set aside in this place—that is, outside the series of canonical books, which the holy Tridentine Synod accepted, and determined should be taken up for canonical—lest they should perish completely, since they are sometimes cited by some of the holy Fathers, and they are found in some Latin books, both manuscript and printed.As this note suggests, it was the decree on the biblical canon by the Council of Trent (1546) that created the situation in which it made some sense to print the Vulgate with an appendix containing non-canonical books. For it was only at Trent that the biblical canon was definitively settled. Thenceforth, editions of the Vulgate for a Roman Catholic readership would need to conform to the canon approved by Trent, and so it would no longer be appropriate to print the Prayer of Manasseh and 3–4 Ezra among the other biblical books, as the Gutenberg Bible had done, for instance. An edition of the Vulgate could completely omit any non-canonical works, as the Sistine edition had done, but the editors of the Sixto-Clementine edition were concerned that these venerable though non-canonical books might no longer be available, even though previous generations of Christian authors had sometimes referred to them.
Thus was born the Vulgate appendix.