There's definitely some overhead. ES is used as a DB to integrate search functionality. I know you can do close or similar with other DBs, but that's what they went with. It works pretty good on my archive (as far as the team is aware, I'm the heaviest user). I've got comments, subtitles in ES as well, all searchable.
You can limit the RAM ES has available to it with Java arguments if you care.
I'm more than willing to take on your hoard if you want to share. I've been meaning to get into APK cataloging.
I'm using TubeArchivist as well. My collection is slightly larger than the OP's at this point. I collect everything from personally-enjoyable and re-enjoyable content, random stuff at risk during policy changes, to stuff from my early internet years that seems to slowly rot away on the internet. I also like the new themes
With TubeArchivist, all the metadata is stored in an ES database and can be queried via API. There's already a Plex and Jellyfin plugin folks are using.
I was using Podgrab, but it's basically not maintained at all. There's a whole set of podcasts that aren't accessible because the useragent string hasn't been updated and other issues that have gathered. I'd consider it a dead project, which sucks, it was simple and worked. There have been some forks, but I seem to remember issues with them as well and they also appear dead.
I used to use GPodder before that. Don't quite recall why I moved away (probably because it was more a desktop application than a service). Might have to give it a go again.
There's another recent project PodFetch that's been self-promoting in the issues sections of other projects, but it seemed young and not in a useable state when I tried it.
I too want to like Audiobookshelf but for my collection, it's been very unstable and unreliable.