I recently read the trilogy, and absolutely fell in LOVE with this type of story. To clarify, to me, the three body problem trilogy felt like a modern take on cosmic horror, while im not looking specifically for horror. I'm looking for the feeling that this trilogy gave me, a sort of feeling of awe and excitement over the unknown. I love the unknown, but especially i love the idea of the unknown in reality, so stories that ground science fiction worries in reality a bit, but also aren't entirely afraid to pull out some less realistic ideas to get across certain points. I am looking for any books that remotely feel like this trilogy in that sense.
I feel like such books would be very uncommon however.
EDIT: Do any of you listen to music while you read? I started recently making specific playlists for reading various kinds of books, so far i’ve made a playlist for reading warhammer 40k books and a playlist for reading cosmic horror and creepy stories in general, i came up with the idea while reading three body problem. after i started listening while reading, it made the experience even creepier, which i loved.
EDIT EDIT: For anyone else making suggestions, since theres a few suggestions i already have in my collection i plan to read here. i will provide a list of books i have in my collection that i plan to read. This isn't all of them, this is just my "cosmic stuff" folder. Some of it is a bit less sci fi and is more just personally interesting for other reasons.
Ballad of black tom- victor lavalle
Blindsight+echopraxia- peter watts (suggested by this thread, already had it, more inclined to read now)
The void trilogy- peter hamilton
Phyl Undhu, Abstract horror, exterminator- Nick Land
The killing star- charles R pellegrino
The revelation space collection- alastair reynolds (newly added thanks to this thread)
The tinfoil dossier trilogy- Caitlin R Kiernan
vacuum diagrams/xeelee sequence- Stephen Baxter (newly added from this thread)
The expanse- james S.A. Corey (haven't grabbed yet, but plan to)
Project Hail Mary might fit the bill. It's a story of a person who finds himself inside a spaceship, traveling somewhere in space to some location in space. He has lost his memory, so he has no idea why he is there or where he is going. All he knows is, that while he survived, his crewmates did not, and so, he is alone in space with somewhat annoying artificial intelligence as his only companion. He needs to save earth from something, but he doesn't know that yet.
I'm not 100% sure if it is what you are looking for, but it is very highly regarded sci-fi book. Also it's a standalone book (500 pages), not part of any kind of series of books, so it's less daunting to read than something like The Expanse series. There are some violence, but I don't remember how graphic it is.
I'll just have to read it and see!