You're not crazy. It's not really about the back button. Tiktok uses an algorithm that suggests you content that you like based on how you use your feed. It uses many different factors like the time you stay on a specific video/content, your likes, comments, how long you watch someone's profile, and more to suggest you things that will keep you there.
It's quite advanced but there are some concerning factors to that because it can't tell from right or wrong. So if you were to be browsing for something inappropriate, then yes it'll suggest you just that.