In order for the Chinese room to convince an outside observer of knowing Chinese like a person the room as a whole basically needs to be sentient and understand Chinese. The person in the room doesn't need to understand Chinese. "The room" understands Chinese.

The confounding part is the book, pen and paper. It suggests that the room is "dumb". But to behave like a person the person-not-knowing-Chinese plus book and paper needs to be able to memorize and reason about very complex concepts. You can do that with pen and paper and not-understanding-Chinese person it just takes an awful amount of time and complex set of continuously changing rules in said book.

Edit: Dall-E made a pretty neat mood illustration