Name it not "close", but "resolved" (a checkbox with checkmark/or no checkmark („resolved“/„unresolved“). AND it automatically will marked as „unresolved“ if the reviewer adds an new answer.
The pin should be convert from number to checkmark if the comment is marked as resolved.

With this I can use the comments as a list of tasks which I can work off step by step.
If I finish a task I mark is at resolved. The reviewer get an E-Mail (at the end of day all resolved tasks collected to one E-Mail) and he can now review this and do an answer if necessary. If a resolved task get an answer it automatically get marled as unresolved and will show me at the list of unresolved tasks/comments.