Also, if there are multiple top-level comments without any replies next to each other, clicking the down arrow will skip past all of them and jump to the next thread that does have comments and puts that last comment from the previous thread at the top to show the next top-level comment below that thread.
It took me a while to figure out what was happening for this one. It can jump really far if there are lots of top-level comments with no replies.