Following a pub does not cause it to follow you back. Using an invite code does.
Or does my client maintain any sort of link with the pubs through which regular Gossip is happening?
Yes, when your ssb-server is connected with a pub/peer, it is gossipping with it. In a gossip connection, you are requesting feeds from it, and it is requesting feeds from you. If it is requesting your feed, your ssb-server will send it your new messages. Or it may get them via another peer.