Etiquette?

This will just be a short little post since I'm really curious, but it's time for my nap!  :)

I've been making so many new friends lately and I find myself hopping all over the place reading and checking comments and such, so the question to all you experienced bloggers is this....

When someone comments on your blog, should you reply there or go reply on their blog?

I try to go around and leave some love on various blogs every day, and obviously it's a lot easier to reply on your own, but do people check back like I do to see if the blog author replied to them specifically?

I don't know, and I want to do this right, any suggestions?