If you're worried, then see if you can get replacement batteries. Although if they hold a good charge even though they are older refurbished items, chances are they were given new batteries already when you bought them. I think that tends to be protocol with most refurbished items.
But if you check them periodically and all looks fine then I wouldn't worry about it. I mean a battery that has leaked or has gone bad in some other way (other than just dying) tends to be visually obvious in one way or another.
Technically, forms of lithium have been used in some medications -- so if some people are ingesting forms of it, then it's probably not some horrible thing that is going to kill you via skin contact... If you somehow did get something from the batteries on your hands then, unless your skin is super sensitive, you'd probably be fine going and washing it off right quick before it causes any irritation if possible.