Are you wanting to clean the contacts on the inside of the relays? I thought most automotive relays were sealed.
Anyway, you can find "electrical contact cleaner" and use that. I'm pretty sure I have seen it at auto stores like Advance, Oreilly's, NAPA, etc. That is designed for cleaning relay contacts.
If you just want to clean the connection blades where the wires plug onto the relay, It depends what they are dirty with. For grease and grime, contact cleaner, brake cleaner, carb cleaner or even rubbing alcohol with a Q-tip should work. If they have green corrosion on them, you may need a small wire brush to remove it. If they are really that old and crusty, you might be better off replacing them.