Well something got miss wired, or is shorted.
Solenoid has two larger posts, one post connects to starter motor the other has the battery "+" cable, the post with the battery "+" cable will also have the alternator cable.
Solenoid will also have 1 or 2 smaller posts, the "S" post is the one that closes the solenoid and passes power to the starter from the battery.
The "I" post, if it has one, would go to the Coil.
Starter motor----------------solenoid------------------Battery "+"
........................................../
........................................"S"
12v from ignition switch, only when key is turned all the way to START
With key off:
If you have 12v on the starter motor post then solenoid is closed, new or not it is closed.
Remove the smaller "S" wire from it's smaller post on the solenoid, if solenoid "clicks" then that wire has a 12v short, this wire connects to ignition switch and should only have 12volts when key is in the START position, not ON, only START.
If this is the case then ignition switch may have a short, not the Key switch, the ignition switch under the steering column, it is operated by the key being turned.
As for the battery, when a battery is older and it gets drained very low it can short out inside, so battery may be unchargable, it will actually drain voltage from other battery being used to jump.
Test your battery voltage, under 12v is bad, if voltage starts dropping with just the test meter hooked up, I would shop for another battery, AFTER you figure out the wiring issue.
Good battery will have 12.4volts to 12.8volts