Update on the starter troubleshooting. Took the starter back out, went through it again, found nothing. Only thing I could think of is that it wasn't grounding through the block, so I took it down to bare metal at the mounting point and block with a wire wheel. Checked the pinion clearance and it was sat. Side note: while trying to figure out how to check the pinion clearance I realized that you can essentially "pin" the pinion in the out position by using (what I thought was) just the drain hole on the bottom of the pinion housing. There really isn't any other way to get the pinion out when the starter is installed without engaging the coil, so i just pulled it out with a hooked scribe, and then put a small pin punch in the hole which held the pinion extended in the out position, then lifted it up and bolted it on. Anyways, hooked up the coil wire from the newly installed starter relay under the dash and left the main starter wire disconnected to first check if the pinion was shooting out all the way. Well I went to turn it over and heard the same click that I'd been hearing-pinion shoots out and half way engages the flywheel. So I went down to check starter pinion engagement and then went up to check a couple voltages on the battery and whatnot. Well then I saw the smoke coming out...disconnected the batteries but the damage was already done. Smoked the starter coil. Took me a few minutes to figure out why but eventually I realized that when I took the ignition to the start position, it immediately welded the starter relay shut and since power is always applied to the starter relay, it just kept the coil energized for a few minutes and fried it.
So I took this opportunity to stop messing with the Chinese garbage 15591718 replacement relay and replaced it with a 200 amp 12 volt relay that I had laying around from an old painless wiring battery disconnect kit.
Now I have the starter and ignition switch circuits tested and functioning correctly again and need a new starter coil (solenoid).
I suspect that even though the starter and starter solenoid tested electrically okay, that the coil was weakened and perhaps had water intrusion at some point. This led to its lack of engagement, which wasn't allowing the motor contacts to close and spin the engine. At least that is what I'm hoping.
P.S. I did find a local starter and alternator shop that will test alternators and starters (for free) and they are also familiar with CUCV alternators and can test 24 volt starters, as well as conduct any necessary repairs. Shop is called Browning's Alternator and Starter Exchange in Virginia Beach, VA.