Had a similar issue with my '94 4.0l
Turned out to be a partially clogged heater core.
On this model the heater core must always have coolant flowing thru it, it is the bypass for the cooling system, there is no "heat control valve" on either heater hose.
temp gauge would go up and down randomly, never overheated just above 1/2 then back down.
Good thing is that it is relatively cheap fix, heater core is $28, and only 4 screws and two hose clamps to swap it out.
If your heat inside is just warm not hot and one heater hose in engine compartment is hot and the other just luke warm then core is clogged, as there is no valve both hoses should be the same temp.
To test if this is your problem unhook heater hoses at firewall and connect them together, or just remove one hose from firewall and connect to engine, so loop one hose from water pump to intake manifold, bypassing the heater core.
If up and down temp goes away replace core.
Jiggle valve(hole in t-stat plate) is not required, but does help when refilling system.
I drill an 1/8" hole in the plate if it doesn't have a jiggle valve.
And no that is not your problem.
Make sure your overflow hose is tight and free of cracks/holes, check to make sure coolant level in overflow tank goes up when engine is hot, and then goes down after engine is cold, this means any air in the engine will be purged and both rad cap valves are working, yes there are two valves in a rad cap
Never let overflow tank run dry, that will cause air to be sucked into the rad/engine.
If you are worried about a goof on the new head gaskets, do a Glove Test it is free and results are instant.
Welcome Visitor! Please take a few seconds and 
