Yes probably t-stat from description.
Use a 195deg t-stat with "jiggle valve"
Jiggle valve is a small hole in the t-stat plate that allows air out when refilling system, when placing the t-stat the jiggle hole goes at 12:00 position.
Engines run best at 200-220deg, so 192-195deg t-stat will get engine up to best operating temp faster.
The '94 with 4.0l uses the heater core as part of the cooling system, you will notice there are no heat control valves on the 2 heater hoses in the engine compartment, they run directly from the engine to the heater core.
If you notice temp gauge going up and down in what seems like a random fashion, after t-stat replacement, then I would replace the heater core.
Takes less than 20mins first time, cost is under $30
I would do it just as a preventative measure when doing the t-stat since you have to drain the system anyway.
Remember to fill the core with coolant after install via a heater hose, this helps when purging air after draining.
The core can get partially blocked over time, it still blows heat but limits full flow so engine temp fluctuates.