Ah, this all sounds so familiar to my experience when I bought my 1990 Custom Super Cab 2WD w/2.9L & A4LD back in 2011. I've learned a lot along the way, and am planning to do a top end overhaul this winter to try to deal with worsening valve train noise and performance issues. But there is really no way to make them quiet. Even if you get the valve lash well adjusted, they are just noisy. And it typically varies, and comes and goes.
While the 2.9L Cologne motor does have a severe design flaw, people are correct that they are still tough motors. If you take care of it, it will serve you well. They actually produce more HP per unit of displacement than the 302/5.0L. But it will be a passion project, unless you decide to do the well-advised 4.0L swap. I'm stubborn, so I'll keep mine. At least until the day I do a V8 conversion. Take a look on Youtube for some, there are very impressive builds out there. But $$$$$
Here's my write-up from a few days ago.
Hi all, So, I have a project 1990 Custom Super Cab Ranger with the infamous 2.9L hitched to the equally dubious A4LD. I've had this rig for almost 10 years now, and of course in retrospect I wish I had heeded the warnings about the 2.9L. The whole project got way more involved and expensive...
www.therangerstation.com
PetroleumJunkie412 is correct that you should use a Zinc additive. I wish I had known this sooner, and I suspect that my cam and bearing wore prematurely due to not having this. That, and I probably caused damage by over tightening my valves to try to get the ticking quiet. At a certain point, you just have to accept that it can only get so good.
Because of the valve train lubricating flaw in the design, it's generally better to use 5w30 or even 5w20 oil. Most recommend not going above 10w30, and probably only in hotter climates. This is not like you're typical motor. Higher viscosity oil will probably just make it tick worse. And as long as your main bearings are good, you won't benefit from higher viscosity.
Many have written that the stock builds also have weak valve springs, which results in valve float under high load. I believe mine suffers from this, in addition to worsening valve train lubrication. When climbing mountain passes, for example, it will get very loud and struggle. For now, I just keep it under 65 and that does help. At higher speeds/RPM the lubrication issue gets worse, and the lifters heat up and expand, as well as lose their hydraulic pressure. And it will sound like it's going to fly apart.
These are old motors now, and performance parts are difficult to find. I know of one place, but man that stuff is spendy.
I'm currently trying to find other sources for parts other than OEM spec, and it's hard to find much.
I have also read about modified builds where they use solid lifters with a matched cam. That's probably more involved than I would choose, but since you're learning about this motor I mention it in case you want to research that. Also, the fuel injectors can be noisy and can be mistaken for lifter tick. But I think that's a minor consideration.
And finally, about the varying ticking symptoms, it's generally stated that if the cam bearings have worn you're likely to get worsening ticking after being at freeway speeds for a while. And I think that's most of my current problem.
Good luck!