Cold air is better for performance, but burns more fuel because the ECM makes the fuel mix richer. Generally speaking, warm air is better for mpg's than cold air, because the computer leans the fuel mix out a bit. Warmer intake air can also reduce engine warm up times, and warm engines are more efficient than cold ones. It's one of the many reasons people get better fuel mileage in the warmer months than the middle of winter.
I don't have figures for removing the air silencer, but removing the airbox completely and switching to a full air intake kit on my 3.0 did increase my fuel economy by an mpg or two.
To be honest, I doubt you'd see much change, if any by just removing the air silencer. It's a free mod though, and doesn't take much time or skill to do, so I'd say it's worth a shot.