The symptoms sound familiar & it was caused by a misfire for me.
http://www.mainelysubarus.com/forums/topic/12497-p0304-cynlinder-4-misfire/
Has the CEL come on or any codes from the OBD2?
When was the last time the plugs or wires were changed (worked for me but possibly due to voodoo)