The same QoS considerations apply to jitter as to loss. Naturally, jitter is easier to overcome with simple buffering, but this hurts interactivity.
Very high jitter can additionally adversely affect TCP performance, since packets may be potentially reordered. When a packet gets behind several other packets (the critical number may vary from 3 to 8 depending on sender's and receiver's TCP implementation and network conditions), it essentially counts as lost, since TCP fast retransmit kicks in.