Yes and no. e4 is often a plan but it's not the plan, and how effective it is depends a lot on concrete details and the setup chosen by Black.
The main point is that, if you have d4 and c4 and e4 played you are much more extended than with the spanish center c3-d4-e4. After the push c5 or e5 Black manages to weaken the central dark squares (either after d5 by White or xd4 by Black) and gets play around them. It is more effective to achieve e4 if Black's setup is aimed at preventing it, like in the Queen's Indian or the Nimzo, but even there it's not always clear-cut. And in d5 defences, either e4 leads to a trade (then Black hits at d4 again with c5 or plays on the weakened light squares d5 and e4) or you have to prepare e4 with f3, but it takes time and again d4 becomes weaker.
Examples :
i) in the King's Indian, White plays e4 at move 4 usually and the space advantage is real, but there are perfectly playable systems without it, like the Smyslov where White plays Bg5 and e3, or the fianchetto lines where White avoids e4 to keep the long diagonal open.
ii) in the Grünfeld defence, the exchange variation allows White to play e4 quickly but Black gets counterplay against d4 just as quickly. Therefore the lines with Bg5 or Bf4 and e3 are about just as good (after 4. Bg5 Ne4 5. Bh4 Nxc3 6. bxc3 dxc4 White plays 7. e3, not e4, again because d4 needs support).
iii) in the Nimzo 3. Nc3 Bb4 4. Qc2 0-0, 5. e4 was considered bad for a long time (Black strikes with d5, Ne4 and c5 and White's center will eventually get destroyed). Now it is perfectly respectable and got played quite a bit at high level but 5. a3 is still the main move.
iv) to cure the usual Bogo dull game where you have little to show for the two bishops compared to the Nimzo, Black players started to try 3. Nf3 Bb4+ 4. Nbd2 0-0 5. a3 Be7 allowing 6. e4; but 5. e3 and 5. g3 are perfectly fine.
v) in the exchange Queen's Gambit Declined 2. c4 e6 3. Nc3 Nf6 4. cxd5 exd5 5. Bg5 c6 6. Qc2 Be7 7. e3 Nbd7 8. Bd3 0-0, 9. Nf3 and 9. Nge2 (intending f3 and e4) are equally good and popular.
vi) in the Botvinnik pawn roller variation, Black can be dead lost as soon as White achieves e4 without losing something, which is quite remarkable.
When White plays 1.d4 2.c4 or 1.d4 2.Nf3 3.c4, does White then generally play e4 when he can? Or is it okay to just play the more solid e3? Is getting Pawns on e4 and d4 only a goal if White starts 1.e4 or also if White starts 1.d4?