I would have taken en passant because it leads to being able to isolate the d5 pawn on e4 in the next few moves. En passant, If bishop takes trade knight then later gang up on the pawn.
You mentioned that you wanted to get rid of the knight on e4. I would want to as well but you allowed him to open a file for his rook and connect the f5 pawn to a pawn chain in the middle of the board. Im not an expert but that looks like a positional mistake and you are restricting yourself.
Later when he took your bishop, it was probably better to take with your f pawn. Also opening the file for your rook and solidifying your weak pawn on d4. Taking with the queen looks wrong.
The rest of the game just looks like he converted from those 3 mistakes.
I lost while playing down because I got outplayed in the middlegame and endgame. I would like to know what my big mistakes were and what I should have done differently. This was an OTB game with a time control of 90+10. My opponent was under time pressure and I wasn't, but he played more accurately in the endgame. I have not done the computer analysis.
My opponent didn't even have much tournament experience. His rating was provisional. But I made him look like an expert.