Is Sicilian the best play for black
Yes... main reasons is there are just major downsides to everything else in e4.
CK -> white chooses the variation and has tons of options, so the greater theoretical burden is on black, plus there are alot of strong lines including attacks on the king.
e4/e5 -> too much rote memorization of theoretical tactical lines, too overplayed, again white has the ability to choose the variation for the most part
e6 -> decent but very boring, and black can end up restricted
d6/g6 -> hypermodern positions aren't fun and for the most part they're just bad
d5 -> it's fun but it's dubious, really if white knows the right lines he gets a great game and black has very little really
Nc6 -> after Nf3 there's nothing to do but transpose with e4/e5 or play a bad hypermodern position where it'd have been better to just play the modern
Nf6 -> response for white is algorithmic and the game is relatively easy on white, or white can also transpose into the vienna... it's an interesting try but the nimzowitsch sicilian is just better than this in practice, really I would just recommend people play that instead.
c5 -> black chooses the mainline variation, most of the pressure for knowing theory ends up being on white. Positional so doesn't require as much rote memorization, for the most part... giant opening so despite being played commonly the opponent doesn't have the theory memorized usually. White often opts out into an anti-sicilian which really is not challenging... Tons of options for black in almost every line, including rare options and rare mainlines... No downside really.
e4/e5 is probably the second best option, but to play that you really have to know all the theory... you're gonna have to spend so much time studying, it'll be quite a pain. And even when you do learn it all... white will know it too since it's so common, it's just a stalemate, so not even much payoff to it.
There is no such thing as "best", it depends what you want. Sicilian has 8% less draws than e4 e5 at master level, so to avoid a draw is a good reason for choosing the Sicilian.
... hypermodern positions aren't fun ...
I suppose everyone has different perspectives.
For me, I'd say the main reason to play hypermodern openings is specifically because they're fun due to how unconventional the play can be.
Regarding the original question: I'd say the Sicilian is Black's best try for a dynamic game against 1.e4. But it's also arguably the most difficult e4 defense to master, as it involves more risk on Black's part, as well. And the variation trees can extend pretty far in many directions.
The Najdorf, especially, can be pretty challenging to understand in many variations ... Some lines can only be understood by using an engine and scouring through different lines, to see the tactical outcomes.
In many variations, the logical or principled move for Black actually loses due to some hidden tactical resource that White has farther down the line ... This is due to the unbalanced nature of the game.
The Najdorf is pretty low on the list of sicilians I'd recommend below master level for the reasons you mentioned.
But these days ~40% of the opponents play an anti-sicilian anyway, and almost all of those are easy to deal with.
re hypermodern openings: the opening may be novel but once both players are out of book typically you just have a middlegame where black is struggling to undermine an advanced pawn and white has pretty straightforward play with the space advantage. It's just a struggle most games, I prefer winning to struggling.