I've done sports announcing, and come from a journalism family where my dad taught radio broadcasting.
Sports casting is hard. Like really, really hard. It is very easy to criticize the way someone does it, but it is incredibly difficult to fill hours of silence. I did live commentary for college wrestling, and I was a very knowledgeable high school wrestler, but frankly sometimes there just isn't something exciting or even describable happening. Jockeying for control, positioning, or feeling out an opponent - sometimes the announcing is "they continue struggling!" Then you think of a sport that isn't nonstop action like American football, or God forbid, baseball? Huge swaths of time where there is nothing to say. This is why professional sports casts on major networks have huge teams. They can pull up obscure stats that don't really mean anything, instant replay analysis done nearly live, and a ton of graphics to keep things moving and exciting.
Then you have the issue others have talked about, where your audience may have almost no knowledge of what to you is a deeply technical sport. So every time you explain a wrestling move, or defensive pass coverage, you have to assume no knowledge. You have to explain why someone is doing something, but luckily that actually fills up a bit more time because God forbid you have dead air on a broadcast, so of course you do it. And the type of deep analysis a knowledgeable fan might want is actually really hard to not only come up with live, but while watching something live without the benefit of watching a replay or a better camera angle.
Anyway, my point is that you should try to do an entry level sports broadcasting exercise. Turn the sound off on a game, and try to cast it and record yourself. You will be absolutely shocked at how much silence there is, or how many asinine things you say. Even the "worst" broadcasters that you experience on any major network have such insanely deep knowledge and an ability to just keep spewing information and anecdotes out that I promise you would be so much more impressive if you heard an amateur, or better, tried to do it yourself.