Generally speaking, no.
That being said, I would run some other tests, to other endpoints and with other speed test providers, to confirm or deny what’s going on. Then, if I could nail down a pattern (e.g., routes to Madtown suffer but routes to Portland ME are low latency), I’d run it up my ISPs flagpole as a service issue.