The server location will affect load time, for example our Seattle location has a ~110ms delay for me. This is in addition to the time required for the server to handle my request (interact with php/mysql etc).
The distance does affect site performance, but 120ms is less than 1 second, so it's not that big of a hit alone. What will affect load time more is an overloaded server, which can be common with larger providers. A shared server with 1200 sites is better than one with 600 sites when it comes to profit. The performance suffers as a result, and if the turnover (people canceling) remains normal they are unlikely to care.
If you aren't happy with your host, and they aren't working to fix things - switch. Your business is important take it where the host recognizes that and tried to help you.
In regards to locations we (SkyToaster) do offer hosting in the UK and NL, despite being based in the US. If your familiar with modmore your familiar with our hosting