Website speed is based upon two conditions:
(1) Download speed;
(2) Processing speed.
The download speed might be available to your web site server via a check of how fast the web page is being downloaded by the end user.
The processing speed is two part in itself:
(a) the processing of the web server at the time of the request by the end user. This might be slow if the web server has sufficient requests for download that it cannot keep up with them efficiently.
(b) the processing of the end user's system.
In answer to your question, (conditionally) yes there are ways to "speed up" your "website" "further."
Throttling:
First, make certain that your web server is not throttling your transmissions. A common way that throttling is done is to allow the initial upload/download to be fast, and then over time slow it down to nearly nothing. This used to be common practice for ISP's (Internet Service Provider facility sites). The ISP's would advertise fast internet access; their dial-up customers would log in and check their speed and assume that the initial speed would be the common speed. The ISP would then slow down (throttle) the customers' speed toward almost nothing. The ISPs lied in their advertising that they supplied "unlimited access." (It is amazing that lawyers have not made billions of dollars from suing ISP's) The ISPs did actually "limit" the user's "access." A way to check this is to go to a major hotel chain and ask permission to use their computer to look at your web site. Have a large (very large) image or other file on your web page that you click on to download. Watch the download progress bar. If it shows a 50MB file that starts downloading fast enough to take 1 minute to download and in the end it took 10 minutes to download, then your web server is probably throttling your connections. If they are doing that, then do *not* talk to them about it. No no no! Do not tell them anything. Go find another place to put your web page. Then when your new site is up and running, simply end your agreement with the throttled site and move on. Test the new site and do this until you find a site that is not cheating you.
Download Speed:
When you have your web page on a server that is not throttling you, then and only then, check that download speed. If your customer has an internet connection that has a slow download speed then you should know this speed for your web page setup.
If you have a dynamic web page, meaning that the server writes the web page custom for each user, then this is where you might make the page smaller in overall bytes for slower speed customers. But, a difficulty with this might be that Google and other search engines have difficulty in cataloging your web pages. Thus, have some static part of your web page for the search engines to catalog, and some (maybe images) dynamic part of your web page that you can make smaller for users which have slow internet connections.
Processing Speed:
Make all of your web pages that you want to be universally fast(er) in almost all straight HTML. Use very little JavaScript and even then do not use advanced fancy "latest & greatest" JavaScript effects. Try to write the entire page in HTML if you can still make it look nice. This can help your web server and the end user's system.
Do not use animations if you can avoid them and still have a nice looking web site.
If you check the user's browser and find it is Internet Explorer (any version), then assume the worst in that nothing other than HTML will probably work right.
If you check the user's browser and find it is Firefox beyond 54, then assume a mess of hidden backdoors sending and receiving information to other sites stealing the user's bandwidth, thus making your site look slower to them, and blocking or at least disrupting some of your JavaScripts.
Be diligent with this.
Do *not* be flippant with this.
It is *your* site, it is up to *you* to force it to work right.
Hope that helps.