There is always room for improvement. And unless you test, you will never know.
Optimization always involves testing. What works for one user, might not work for another, what's good for one website, might be bad for another.
Do you know how much money you are leaving on the table because your contact form is too long and therefore causes too much friction? What your shopping cart abandonemnet rate is? How high your bounce rate is?
Knowledge is power — the more you know the better your site will perform.
How do you find out if your site can perform better? By testing it.
Test its performance in the search engines, test the link popularity using Google. Test the overall user experience — maybe by conducting a survey. Have someone outside your business make a test purchase to evaluate user experience. Have them find a particular piece of information or a specific product on your site to test usability. Verify the validity of your SSL certificates and the exisitence of additional trust-building factors like seals to ensure and build trust.
It's up to you how much or how little you want to test. However, by conducting too many tests at one time, the results become harder to interpret. Testing one, two or maybe three elements and/or versions at a time provides you with the best chance to gather reliable data.
Check out a more detailed look on how website optimization works.