It depends on your site actually. If you are hosted on a shared hosting, you basically have no dedicated CPU or RAM resources. If other sites are having high load, you are impacted.
[Broken External Image]:
http://tiny.cc/jr5nax But with your own resources, your site is assured to get its RAM to work on its loading time. I just moved from shared to VPS, bcos shared accounts basically competes with a few hundred other accounts for RAM , CPU . With your own VPS, you can manage your expectation given the right RAM vs your utizliation