You can already control with Plug-ins are available by either removing the plug-ins from the Photoshop>Plug-Ins folder or by putting a tilde (~) in front of the plug-ins name.
Photoshop does not actually "load" the plug-in into memory until the plug-in is used. Photoshop simply gets the name of the plug-in to populate the menus in Photoshop (Filter, File Format, etc) which is actually very fast.
Photoshop's launch time has been improved markedly since CS2.
If you want to improve launch time and Photoshop performance in general, consult this topic: