It's not the OS's fault so much as you being a stingy bugger and not having enough RAM or a big enough pagefile
Anytime you run something in PS or PSP you're using a MASSIVE amount of memory even for a relatively small task, for image manipulation AND for your undo/redo steps. If you're using a big image size for quality, especially over 2000x2000 say, you're pushing your luck
I use 2Gb RAM and a 4Gb pagefile on a dedicated partition ... if I play with files around 5,000x10,000px even only 1 or 2 layers, then I start getting memory errors ... it's the software that's hogging everythingcheck your taskmanager for more info!