Realm Works is currently a 32-bit product. As such, it can only use a maximum of 4GB (best case), regardless of how much RAM your computer has installed.
There is no way to remove or replace the image without first loading the smart image into the content pane. So you'll need to load the smart image, clear its image, and then select the new image in its stead. After doing this for some number of images, you may end up crashing due to the out-of-memory issue. At that point, you should be able to restart and resume where you left off. You should not find yourself unable to restart, since the issue is memory fragmentation, which gets reset when you restart.
In other news, I'm just silly tired. I saw your signature at the bottom of your posts and misread "Mythras" as "Zathras", which made me go WTF and laugh out loud. The thought of somebody playing Zathras as a PC is absolutely hilarious. For those not familiar with Zathras, he's a fantastic character from the old Babylon 5 TV show (from back in the mid-90s). Here's a link to a classic scene of his below. Now imagine a game where someone plays him as a PC. I'll just sit back with a bucket of popcorn and watch...
https://www.youtube.com/watch?v=1j-76eLz1hc