Lone Wolf Development Forums  

Go Back   Lone Wolf Development Forums > Realm Works Forums > Realm Works Discussion

Notices

Reply
 
Thread Tools Display Modes
Merion
Senior Member
 
Join Date: Mar 2014
Location: Germany, so please bear with my English
Posts: 378

Old November 23rd, 2017, 08:39 AM
Quote:
Originally Posted by kbs666 View Post
Ask yourself this, what links to all that stuff outside those articles and how likely are you to ever follow one of those links?
I certainly would - if opening a new tab in RW wouldn't take longer than starting up a browser and navigating to site in question on the PFSRD...
Merion is offline   #11 Reply With Quote
Farnaby
Senior Member
 
Join Date: Aug 2013
Posts: 108

Old November 23rd, 2017, 10:27 AM
Quote:
Originally Posted by kbs666 View Post
Not sure which CSV file you used but all the pathfinder feats and spells is a huge amount of data. It shouldn't be anything like 33 GB though unless you did the import several times or something.

I've told people this before and I'll say it again, I don't think there is much if any utility in adding all of that rules crunch to RW.

Ask yourself this, what links to all that stuff outside those articles and how likely are you to ever follow one of those links?
I get where you are coming from but I am running a campaign and for the first time am in the higher (12+) levels. I cannot memorise the feats and spells and need to look them up during combat.

Do you have other suggestions? Like a different, faster app to look up spells and feats?

Last edited by Farnaby; November 23rd, 2017 at 11:01 AM.
Farnaby is offline   #12 Reply With Quote
Farling
Senior Member
 
Join Date: Mar 2013
Location: Greater London, UK
Posts: 2,623

Old November 23rd, 2017, 10:56 AM
Quote:
Originally Posted by Exmortis View Post
Submit a bug report, but don't expect anything anytime soon if it requires a patch.
It is a known bug that doing imports generates wasted space in the database, which is why the "Compress database" option was added as a work-around.

Farling

Author of the Realm Works Import tool, Realm Works Output tool and Realm Works to Foundry module

Donations gratefully received via Patreon, Ko-Fi or Paypal
Farling is offline   #13 Reply With Quote
Farnaby
Senior Member
 
Join Date: Aug 2013
Posts: 108

Old November 23rd, 2017, 10:58 AM
This is my reasoning behind my realm.

I want a realm with the details of the Pathfinder Golarion campaign setting as well as the crunch. This I can take and export portions of it into my campaign realm as needed. E.g. Rise of the Runelords doesn't need anything apart from 2-3 lands. I don't need the rest.

So I started from scratch again and wiped every realm locally to have a clean slate, I uninstalled RW and deleted all the data apart from the structure and export files.

I created a brand new test realm, imported the structure and then imported the spells and feats. The following are the changes in the MASTER.REALM size after each step:

Start 5,040 KB
Global Data Sync 6,812 KB
Creating new realm 7,235 KB (pathfinder, structure file with 3 new categories)
Adding 9,828 KB export file with 2,777 feats 1,407,660 KB
Adding 16,166 KB file with 2,829 spells 3,986,780 KB

I have used and installed Lotus Notes back in the day and have a background in intercontinental networking as well as IT security. I mention this to prove that I know a bit about IT as a whole.

My question is how (insert random profanity here ) does the database expand from 7.2 MB to 1.4 GB and then 3.98 GB after importing 9 MB and then 16 MB text? No graphics, just a csv file created by using the csv2rw app.

And now here comes the really interesting part.
Seriously I did not expect this and I suggest everybody now should do it on a regular basis.

I went into tools under database management in the starting window and chose Compress Database. Even though I hadn't deleted anything, imported multiple times or do anything to cause bloat it reduced the MASTER.REALM down to 168.68 MB, which is still a about 130 MB more than expected, but a lot less then multiple GBs.

Despite my surprise, I have no problem with this, if RW works like this then okay. But I personally think this should be put in a prominent place so that everyone is aware of this and should do the compression on a regular basis.

And from a pure professional aspect, it would be interesting to know where the 130+ MB size comes from. The answer is going to be rational but I can't put my finger on it and that's annoying.

P.S. Thanks for all the replies, it's interesting to find out what more experienced users have realised.
Farnaby is offline   #14 Reply With Quote
kbs666
Senior Member
 
Join Date: Oct 2014
Location: Chicago, IL
Posts: 1,690

Old November 23rd, 2017, 01:15 PM
Quote:
Originally Posted by Farnaby View Post
I get where you are coming from but I am running a campaign and for the first time am in the higher (12+) levels. I cannot memorise the feats and spells and need to look them up during combat.

Do you have other suggestions? Like a different, faster app to look up spells and feats?
I use either HL or the PFSRD. Those are both way faster and easier than bloating RW with it.

As to your compress DB issues, 4 GB does seem excessive for just two imports. I'd really like to take a look at the script they use for inserting rows from a foreign source. I think something is hinky. The fact that you can run a utility and get 90+% of the space back says something is badly wrong somewhere.

Whether 130MB is really that large for the DB keep in mind a CSV is nearly as unformatted as data can be. A DB is a much more ordered method of storing data, one that usually relies on things like fixed record sizes and indexes to speed up searches both of which add bulk to the DB.

my Realm Works videos
https://www.youtube.com/channel/UCZU...4DwXXkvmBXQ9Yw
kbs666 is offline   #15 Reply With Quote
Farnaby
Senior Member
 
Join Date: Aug 2013
Posts: 108

Old November 24th, 2017, 08:45 AM
Quote:
Originally Posted by kbs666 View Post
Whether 130MB is really that large for the DB keep in mind a CSV is nearly as unformatted as data can be. A DB is a much more ordered method of storing data, one that usually relies on things like fixed record sizes and indexes to speed up searches both of which add bulk to the DB.
See, that was what I was missing, fixed length fields.
That comes of choosing networks over databases
Farnaby is offline   #16 Reply With Quote
Conandy
Member
 
Join Date: Apr 2017
Location: Denver, CO
Posts: 62

Old October 5th, 2018, 11:55 AM
I have noticed my loading screen (time it takes to double click the icon on my desktop until I get the "login" prompt has been really slow. I just timed it at 44 seconds. It has never been instant, but it never took this long until just recently.

My take is that this should not be related to the database size at this point in the process, or should it?

For the record, actually logging in, and loading a realm, take normal reasonable amounts of time. A few seconds.
Conandy is offline   #17 Reply With Quote
kbs666
Senior Member
 
Join Date: Oct 2014
Location: Chicago, IL
Posts: 1,690

Old October 5th, 2018, 04:28 PM
I've never had that problem. I'm guessing you should try uninstalling and reinstalling. That is almost always a good first step in fixing these sorts of issues.

my Realm Works videos
https://www.youtube.com/channel/UCZU...4DwXXkvmBXQ9Yw
kbs666 is offline   #18 Reply With Quote
Farling
Senior Member
 
Join Date: Mar 2013
Location: Greater London, UK
Posts: 2,623

Old October 6th, 2018, 02:29 AM
Quote:
Originally Posted by Conandy View Post
I have noticed my loading screen (time it takes to double click the icon on my desktop until I get the "login" prompt has been really slow. I just timed it at 44 seconds. It has never been instant, but it never took this long until just recently.

My take is that this should not be related to the database size at this point in the process, or should it?

For the record, actually logging in, and loading a realm, take normal reasonable amounts of time. A few seconds.
Try compressing your database, it might have grown over time to something rather large. It needs to scan the database to get a list of realms.

Farling

Author of the Realm Works Import tool, Realm Works Output tool and Realm Works to Foundry module

Donations gratefully received via Patreon, Ko-Fi or Paypal
Farling is offline   #19 Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -8. The time now is 08:13 AM.


Powered by vBulletin® - Copyright ©2000 - 2024, vBulletin Solutions, Inc.
wolflair.com copyright ©1998-2016 Lone Wolf Development, Inc. View our Privacy Policy here.