Lone Wolf Development Forums  

Go Back   Lone Wolf Development Forums > Realm Works Forums > Realm Works Discussion

Notices

Reply
 
Thread Tools Display Modes
Bobson
Member
 
Join Date: Sep 2011
Posts: 36

Old February 23rd, 2017, 08:39 PM
Quote:
Originally Posted by Farling View Post
Just like EightBitz, I've been working with the CSV files from pathfindercommunity.net. (My approach is to use XSLT to do the transformation rather than PowerShell.)
I'd love to help, but my skills are primarily C#, with some perl and a smattering of other languages on the side. So I'd probably be yet another person doing the same thing, rather than useful to either of you.

That said, I have experience with parsing & transforming data, as well as automatically scraping websites to get said data, so if either of you need anything specific, I might be able to help.

Separately, I'd like to suggest reaching out to Nethys of the eponymous archives. I believe he already has a clean, sanitized, and source-linked database of everything that's CUP- and/or OGL-legal (possibly up to last summer) which he powers the website from. If he were willing, it should be trivial for him to export it in a form that you can easily apply your transform tools to.
Bobson is offline   #21 Reply With Quote
BigSkyRPG
Junior Member
 
Join Date: Jan 2017
Location: Billings, MT
Posts: 20

Old February 23rd, 2017, 09:01 PM
I guess I'm trying to figure out why all of these elaborate import and export options are being created by users to move data; is there no plan to allow spreadsheet/csv imports at some point? This seems like basic functionality that is missing or is this intentional?
BigSkyRPG is offline   #22 Reply With Quote
EightBitz
Senior Member
 
Join Date: May 2013
Posts: 1,458

Old February 23rd, 2017, 09:34 PM
Quote:
Originally Posted by BigSkyRPG View Post
I guess I'm trying to figure out why all of these elaborate import and export options are being created by users to move data;
Because we can. What some of us (two, at least) are working on now has no bearing on what LWD may or may not do in the future.

Quote:
is there no plan to allow spreadsheet/csv imports at some point? This seems like basic functionality that is missing or is this intentional?
That fact that users in the community have started working on their own import scripts, that doesn't mean that LWD has nothing planned for the future, or that even if they had nothing planned that they won't adapt to customer expectations.

Right now, they're focusing on the bread and butter of RW, the content market. They're also working on custom calendars, which has been a long-promised feature that initially gave them more trouble than they expected.

I can't tell you what they might or might not do in the future, but in the meantime, some of us are trying to do what we can to facilitate things.

Farling has has his XSLT script, which I'm sure works fine, but I like to do things a little differently than others sometimes, so I'm working on my own script as well.

And that's really as far as it goes right now. We don't know all the future plans, but those of us who can help fill in some gaps right now are sharing what we can do.

My script still has a some issues I have to figure out (not the least of which is that it expects an empty realm), and my aim is to iron those out, expand it for other types of topics (not just spells), and make it more modular and flexible.

If LWD or someone else comes up with something better by then, so be it. But for now, I'll do what I can, and I'm excited that, though Rob may wish to discuss some concerns and coordinate some things, he is nonetheless supportive about our projects and our willingness to share.
EightBitz is offline   #23 Reply With Quote
Parody
Senior Member
 
Join Date: Jan 2013
Location: Rochester, MN
Posts: 1,515

Old February 23rd, 2017, 10:06 PM
Quote:
Originally Posted by BigSkyRPG View Post
I guess I'm trying to figure out why all of these elaborate import and export options are being created by users to move data; is there no plan to allow spreadsheet/csv imports at some point? This seems like basic functionality that is missing or is this intentional?
In my opinion, it's highly unlikely LWD would ever add a feature for going directly from, say, any reasonably unknown (to them) chunk of CSV to Topics/Articles and Snippets. There's no good way for them to know what means what in your data.

RW's XML setup lets LWD define how you can tell Realm Works what each piece of data means. That allows interested folks to write the glue that goes between some set of data and how they want it to appear in RW. This flexibility comes at the cost of ease of use for the person who has to write the glue and perhaps for those using the results.

There's also the ongoing side issue of duplication of effort for this particular system. If you were planning on incorporating purchased products in your own realms, you'll probably want to wait until those purchased products are available and the folks doing conversion have fixed their converters to make Articles/Topics that will integrate well. (Alternatively, you may decide to just buy LWD's version of them.)


Last edited by Parody; February 23rd, 2017 at 10:10 PM.
Parody is offline   #24 Reply With Quote
Farling
Senior Member
 
Join Date: Mar 2013
Location: Greater London, UK
Posts: 2,623

Old February 24th, 2017, 12:00 AM
Quote:
Originally Posted by Parody View Post
RW's XML setup lets LWD define how you can tell Realm Works what each piece of data means. That allows interested folks to write the glue that goes between some set of data and how they want it to appear in RW. This flexibility comes at the cost of ease of use for the person who has to write the glue and perhaps for those using the results.
This mostly. Because in the CSV you might have a column which needs mapping to a tag, and another column which might be a comma-separated list of values (like classes) which needs mapping to one or more tags for that single field.

There are other columns where you need the plain text, and other fields where maybe you need a tag for the basic data but also need an annotation added to appear next to the tag.

The number of cases is so vast that no single tool could possibly provide this import capability without a mass of options to select between. For this reason, the task may as well be left to a separate tool (or collection of scripts) to perform this task.

(Note that exporting a CSV from Excel as XML doesn't work very well, since it cheats when skipping blank cells in the grid making it more difficult to parse the resulting XML.)
Farling is offline   #25 Reply With Quote
Zaphod Beebledoc
Senior Member
 
Join Date: May 2013
Location: Birmingham, UK
Posts: 459

Old February 24th, 2017, 04:48 AM
Is there any text you could put into blank cells that could then be stripped out in Notepad++ or similar?

Sleet was enjoying a tasty beverage at his local tavern, when a Tarrasque showed up in the local area. He managed to valiantly get on it's back and ride it. How he did it is a mystery to this day...

RW: Engine Heart, I Love The Corps! Home Brew: Star Gate: Avalon, Monda Minutia. I'm good with: OpenOffice, Paint, Lego Digital Designer. & not so good with: Realm Works, Hero Lab, CC3+, GIMP, Cityographer, Hexographer, Fractal Mapper, AstroSynth, Inspiration Pad Pro. RW Kickstarter Supporter.
Zaphod Beebledoc is offline   #26 Reply With Quote
Kirbdog20
Junior Member
 
Join Date: Jun 2016
Posts: 4

Old February 24th, 2017, 09:45 PM
OK my issue is, i bought some books for pathfinder with 'drawbacks'... on my PCs i made up i can not see these new features... but on new PC;s i can... please help me so i know how to import all my stuff so regardless, NEW or old PC's the data will be there...
Kirbdog20 is offline   #27 Reply With Quote
EightBitz
Senior Member
 
Join Date: May 2013
Posts: 1,458

Old February 24th, 2017, 09:50 PM
Quote:
Originally Posted by Bobson View Post
I'd love to help, but my skills are primarily C#, with some perl and a smattering of other languages on the side. So I'd probably be yet another person doing the same thing, rather than useful to either of you.

That said, I have experience with parsing & transforming data, as well as automatically scraping websites to get said data, so if either of you need anything specific, I might be able to help.

Separately, I'd like to suggest reaching out to Nethys of the eponymous archives. I believe he already has a clean, sanitized, and source-linked database of everything that's CUP- and/or OGL-legal (possibly up to last summer) which he powers the website from. If he were willing, it should be trivial for him to export it in a form that you can easily apply your transform tools to.
Thanks. Request sent.
EightBitz is offline   #28 Reply With Quote
Parody
Senior Member
 
Join Date: Jan 2013
Location: Rochester, MN
Posts: 1,515

Old February 24th, 2017, 11:19 PM
Quote:
Originally Posted by Kirbdog20 View Post
OK my issue is, i bought some books for pathfinder with 'drawbacks'... on my PCs i made up i can not see these new features... but on new PC;s i can... please help me so i know how to import all my stuff so regardless, NEW or old PC's the data will be there...
Questions about Hero Lab should go to one of the Hero Lab forums.

FWIW: When you purchase new products for a system in Hero Lab and then open an existing character portfolio, you'll need to reconfigure the character (Character/Configure Hero or Ctrl-K) and enable the new products if you want to have them available for that character. Make sure you save afterwards.


Last edited by Parody; February 24th, 2017 at 11:23 PM.
Parody is offline   #29 Reply With Quote
Korpah
Junior Member
 
Join Date: Apr 2016
Posts: 19

Old July 21st, 2018, 10:57 AM
Quote:
Originally Posted by Farling View Post
Hello folks,

Just like EightBitz, I've been working with the CSV files from pathfindercommunity.net. (My approach is to use XSLT to do the transformation rather than PowerShell.)

I have all four databases translated into

After a final bug is fixed in the import scripts in theory rwexport files can be created for all the information. However the realm would become stupidly large if you imported all the data, since you would end up with:

2,766 feats (5.8 MB import file)
5,271 monsters (47.4MB import file)
2,713 spells (11MB import file)
3,901 items (22MB import file)

On a test import, it took several hours to import all the data, and the master.realm increased in size by 5 or more GB!

The most obvious filter to start with is to make all the mythic feats a separate import, but how would all the other mass of topics be broken down to something more manageable.

The spells could possibly be reduced to maybe three blocks based on spell level. So you could import only the blocks required for your campaign.

A lot of the monsters appear to be named creatures, rather than generic monsters, so removing uniquely named monsters might reduce the database massively.

The intention would be to release the RWEXPORT files directly, since they contain no modifications to the information other than the format in which they are stored.
sorry for necroing this thread but couldn't find it.

is an general import of pathfinder monsters items etc already available now since we can import and export data now in realmworks?
Korpah is offline   #30 Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -8. The time now is 08:12 AM.


Powered by vBulletin® - Copyright ©2000 - 2024, vBulletin Solutions, Inc.
wolflair.com copyright ©1998-2016 Lone Wolf Development, Inc. View our Privacy Policy here.