Lone Wolf Development Forums

Lone Wolf Development Forums (http://forums.wolflair.com/index.php)
-   Realm Works Discussion (http://forums.wolflair.com/forumdisplay.php?f=67)
-   -   Import of PRD spells, feats, items, monsters (http://forums.wolflair.com/showthread.php?t=57792)

Bobson February 23rd, 2017 08:39 PM

Quote:

Originally Posted by Farling (Post 245020)
Just like EightBitz, I've been working with the CSV files from pathfindercommunity.net. (My approach is to use XSLT to do the transformation rather than PowerShell.)

I'd love to help, but my skills are primarily C#, with some perl and a smattering of other languages on the side. So I'd probably be yet another person doing the same thing, rather than useful to either of you.

That said, I have experience with parsing & transforming data, as well as automatically scraping websites to get said data, so if either of you need anything specific, I might be able to help.

Separately, I'd like to suggest reaching out to Nethys of the eponymous archives. I believe he already has a clean, sanitized, and source-linked database of everything that's CUP- and/or OGL-legal (possibly up to last summer) which he powers the website from. If he were willing, it should be trivial for him to export it in a form that you can easily apply your transform tools to.

BigSkyRPG February 23rd, 2017 09:01 PM

I guess I'm trying to figure out why all of these elaborate import and export options are being created by users to move data; is there no plan to allow spreadsheet/csv imports at some point? This seems like basic functionality that is missing or is this intentional?

EightBitz February 23rd, 2017 09:34 PM

Quote:

Originally Posted by BigSkyRPG (Post 245124)
I guess I'm trying to figure out why all of these elaborate import and export options are being created by users to move data;

Because we can. What some of us (two, at least) are working on now has no bearing on what LWD may or may not do in the future.

Quote:

is there no plan to allow spreadsheet/csv imports at some point? This seems like basic functionality that is missing or is this intentional?
That fact that users in the community have started working on their own import scripts, that doesn't mean that LWD has nothing planned for the future, or that even if they had nothing planned that they won't adapt to customer expectations.

Right now, they're focusing on the bread and butter of RW, the content market. They're also working on custom calendars, which has been a long-promised feature that initially gave them more trouble than they expected.

I can't tell you what they might or might not do in the future, but in the meantime, some of us are trying to do what we can to facilitate things.

Farling has has his XSLT script, which I'm sure works fine, but I like to do things a little differently than others sometimes, so I'm working on my own script as well.

And that's really as far as it goes right now. We don't know all the future plans, but those of us who can help fill in some gaps right now are sharing what we can do.

My script still has a some issues I have to figure out (not the least of which is that it expects an empty realm), and my aim is to iron those out, expand it for other types of topics (not just spells), and make it more modular and flexible.

If LWD or someone else comes up with something better by then, so be it. But for now, I'll do what I can, and I'm excited that, though Rob may wish to discuss some concerns and coordinate some things, he is nonetheless supportive about our projects and our willingness to share.

Parody February 23rd, 2017 10:06 PM

Quote:

Originally Posted by BigSkyRPG (Post 245124)
I guess I'm trying to figure out why all of these elaborate import and export options are being created by users to move data; is there no plan to allow spreadsheet/csv imports at some point? This seems like basic functionality that is missing or is this intentional?

In my opinion, it's highly unlikely LWD would ever add a feature for going directly from, say, any reasonably unknown (to them) chunk of CSV to Topics/Articles and Snippets. There's no good way for them to know what means what in your data.

RW's XML setup lets LWD define how you can tell Realm Works what each piece of data means. That allows interested folks to write the glue that goes between some set of data and how they want it to appear in RW. This flexibility comes at the cost of ease of use for the person who has to write the glue and perhaps for those using the results.

There's also the ongoing side issue of duplication of effort for this particular system. If you were planning on incorporating purchased products in your own realms, you'll probably want to wait until those purchased products are available and the folks doing conversion have fixed their converters to make Articles/Topics that will integrate well. (Alternatively, you may decide to just buy LWD's version of them.)

Farling February 24th, 2017 12:00 AM

Quote:

Originally Posted by Parody (Post 245126)
RW's XML setup lets LWD define how you can tell Realm Works what each piece of data means. That allows interested folks to write the glue that goes between some set of data and how they want it to appear in RW. This flexibility comes at the cost of ease of use for the person who has to write the glue and perhaps for those using the results.

This mostly. Because in the CSV you might have a column which needs mapping to a tag, and another column which might be a comma-separated list of values (like classes) which needs mapping to one or more tags for that single field.

There are other columns where you need the plain text, and other fields where maybe you need a tag for the basic data but also need an annotation added to appear next to the tag.

The number of cases is so vast that no single tool could possibly provide this import capability without a mass of options to select between. For this reason, the task may as well be left to a separate tool (or collection of scripts) to perform this task.

(Note that exporting a CSV from Excel as XML doesn't work very well, since it cheats when skipping blank cells in the grid making it more difficult to parse the resulting XML.)

Zaphod Beebledoc February 24th, 2017 04:48 AM

Is there any text you could put into blank cells that could then be stripped out in Notepad++ or similar?

Kirbdog20 February 24th, 2017 09:45 PM

help with new stuff showing up with old stuff...(PATHFINDER)
 
OK my issue is, i bought some books for pathfinder with 'drawbacks'... on my PCs i made up i can not see these new features... but on new PC;s i can... please help me so i know how to import all my stuff so regardless, NEW or old PC's the data will be there...

EightBitz February 24th, 2017 09:50 PM

Quote:

Originally Posted by Bobson (Post 245121)
I'd love to help, but my skills are primarily C#, with some perl and a smattering of other languages on the side. So I'd probably be yet another person doing the same thing, rather than useful to either of you.

That said, I have experience with parsing & transforming data, as well as automatically scraping websites to get said data, so if either of you need anything specific, I might be able to help.

Separately, I'd like to suggest reaching out to Nethys of the eponymous archives. I believe he already has a clean, sanitized, and source-linked database of everything that's CUP- and/or OGL-legal (possibly up to last summer) which he powers the website from. If he were willing, it should be trivial for him to export it in a form that you can easily apply your transform tools to.

Thanks. Request sent.

Parody February 24th, 2017 11:19 PM

Quote:

Originally Posted by Kirbdog20 (Post 245190)
OK my issue is, i bought some books for pathfinder with 'drawbacks'... on my PCs i made up i can not see these new features... but on new PC;s i can... please help me so i know how to import all my stuff so regardless, NEW or old PC's the data will be there...

Questions about Hero Lab should go to one of the Hero Lab forums.

FWIW: When you purchase new products for a system in Hero Lab and then open an existing character portfolio, you'll need to reconfigure the character (Character/Configure Hero or Ctrl-K) and enable the new products if you want to have them available for that character. Make sure you save afterwards.

Korpah July 21st, 2018 10:57 AM

Quote:

Originally Posted by Farling (Post 245020)
Hello folks,

Just like EightBitz, I've been working with the CSV files from pathfindercommunity.net. (My approach is to use XSLT to do the transformation rather than PowerShell.)

I have all four databases translated into

After a final bug is fixed in the import scripts in theory rwexport files can be created for all the information. However the realm would become stupidly large if you imported all the data, since you would end up with:

2,766 feats (5.8 MB import file)
5,271 monsters (47.4MB import file)
2,713 spells (11MB import file)
3,901 items (22MB import file)

On a test import, it took several hours to import all the data, and the master.realm increased in size by 5 or more GB!

The most obvious filter to start with is to make all the mythic feats a separate import, but how would all the other mass of topics be broken down to something more manageable.

The spells could possibly be reduced to maybe three blocks based on spell level. So you could import only the blocks required for your campaign.

A lot of the monsters appear to be named creatures, rather than generic monsters, so removing uniquely named monsters might reduce the database massively.

The intention would be to release the RWEXPORT files directly, since they contain no modifications to the information other than the format in which they are stored.

sorry for necroing this thread :) but couldn't find it.

is an general import of pathfinder monsters items etc already available now since we can import and export data now in realmworks?


All times are GMT -8. The time now is 08:56 AM.

Powered by vBulletin® - Copyright ©2000 - 2024, vBulletin Solutions, Inc.
wolflair.com copyright ©1998-2016 Lone Wolf Development, Inc. View our Privacy Policy here.