Lone Wolf Development Forums

Lone Wolf Development Forums (http://forums.wolflair.com/index.php)
-   Realm Works Discussion (http://forums.wolflair.com/forumdisplay.php?f=67)
-   -   RealmWorks CSV Import Tool (http://forums.wolflair.com/showthread.php?t=58156)

Farling November 6th, 2018 02:41 PM

Quote:

Originally Posted by Sertas (Post 272370)
Can you use the same structure file to add different excel files. I have three excel files and I'm trying to add them one at a time to my realm but the second file isn't actually importing.

Do you get any specific error?

Is each import using a different import name? I think there is a button next to the GENERATE button which allows you to define various things (including the name of the import) about the realm that will be added to the import file.

Sertas November 7th, 2018 03:31 PM

Each file has a different import name.

I'm getting the following error:

The following errors were encountered during import:

C++ Exception: Not in a span at line 1621 in file utility.cpp

Farling November 7th, 2018 03:38 PM

Quote:

Originally Posted by Sertas (Post 272420)
Each file has a different import name.

I'm getting the following error:

The following errors were encountered during import:

C++ Exception: Not in a span at line 1621 in file utility.cpp

Oh, that is a problem in the formatting within the file. Does your source data contain HTML markup? That sometimes causes issues with the correct encoding for putting into the RW import file.

Sertas November 7th, 2018 04:35 PM

I'm not sure if HTML is there. If so I'll not include it and see what happens. Thanks for the quick responses. This tool is AMAZING.

Sertas November 8th, 2018 03:24 AM

Thanks for the direction. I didn't import a column in my excel file and it imported just fine.

Again, love the tool. It's saving me so much time. Now, I just need more data....lol

Farling November 11th, 2018 08:01 AM

Release 1.45 is out

I've fixed a bug where a parent topic title was being processed as if it was a regular expression rather than a fixed string.

The title can now contain special characters such as parentheses without causing import problems.

Merion November 13th, 2018 03:11 PM

Thanks again for this great tool!

BTW, can someone with mod rights please make this topic sticky!?

Farling November 14th, 2018 05:25 AM

Release 1.46 is available

(Note: old saved projects will not load into this version.)

(See point 2 in http://forums.wolflair.com/showpost....&postcount=328)
(Zaphod, this is for you from post #61, http://forums.wolflair.com/showpost....2&postcount=61)

This version allows you to create relationships in the output file.

You only need to specify the relationship in one direction; Realm Works will automatically create the reverse relationship (if required).

The names of the relationships in the tool are the names supported in the XML file, rather than the names that you might see in Realm Works (there isn't an exact one-to-one map).

The simplest example is using topic names (although you can actually choose any field to search on)...

In your data you need to add one or more additional columns which will contain the name of the topic to which a relationship should be made.

In the tool, you will specify the column containing the name of the related topic, and also the column to be search for that name. (The flexibility in the tool means that you can link two articles not merely based on the actual topic name, but on any unique column.)

(Of course, immediately after writing the above parenthetical clause, I realise that the tool could be extended to automatically link to ALL matching topics instead of only the first match. - see V1.47)

You can set up multiple relationships, each possibly of a different type. If the "link" column in the model is left empty in some rows, then no relationship will be created for the topics created from those rows.


The supported relationships are:

Master_To_Minion / Minion_To_Master (RW calls them "Comprises or Belongs" and "Belongs To or Within"):
(V1.47 removed "Minion_To_Master" since RW ignores it during import.)
= Affiliation / Member
= Depends On / Depended Upon By
= Employer / Employee
= Master / Minion
= Organization / Chapter
= Owner / Subsidiary
= Parent / Child
= Residence / Resident
= Supervisor / Subordinate"

Generic:
= Peer
= Equivalence
= Personal
= General Family

Public_Attitude_Towards / Private_Attitude_Towards:
= Gracious
= Friendly
= Pleased
= Neutral
= Annoyed
= Angry
= Hostile

Arbitrary / Union / Parent_To_Offspring / Offspring_To_Parent
(V1.47 removed "Offspring_To_Parent" since RW ignores it during import.)
= (no qualifiers for these)

Farling November 15th, 2018 03:03 PM

Release 1.47 is available

This version removes the Minion_To_Master and Offspring_To_Parent options for relationships, since Realm Works ignores them during import. These relationships can be created in the opposite direction using Master_To_Minion and Parent_To_Offspring.

When creating relationships, one relationship will now be created for each match between the KEY field and the SEARCH column. (V1.46 only created a relationship for the first match).

The KEY and SEARCH fields for a relationship now have some placeholder text to give a hint as to their use.

----

An example
You have a table of characters, which includes columns: name, father, mother.

You could set a single relationship for your Character topic category of "Parent_To_Offspring", setting the KEY field to "name" and the SEARCH field to "father".
A second relationship of the same type could also be set up for "name" to "mother".

JustinThomason November 18th, 2018 04:54 PM

Farling,

I tried the latest version of the CSV importer, and ran into problems importing into Realm Works.

I'm kind of pushing the original intent of the program. I'm working to put all of Rappan Athuk into a realm. My intent is to build a structural skeleton in Excel, then import that using your tool so that when I start entering the actual text of the (ginormous) module place holder topics for all the specific areas already exist and allow easy linking regardless if I've entered the text for a specific location or not.

I've got everything mapped out in the CSV, and in fact, I've imported essentially the same data in a prior version of the program. That version of the tool didn't allow setting the linking priority on the main titles of topics, so I was getting a lot of false positives for generic location names like "Entryway".

Anyway, I made a new realm and created a new export file using the new version of the tool. When I attempted to import into the clean realm, it got to 22% and crashed.

I have all the pertinent files - my realm structure file, the original CSV, and the export file created by the tool. I can forward any or all of this to you if it would help your diagnosis.

Thanks again for your help to the community creating and maintaining this tool - it is SUPER useful!

Farling November 18th, 2018 09:50 PM

Quote:

Originally Posted by JustinThomason (Post 272848)
I have all the pertinent files - my realm structure file, the original CSV, and the export file created by the tool. I can forward any or all of this to you if it would help your diagnosis.

Hi JT,

All three files would be useful, so I can see what might be going wrong.

Merion November 18th, 2018 11:09 PM

Quote:

Originally Posted by Merion (Post 272649)
BTW, can someone with mod rights please make this topic sticky!?

Thanks, btw, to whoever stickied this!

Rone November 19th, 2018 05:29 AM

You are most welcome!

JustinThomason November 19th, 2018 05:54 AM

1 Attachment(s)
Quote:

Originally Posted by Farling (Post 272853)
All three files would be useful, so I can see what might be going wrong.

Zipped them for easier transfer.

The main import topic type was set to "Dungeon Room" and the parent type was set to "Dungeon Level." Otherwise everything should be pretty self-explanatory from the column names.

Let me know if you have any questions

daplunk November 19th, 2018 12:29 PM

There's an issue with adding multiple names. Joe and I have both experienced crashes with it.

Farling November 23rd, 2018 10:49 PM

Quote:

Originally Posted by JustinThomason (Post 272848)
I've got everything mapped out in the CSV, and in fact, I've imported essentially the same data in a prior version of the program. That version of the tool didn't allow setting the linking priority on the main titles of topics, so I was getting a lot of false positives for generic location names like "Entryway".

Hi JT,

I've had a look in your RWEXPORT file and I can see some topics appearing more than once. With recent changes to support relationships, each row of the CSV is now allocated a unique topic number (either the row number, or the row number -1, I don't remember). This means that if the row is selected more than once to appear in the file, then the final file won't import.

I was aware that this limitation might not be ideal, and could cause problems; but I also thought that nobody would probably want the same CSV row appearing more than once in the final realm.

In your source CSV table, there are some duplicate entries, which you probably don't want in your final realm anyway :-)

Code:

(topics 22-100 are in both)
Level 0B: The Cloister of the Frog-God
Level 0B: The Cloister of the Frog-God: The Cloister
...
Level 0B: The Cloister of the Frog-God: Within the Vaults

(topics 365-389 are in both)
Level 4B: The Gut
Level 4B: The Gut: A. Zombie Stables
...
Level 4B: The Gut: F. The Gates of Gloom

(topics 645-690 are in both)
Level 9D: The Bloodways
Level 9D: The Bloodways (Supplemental Areas)


Farling November 23rd, 2018 11:22 PM

Quote:

Originally Posted by daplunk (Post 272880)
There's an issue with adding multiple names. Joe and I have both experienced crashes with it.

Is this a crash of the tool, or a crash of Realm Works during import?

Can you give any examples of what might crash?

Farling November 24th, 2018 01:14 AM

Version 1.48 is available

This version doesn't add new functionality, it merely performs some additional checking during generation. An error window will be displayed showing any topics which would appear more than once in the generated RWEXPORT file, and tell you that the Realm Works will reject the import.

This should help JustinThomason avoid creating duplicate topics in his realm.

JustinThomason November 25th, 2018 05:31 PM

Farling,

I don't quite understand what is wrong with my CSV. Is it because there are room names that are the same (like "Rat Tunnels" or "Entrance"?). If that's the case, that's an annoyance of this module and, frankly, lots of other modules as well - I might be able to deal with it, but it would be a pain. However, When I tried to import, I got references that topics were said to be used more than once, but I could only find a single instance of the name in the CSV.

Is it because the parent names repeat? If that's the case, how can you have multiple topics import under the same parent?

Sorry I'm being a little dense about this - I just don't quite grok the minutiae.

Failing that, would it be possible to get the last version before Relationships were added? I'm not sure if it would warrant a feature request to have Relationships be optional, but If I can get this skeleton entered it will make entering the 550+ pages SOOOOO much easier.

Once again, thanks so much for your help and the time you've given to the community.

daplunk November 25th, 2018 05:54 PM

Quote:

Originally Posted by Farling (Post 273022)
Is this a crash of the tool, or a crash of Realm Works during import?

Can you give any examples of what might crash?

A crash during import. I would need to replicate the issue. But in our experience adding multiple names to snippets is a sure fire way to get the import to crash and in all cases we have been able to get around the crash by removing multiple names from the equation and nothing else.

Farling November 25th, 2018 10:31 PM

Quote:

Originally Posted by JustinThomason (Post 273073)
I don't quite understand what is wrong with my CSV. Is it because there are room names that are the same (like "Rat Tunnels" or "Entrance"?). If that's the case, that's an annoyance of this module and, frankly, lots of other modules as well - I might be able to deal with it, but it would be a pain. However, When I tried to import, I got references that topics were said to be used more than once, but I could only find a single instance of the name in the CSV.

Hi,

The "topic more than once" error is when the same row in the CSV table is
being inserted more than once. You can have the same name on different rows without causing any issues (other than if you try to create a relationship, since it will pick the first [or all?] topics).

The problem I saw was that the way you had the parents set up, the same row from the table was being added to several different parents.

e.g. several topics were being put under both the parents "Level 0B: The Cloister of the Frog-God" title and "Level 0B: The Cloister of the Frog-God: The Cloister".

I'm not sure if you intended the same topics to appear twice in your realm.

Farling November 25th, 2018 10:32 PM

Quote:

Originally Posted by daplunk (Post 273078)
A crash during import. I would need to replicate the issue. But in our experience adding multiple names to snippets is a sure fire way to get the import to crash and in all cases we have been able to get around the crash by removing multiple names from the equation and nothing else.

Do you have an example of the generated RWexport file that I could examine, so that I can see what is being put into the file?

daplunk November 25th, 2018 11:15 PM

I just tried to replicate it using a file I had made a few months ago. I deleted the original export so this was a fresh from using the csv. Unable to replicate which is a good thing. If I can make it happen again in the future I'll let you know.

Joe might have some crashing exports left. I tend to clean up the stuff that doesn't work.

JustinThomason December 1st, 2018 02:30 PM

2 Attachment(s)
Farling,

I'm really sorry to continue poking at this, but I have gone over my CSV multiple times and I do not see any problem with how I have structured the file. I decided to copy out the lines dealing with the Cloister of the Frog-God as a test since that was definitely causing problems. I'm attaching this as a .txt file so it can be reviewed.

Just using a few lines from each section, I'd expect the import to look something like this:

Quote:

Level 0B: The Cloister of the Frog-God
---Background
Level 0B: The Cloister of the Frog-God: The Cloister
---Entrance
---Upper Temple
---Refectory
.
.
.
Level 0B: The Cloister of the Frog-God: Within the Vaults
---The Lower Temple
---Holding Cells
---Laboratory
---Lost Catacombs
.
.
.
In fact, this is exactly how this same CSV imported the first time I used the tool. That was before you had added the controls to keep the main topic name from being an auto link, so I was having lots of false positives importing the text, but I still have that Realm, and it imported exactly as I would have expected.

http://forums.wolflair.com/attachmen...1&d=1543706642

Sorry to be a pain. I just really don't understand how else I could structure the CSV to get the nested structure that I need. I don't see how I'm calling on the same lines multiple times. Thanks again for your help.

Happydevil43 December 2nd, 2018 08:11 PM

Question:

I may be crazy (actually no don't answer that.... :-) ), but I am sure that in one of the earlier builds there was an option to make a snippet a GM instruction... that seems to have disappeared ?

I can see snippet styles etc...

Did I miss something ?

Parody December 2nd, 2018 09:37 PM

Quote:

Originally Posted by Happydevil43 (Post 273534)
I may be crazy (actually no don't answer that.... :-) ), but I am sure that in one of the earlier builds there was an option to make a snippet a GM instruction... that seems to have disappeared ?

It's still there for Text Snippets: Snippet Options/GM Directions/Convert to GM Directions-only or Ctrl-G.

Happydevil43 December 2nd, 2018 10:00 PM

Quote:

Originally Posted by Parody (Post 273538)
It's still there for Text Snippets: Snippet Options/GM Directions/Convert to GM Directions-only or Ctrl-G.

Ahh - Thanks - It was a tick box.....I think ... I didn't dig deep enough.

Farling December 3rd, 2018 01:33 PM

Version 1.49 is out

This version fixes a bug that JustinThomason found.

The string matching wasn't working properly when deciding which topics should be created as children of which parents. (The API call had been changed to prevent "regular expression" special characters from having any effect, in order to fix a previous bug report.)

If a possible parent name was present within another parent name, then both parents would have the topic created beneath them.

e.g. with the data lines:
Code:

Parent,Prefix,Name,Suffix,Alt Name,Alt Name 2,Status Tag
Level 0B: The Cloister of the Frog-God,0B,Background,0B,0B,Area 0B,Empty
Level 0B: The Cloister of the Frog-God: The Cloister,CL1,Entrance,0B-CL,CL1,Area CL1,Empty
Level 0B: The Cloister of the Frog-God: Within the Vaults,DL1A,The Lower Temple of Tsathogga,0B-DL1,DL1A,Area DL1A,Empty

All three topics would appear under "The Cloister of the Frog-God" (since that title appears somewhere within the text of parent column), as well as appearing under their own (correct) specified parent.

JustinThomason December 3rd, 2018 09:05 PM

Perfect! The new version works like a champ!

Once again, Thank you so much Farling - this tool is top notch!

ErinRigh December 4th, 2018 11:20 AM

Farling, you are a true treasure to the community, mate, thanks for this

Happydevil43 December 6th, 2018 12:33 AM

1 Attachment(s)
Quote:

Originally Posted by Happydevil43 (Post 273540)
Ahh - Thanks - It was a tick box.....I think ... I didn't dig deep enough.

2 Questions - and I admit its been a while since I used this.... I just downloaded 1.49....

I am sure this file worked the last time I used it.. (I was starting a new realm)…

1. The GM Snippet thing - I cant find that in Farlings awesome tool - It used to have it there. Yes I can do it once it was imported - but thought I could do it on import. I cannot see any function anywhere for it.


2. Now I have the attached errors - I for the life of me can't see what's wrong. I am going to go back to an older version and see what happens. I can see one entry for each one...

Happydevil43 December 6th, 2018 01:07 AM

Well blow me with a feather... after a few goes (and changing nothing), its decided to work...
The only thing I did different - I saved after I put each column in its corresponding place

Still cant find the GM Snippets.

Farling December 6th, 2018 01:27 PM

Quote:

Originally Posted by Happydevil43 (Post 273653)
Still cant find the GM Snippets.

I think there is mention of "GM only" in the usage notes, but the button was removed very early on because it didn't do anything.

I have something working that generates GM-only parts to snippets, but it needs some more testing before I'll release it.

Happydevil43 December 6th, 2018 02:57 PM

Quote:

Originally Posted by Farling (Post 273677)
I think there is mention of "GM only" in the usage notes, but the button was removed very early on because it didn't do anything.

I have something working that generates GM-only parts to snippets, but it needs some more testing before I'll release it.

Excellent!

You are a god amongst men.

Farling December 13th, 2018 02:18 PM

Version 2.0 is out

The tool now supports generating GM Directions from CSV columns (including creating multiple snippets in the same section from multiple columns).
The tool has moved to the latest version of Qt (5.12.0) which supports 64-bit on Windows.
The tool has been renamed to reflect the fact that it no longer only supports CSV files.

Happydevil43 December 13th, 2018 03:41 PM

Quote:

Originally Posted by Farling (Post 273943)
Version 2.0 is out

The tool now supports generating GM Directions from CSV columns (including creating multiple snippets in the same section from multiple columns).

You da man! Love your work! This is going to save me hours!

Thank you.

ErinRigh December 22nd, 2018 11:58 AM

1 Attachment(s)
Hey mate, I'm having some trouble with version 2.0 I get an error and CTD when I try to import, I attached the relevant files I think, anything else you need to help?

Farling December 22nd, 2018 12:32 PM

Quote:

Originally Posted by ErinRigh (Post 274255)
Hey mate, I'm having some trouble with version 2.0 I get an error and CTD when I try to import, I attached the relevant files I think, anything else you need to help?

Hi Erin,

Can you do a divide-and-conquer to work out which part of the CSV is causing problems? It is 1,387 records long.

e.g. try doing only first half, and then only second half. Then split up the offending half, etc.

ErinRigh December 22nd, 2018 12:45 PM

OK, I'll check it out

ErinRigh December 22nd, 2018 04:23 PM

1 Attachment(s)
Well it took me the better of 3.5 hours, but I narrowed it down to this record (attached), though I don't know what is causing it, it seems to be something to do with the Callsign field when added to other names


All times are GMT -8. The time now is 10:25 AM.

Powered by vBulletin® - Copyright ©2000 - 2024, vBulletin Solutions, Inc.
wolflair.com copyright ©1998-2016 Lone Wolf Development, Inc. View our Privacy Policy here.