Lone Wolf Development Forums  

Go Back   Lone Wolf Development Forums > Realm Works Forums > Realm Works Discussion

Notices

Reply
 
Thread Tools Display Modes
Farling
Senior Member
 
Join Date: Mar 2013
Location: Greater London, UK
Posts: 1,725

Old April 15th, 2017, 03:46 AM
Hello RW Community,

I have been quietly working on a generic CSV import tool which (hopefully) requires no programming knowledge for the user.

For non-ASCII characters, the CSV file should use the UTF-8 encoding (there's probably an option in Excel, or your other favourite spreadsheet program, to encode as UTF-8 rather than some other form of encoding).

As of version 1.38, it is now possible to directly import Excel (xlsx) files. There is no longer the need to save them as CSV files.

There are several steps to using the tool:

0. Download the installer from https://github.com/farling42/csv2rw/....47/CSV2RW.exe

1. Prepare data from RealmWorks

1.1 Create a structure only export of the realm.
(Go to the "Share" tab, select "Manage Exports"; in the new window select the "Export Realm Structure Only" tab, type something into the "Name" box, prese the "Save Changes" button, then press the "Full Export" button to generate the RWstructure file.)

1.1.1 (Optional) Within RealmWorks, if you want additional labelled snippets to be used during your import, then you should add these into the Realm Works category definition (possibly making a copy of a predefined category definition) before creating the structure only export.
(Go to the "Manage" tab, select "Categories", choose the topic/article category which you want to use, select the tool icon and select "Create duplicate of category".)

1.2 If you want to load a collection of images and/or herolab portfolios and/or other data types, then the name of these files should be specified in one of the columns in the CSV file. A separate file will be loaded for each row of the CSV, although you can use the same filename in more than one row of the CSV if required.

2. Use the tool

2.1 Start up the CSV Import tool.

2.2 Press the "Load CSV" button and choose the file containing your CSV data.

2.3 Press the "Load Structure" button and choose the file containing the RealmWorks structure-only export file.

2.4 Select the required category type in the drop down menu below the "Load Structure" button to choose the category that you want your CSV data to be imported into. (This should be the "copy" that you made of the predefined category/article structure.)

2.5 Drag each of the CSV column names from the left panel to the corresponding place in the category in the right panel.

2.5.1 If the snippet should be revealed, then select the first radio box on the line (the circle). If the snippet should be marked as GM-only, then select the second radio box on the line.

2.5.2 If you want to remove a field from the category panel, then just right-click the field to remove it.

2.5.3 If you want a field to contain the same text in all occurrences of the topic/article, then you can add text directly into the field, or choose a particular tag from the drop-down list (instead of dragging one of the CSV column names across).

2.6 If you want all your topics/articles to be imported as children of another topic; then tick "Add Parent", choose a category for the parent, and enter a name for it (optionally with a prefix and suffix for the parent too). If you want to group entries under separate children, you can choose a particular CSV column to use for each group (e.g. you can group your spells with each school of spells under a different parent).

2.6.1 If you want a heirarchy created, then you can create multiple parents, choosing different CSV columns to fill in the title of the parent topics.

2.7 When you have all the fields defined, press the "GENERATE" button at the bottom-right of the window to generate the export file.

3 Import into RealmWorks

3.1 Import the generated file into your realm.
(Go to the "Share" tab, select "Import", click on the box next to "File to Import" and select the rwexport file generated by the CSV tool. Then press the "Import" button.)




4 Complex tasks

4.1 Custom section names
If you want the name of a section within a topic to be the text from a CSV column, then select "..." in the section title and set a column for the "First Section" box (leave the other two section boxes blank).

4.2 Multiple custom subsections
If you want to have the same section duplicated multiple times, then select "..." and set a column for the First, Second, and optionally the Third section box.
The First section defines the CSV column containing the title of the first section.
The Second section defines the CSV column containing the title of the second section.
The Third section (defaults to last column) defines the CSV column containing the title of the last section.
The snippets within the section should be set to use CSV columns between First and Second column.
For the second and subsequent sections, the same "offset" will be used for reading the text from the correct column of the CSV table. (example available in http://forums.wolflair.com/showpost....&postcount=295)

NOTES

It is no longer mandatory to make a copy of a predefined category, however if a copy is made (or a user defined category is used), then the additional greyed-out placeholder text will be displayed within this tool.

The first line of the CSV file is assumed to have the field names in it.

The bottom panel of the window shows the contents of the CSV file, so you can double-check that you are choosing the correct field.

For the domain selection parts of a topic, you can drag the CSV field name directly onto the drop-down marker in the right panel; or if you want the plain text then you can drop it into the "annotation" part of the window.

Note that if the value does not exist in the tags available from RealmWorks, then no value will be put into the RWexport file.

A field from the CSV panel can be dragged to more than one place in the topic/article panel if you want the same data in more than one place (usually used to set both the tag selector and the annotation field for the same item).

Daplunk has created one of his amazing HowTo videos which shows the tool in use. You can see it at https://youtu.be/A1i3FqHfi9E

Last edited by Farling; November 15th, 2018 at 04:01 PM.
Farling is online now   #1 Reply With Quote
Farling
Senior Member
 
Join Date: Mar 2013
Location: Greater London, UK
Posts: 1,725

Old April 15th, 2017, 03:47 AM
Release Notes

v1.47
Remove Minion_To_Master and Offspring_To_Parent relationships, since they are ignored by Realm Works on import.
Create one relationship for each match between the KEY field and the SEARCH column.
Add placeholder text to the relationship field boxes.

v1.46
This version introduces relationships. This allows you to specify a particular column of the table which will contain a value that can be linked to another row of the table.
(Note: old saved projects will not load into this version.)
The names of the relationships are currently based on the actual relationship which will be put into the XML file, rather than the "human friendly" names which are used inside Realm Works.
There are two fields which need to be filled in with a column. The first field identifies which column in the current row provides a search value. The second field identifies the column which will be searched (across the entire dataset) for a matching value - if a row is found, then a relationship will be created to that other topic.
(Note that if you are using selective parents and don't the other topic doesn't end up in the output file, then RW will generate an error during import.)

v1.45
Don't try to treat a parent topic title as a regular expression. It will now be treated as a fixed string, so "special" characters can now be used in topic titles (such as parentheses).

v1.44
All categories are now shown in the category drop-down list (sub-categories were previously being skipped).
It is now possible to set attributes on the public name of the topic/article (e.g. auto-accept, case-matching, match-priority and show-in-nav-pane).
The name of the case-matching has been shortened from "Auto Correct" to "Correct" to make the field shorter.

v1.43
This fixes a bug where a blank element was being put into the contents section when an "overlay" was present in the topic's definition in the structure.

v1.42 (the Meaning of Life edition)
This fixes a bug in that the reveal button on "other" names was set to be exclusive across all names. This is no longer the case.

v1.41
This version simply presents Tag_Standard snippets the same way as Hybrid_Tag snippets, in that a CSV column can be assigned to choose the corresponding domain tag for the field.

v1.40.1
A bug-fix release so that Realm Works doesn't crash when importing a file generated from an XLSX file containing multiple paragraphs in a single cell. (Multiple paragraphs are separated by blank lines in the cell.)

v1.40
Fix XLSX file reading so that "<" are copied to Realm Works properly, as are cells which contain only HTML.

v1.39
If you have BOLD, ITALIC, UNDERLINE and/or STRIKE-THROUGH in your XLSX files, then this formatting will be imported into Realm Works.

v1.38
Provide the option to read XLSX files as an alternative to CSV files.

v1.37
You can now save your current project, and load it back later.
The names of the CSV and RWEXPORT files are saved as part of the project.
Your mappings within the categories is also saved (although if no field is mapped to the topic's name, then that category won't get saved).

v1.36
Fixes a bug where the import would fail if the category has child sub-categories (e.g. the "Location" category has "Settlement" and "Adventure Area" sub-categories; so any attempt to create "Location" topics would fail).

v1.35
Fixes a bug where the import would fail if the category definition has sub-sections within sections.

v1.34
It is now possible to create multiple content snippets in the main body of a section.
Individual snippets can't be duplicated, but the general "contents" snippet of the section can now generate multiple snippets.

v1.33
Allow a section to be marked as starting in the "collapsed" state.
Allow a CSV column to be used as the name of a section within a topic.
Allow a set of CSV columns to be used to create multiple sections within a topic.
See http://forums.wolflair.com/showpost....&postcount=295 for more details on how to use this functionality.

v1.32
A small update to add a menu option to choose what to use for the field separator in the CSV file. You can choose from comma, semi-colon, or the windows list separator.

v1.31
The default separator in CSV files is now read from the Windows Regional Settings (the "list separator").
If you don't want to use your language's default list separator, it can be changed by opening windows Settings (windows 10) and finding the "Region & Language" section. Click on the "Administrative language settings" under "related settings". Go to the "Formats" tab, and click "Additional Settings...". Here you will see "List separator:" where you can change the default separator.

v1.30
You can now define more than one category to be exported from the tool (e.g. spells and items).
Each category definition can have its own set of parents.
The new "key" icon in the title of the category allows you to specify which rows in the CSV table should be displayed in this category (based on a column having a specific "key" value in it).
Note that separate parents are created for each category, so for example it isn't possible to create some spells and some items and have them appear under the same parent topic/article in Realm Works.

v1.29
The same category can now be chosen for each level of parents.
In theory (untested), you could even select the same category that you've chosen for the main topics.
This removes the previous restriction that all parents had to be using a different category.

v1.28
For each snippet, it is now also possible to set the Veracity of that snippet (True, Partially True, Untrue).
The selection of snippet style has been merged into the new Options menu.

v1.27
Adds support for numeric snippets.

v1.26
The "Import Name" text box has been replaced with a button to open a File Details dialog in which the import name, as well as other details, can be entered.

v1.25
Individual "<" are now treated properly so that they don't cause an error on import.
However, any field which starts with "<" and finishes with ">" is left unchanged, since it is assumed to be HTML that you wish to have imported directly.

v1.24
Don't include "overlay" elements in the generated RWexport file.
(There is no v1.23)

v1.22
You can now enter the name of the RW import before generating the output file.
The installer has been changed from QTIF to NSIS.

v1.21
You can now set the Snippet Style on the snippets that you're going to be importing.

v1.20
It is now possible to specify a hierarchy of parents. This means you could have spells organised by sub-school under a more general school topic; or you could have all your traveller planets stored per sub-sector and then per sector.
At the moment, a different category must be chosen for each parent, as well as for the main topic display.
Icons have been added to buttons to make their actions clearer.

v1.19
When defining a parent category, you now have the choice of being able to have more than one parent in your output file.
By dragging one of the CSV columns into the parent's name field, then there will be one parent created for each unique string in that column of the CSV file. The rows of the CSV table which contain that same value will then appear under that parent.
(For example, you could copy the "school" column of a spell CSV into the parent box, and then the RWexport file will define one parent topic/article for each spell school, and will have the spells for that school as children.)

v1.18
(thread comment #166)
True Names and Other Names can now be imported from a CSV file.
You'll find a new "+Name" button at the end of the topic title's row.
Pressing the button once will add a line where you can define the True Name for the topic.
Pressing it again will add additional rows for defining Other Names for the topic.
As usual, the relevant CSV column should be dragged to define the True/Other name. If the CSV field is empty on a particular row, then no alias will be added for that topic.
Several Other Names can be added and the various Realm Works name flags can be set differently on each Other Name (revealed, Case Matching, Auto Accept, Shown in Nav, Priority). So depending on the contents of your CSV file, you could have one CSV column for "revealed" Other Names, and a different CSV column for "unrevealed + priority avoid" Other Names.

v1.17
When reading images from a web URL, follow web redirects which are issued by the web server.
A progress bar is also displayed when performing the GENERATE, so that you can see how quickly (or slowly) it is working.

v1.16
When loading an image (see V1.11) it is now possible to specify a web URL instead of just a local file name. Note that the URL must point directly to the image, not to a web page which contains an image somewhere inside it.
The GENERATE button will now be disabled when the file generation process is in progress. This helps you know when the generation is complete (accessing files using a web URL can be very slow).

v1.14
When importing blocks of text, convert URLs so that they appear in Realm Works as clickable links.

v1.13
A mistake with github meant that the wrong code was tagged, so ignore this version (typical for number 13 anyway!)

v1.12
This version fixes a bug in the CSV reader where a " at the end of a line was not being handled properly.

v1.11
Need to load HeroLab portfolio files for all your favourite NPCs?
This version extends the file loading to allow loading data for the following snippet types: Portfolio, Statblock, Rich_Text, PDF, Audio, Video, HTML, Foreign.

v1.10
Load images into the RWexport file...
A column is chosen in the CSV file to use for the Picture or Smart-Image snippet.
The text in the column is the name of the file to load as part of the GENERATE process.
The file can either be a full or relative path to the file; if no path is given then the file must be in the same directory as the CSV file.
If the file does not exist, then a warning will be generated during the GENERATE process. The resulting file can still be loaded into Realm Works, in which case the snippet will show the name of the file even though the image itself will remain blank.
(Also, upgraded from Qt 5.8 to Qt 5.9.2 - this should have no impact on users.)

v1.9
Multi-line quoted cells are now supported in the CSV file.

Also, when you one of these fields then a blank line is created as a paragraph break when the data is being put into a Realm Works field which is either a Multi_Line or Label_Text field (the two fields which allow more than one line of text).

v1.8
This release fixes an issue where the RWEXPORT file would contain one or more blank topics if the original Realm Works structure file contains a category defined as a child of another category. (E.g. Location in the basic Pathfinder category structure).

v1.7
A small update which ignores blank lines in the CSV file. This prevents an empty topic being created in the RWEXPORT file which caused Realm Works to reject the import.

v1.6
A minor update which remembers the last path you used for each of the three file dialogs (each is remembered separately).
A tidy-up of code means that the text in the structure section is never mixed up with text in the topics section.
Multi-line text is now possible.
The lovely (R) has been added to all uses of the term Realm Works(R).

v1.5
This release allows you to type in some free text into a snippet, or choose a tag for a choice field, instead of having CSV data appear in the field.
You can't mix CSV data and free text in the same snippet (yet) though.
It also fixes a bug where sometimes a partition in the structure part of the XML file would wrongly get text attached to it.

v1.4
The entire topic, as well as the optional containing topic, can now be marked as REVEALED to players (using the circular button to the left of topic name).
This updates the existing functionality where individual snippets could be marked as revealed.

v1.3.1
A minor update to add validation to the new optional parent. You must enter a name, and you must choose a different category for the parent to the one chosen for the CSV data.

v1.3
This release see the addition of an option to allow a parent containing topic/article to be specified for all the topics which are to be created from the CSV file.

v1.2
The installer now adds the application to the Start Menu.
Fixed a bug with the layout of elements when there was a blank snippet in the template.

v1.1
Blank cells in the CSV file don't produce unnecessary elements in the output file.
Ensure that the annotation child appears before the tag_assign child in relevant elements.
Ensure the name of the topic/article is assigned one of the CSV fields.
Remove the "GM only" button from each snippet, since it doesn't do anything.

v1.0.0
The top drop-down menu does nothing.


TO DO
Improve the CSV value => Domain mapping so that partial matches are possible (comma separated lists of multiple tags is already supported).

Won't Do
The following items should really be handled by setting up the required category definitions within Realm Works before exporting the structure:
Provide support for Labeled_Text fields.
Allow the addition of additional text (and other) snippets to an imported category.
Allow the addition of new sections and subsections to an imported category.

The following idea can be achieved by re-organising the contents of the CSV file:
Allow predefined text and CSV field names to be mixed together within certain text snippet types.

Last edited by Farling; November 15th, 2018 at 04:00 PM. Reason: v1.41
Farling is online now   #2 Reply With Quote
Bidmaron
Senior Member
 
Join Date: Dec 2010
Posts: 425

Old April 15th, 2017, 05:50 AM
This looks like a very great addition to our toolkits. Can you elaborate on the instruction regarding how to do domain selection?
Any good CSv sources out there?
Bidmaron is offline   #3 Reply With Quote
Farling
Senior Member
 
Join Date: Mar 2013
Location: Greater London, UK
Posts: 1,725

Old April 15th, 2017, 06:24 AM
Quote:
Originally Posted by Bidmaron View Post
This looks like a very great addition to our toolkits. Can you elaborate on the instruction regarding how to do domain selection?
Any good CSv sources out there?
When you have a snippet type which requires a tag, then my tool will take the text from the CSV column and try to find a matching value for the tag.

From the list of CSV fields in the left panel, drag the name of the field across onto the box that contains the black triangle (to the right of the field name).

For example, with the Pathfinder game, if you have a CSV for spells, then the column for the "spell subschool" might contain the text "scrying". My import tool would take the text "scrying" and find the correct tag to put into RealmWorks for this text.
Farling is online now   #4 Reply With Quote
Absintheminded78
Member
 
Join Date: Jan 2016
Location: Sydney, Australia
Posts: 48

Old April 16th, 2017, 05:04 AM
I must be doing something wrong somewhere along the line as each time I go to import after I've generated the export file I get the message that the category does not exist, any advice? Tried with feats and also magic items.
Absintheminded78 is offline   #5 Reply With Quote
Farling
Senior Member
 
Join Date: Mar 2013
Location: Greater London, UK
Posts: 1,725

Old April 16th, 2017, 09:44 AM
Quote:
Originally Posted by Absintheminded78 View Post
I must be doing something wrong somewhere along the line as each time I go to import after I've generated the export file I get the message that the category does not exist, any advice? Tried with feats and also magic items.
Hi, sorry to hear you're having problems.

Is the message about the category not existing coming from my tool or from Realm Works?

Which fields from a CSV are you trying to copy? It is best not to use fields which have lots of HTML mark-up in them since this will likely conflict with the HTML being added by the conversion tool, and also with what Realm Works might be expecting.
Farling is online now   #6 Reply With Quote
Absintheminded78
Member
 
Join Date: Jan 2016
Location: Sydney, Australia
Posts: 48

Old April 16th, 2017, 02:08 PM
Quote:
Originally Posted by Farling View Post
Hi, sorry to hear you're having problems.

Is the message about the category not existing coming from my tool or from Realm Works?

Which fields from a CSV are you trying to copy? It is best not to use fields which have lots of HTML mark-up in them since this will likely conflict with the HTML being added by the conversion tool, and also with what Realm Works might be expecting.
From Realm Works, when I go to import the file generated by your tool it crashes somewhere between 3-5% and the error report states it's because category does not exist.
I'm just using basic text fields from the CSV files from http://www.pathfindercommunity.net/home/databases/, I have already managed to get the spells imported with EightBitz's script so I attempted to bring in the magic items and feats, both gave that error.
Absintheminded78 is offline   #7 Reply With Quote
Farling
Senior Member
 
Join Date: Mar 2013
Location: Greater London, UK
Posts: 1,725

Old April 16th, 2017, 03:20 PM
Quote:
Originally Posted by Absintheminded78 View Post
From Realm Works, when I go to import the file generated by your tool it crashes somewhere between 3-5% and the error report states it's because category does not exist.
I'm just using basic text fields from the CSV files from http://www.pathfindercommunity.net/home/databases/, I have already managed to get the spells imported with EightBitz's script so I attempted to bring in the magic items and feats, both gave that error.
Is it possible for you to provide the RWEXPORT file generated by my tool so that I can have a look at its output? I'd like to see what is happening with the tool.
Farling is online now   #8 Reply With Quote
Absintheminded78
Member
 
Join Date: Jan 2016
Location: Sydney, Australia
Posts: 48

Old April 16th, 2017, 03:28 PM
Quote:
Originally Posted by Farling View Post
Is it possible for you to provide the RWEXPORT file generated by my tool so that I can have a look at its output? I'd like to see what is happening with the tool.
Sure, thanks for having a look

https://drive.google.com/drive/folde...Vk?usp=sharing
Absintheminded78 is offline   #9 Reply With Quote
Farling
Senior Member
 
Join Date: Mar 2013
Location: Greater London, UK
Posts: 1,725

Old April 17th, 2017, 05:12 AM
Absinthe,

I'm trying to import your "feat text.rwexport" file at the moment into a newly created "pathfinder" realm. My attempt is currently at 23% and still going. What type of realm are you trying to import into?

The first improvement to the import would be to drag the feat type across to the "selector" in the feat template. The selector is the box containing the triangle. (It will detect multiple tags for the same field in the CSV if they are separated by commas within the CSV field.) You can have the same CSV column appear more than once in the template. See the attached picture
Attached Images
File Type: png imported-screenshot2.png (45.3 KB, 35 views)

Last edited by Farling; April 17th, 2017 at 03:17 PM.
Farling is online now   #10 Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -8. The time now is 04:00 AM.


Powered by vBulletin® - Copyright ©2000 - 2018, vBulletin Solutions, Inc.
wolflair.com copyright ©1998-2016 Lone Wolf Development, Inc. View our Privacy Policy here.