Lone Wolf Development Forums

Lone Wolf Development Forums (http://forums.wolflair.com/index.php)
-   Realm Works Discussion (http://forums.wolflair.com/forumdisplay.php?f=67)
-   -   RealmWorks CSV Import Tool (http://forums.wolflair.com/showthread.php?t=58156)

Farling April 15th, 2017 02:46 AM

RealmWorks CSV Import Tool
 
Hello RW Community,

I have been quietly working on a generic CSV import tool which (hopefully) requires no programming knowledge for the user.

For non-ASCII characters, the CSV file should use the UTF-8 encoding (there's probably an option in Excel, or your other favourite spreadsheet program, to encode as UTF-8 rather than some other form of encoding).

As of version 1.38, it is now possible to directly import Excel (xlsx) files. There is no longer the need to save them as CSV files.

There are several steps to using the tool:

0. Download the installer from https://github.com/farling42/csv2rw/...RWimporter.exe

1. Prepare data from RealmWorks

1.1 Create a structure only export of the realm.
(Go to the "Share" tab, select "Manage Exports"; in the new window select the "Export Realm Structure Only" tab, type something into the "Name" box, prese the "Save Changes" button, then press the "Full Export" button to generate the RWstructure file.)

1.1.1 (Optional) Within RealmWorks, if you want additional labelled snippets to be used during your import, then you should add these into the Realm Works category definition (possibly making a copy of a predefined category definition) before creating the structure only export.
(Go to the "Manage" tab, select "Categories", choose the topic/article category which you want to use, select the tool icon and select "Create duplicate of category".)

1.2 If you want to load a collection of images and/or herolab portfolios and/or other data types, then the name of these files should be specified in one of the columns in the CSV file. A separate file will be loaded for each row of the CSV, although you can use the same filename in more than one row of the CSV if required.

2. Use the tool

2.1 Start up the CSV Import tool.

2.2 Press the "Load CSV" button and choose the file containing your CSV data.

2.3 Press the "Load Structure" button and choose the file containing the RealmWorks structure-only export file.

2.4 Select the required category type in the drop down menu below the "Load Structure" button to choose the category that you want your CSV data to be imported into. (This should be the "copy" that you made of the predefined category/article structure.)

2.5 Drag each of the CSV column names from the left panel to the corresponding place in the category in the right panel.

2.5.1 If the snippet should be revealed, then select the first radio box on the line (the circle). If the snippet should be marked as GM-only, then select the second radio box on the line.

2.5.2 If you want to remove a field from the category panel, then just right-click the field to remove it.

2.5.3 If you want a field to contain the same text in all occurrences of the topic/article, then you can add text directly into the field, or choose a particular tag from the drop-down list (instead of dragging one of the CSV column names across).

2.6 If you want all your topics/articles to be imported as children of another topic; then tick "Add Parent", choose a category for the parent, and enter a name for it (optionally with a prefix and suffix for the parent too). If you want to group entries under separate children, you can choose a particular CSV column to use for each group (e.g. you can group your spells with each school of spells under a different parent).

2.6.1 If you want a heirarchy created, then you can create multiple parents, choosing different CSV columns to fill in the title of the parent topics.

2.7 When you have all the fields defined, press the "GENERATE" button at the bottom-right of the window to generate the export file.

3 Import into RealmWorks

3.1 Import the generated file into your realm.
(Go to the "Share" tab, select "Import", click on the box next to "File to Import" and select the rwexport file generated by the CSV tool. Then press the "Import" button.)

https://user-images.githubuserconten...eb6b723f4f.png


4 Complex tasks

4.1 Custom section names
If you want the name of a section within a topic to be the text from a CSV column, then select "..." in the section title and set a column for the "First Section" box (leave the other two section boxes blank).

4.2 Multiple custom subsections
If you want to have the same section duplicated multiple times, then select "..." and set a column for the First, Second, and optionally the Third section box.
The First section defines the CSV column containing the title of the first section.
The Second section defines the CSV column containing the title of the second section.
The Third section (defaults to last column) defines the CSV column containing the title of the last section.
The snippets within the section should be set to use CSV columns between First and Second column.
For the second and subsequent sections, the same "offset" will be used for reading the text from the correct column of the CSV table. (example available in http://forums.wolflair.com/showpost....&postcount=295)

NOTES

It is no longer mandatory to make a copy of a predefined category, however if a copy is made (or a user defined category is used), then the additional greyed-out placeholder text will be displayed within this tool.

The first line of the CSV file is assumed to have the field names in it.

The bottom panel of the window shows the contents of the CSV file, so you can double-check that you are choosing the correct field.

For the domain selection parts of a topic, you can drag the CSV field name directly onto the drop-down marker in the right panel; or if you want the plain text then you can drop it into the "annotation" part of the window.

Note that if the value does not exist in the tags available from RealmWorks, then no value will be put into the RWexport file.

A field from the CSV panel can be dragged to more than one place in the topic/article panel if you want the same data in more than one place (usually used to set both the tag selector and the annotation field for the same item).

Daplunk has created one of his amazing HowTo videos which shows the tool in use. You can see it at https://youtu.be/A1i3FqHfi9E

Donations for efforts can be made via any of Ko-Fi or Paypal

Farling April 15th, 2017 02:47 AM

Release Notes

v2.18
Images can now be embedded in XLSX files.
The data preview window will show only a blank cell for any cells containing images.
The generated RWexport file will have the image stored in PNG format.

v2.17
Change CSV file reading to default to UTF-8 codec, and read newlines properly (QFile::Text on QFile::open)

v2.16
(There is no 2.15)
This version adds a function "row.hasColumn(name)" to the JavaScript support for derived columns.

v2.14
When a complex JSON file is loaded, there is now an option to choose the top-level array to be used as the data source.

v2.13
Save and Load actions have been added for "derived" columns. This allows you to save the custom column definitions and re-use them in another import project. (This means you don't have to copy/paste each individual definition for a new project.)

v2.12
Adds "derived" columns, where the value is based on a javascript expression that you specify. The javascript expression can examine the values of other columns in the current row using the syntax: row.column('columnname')
The new derived columns can be managed via the new "custom columns" button.
Fixes a bug that prevented excel files from being loaded.

v2.11
Adds the ability to launch the tool by double-clicking on your project's csv2rw file.
Adds basic support for JSON and YAML data files. The complex hierarchy is reduced to a simple overall set of column names, using "/" to separate each step of the hierarchy that contains each specific data value.

v2.9
Aliases set up in a topic definition are now saved as part of the project.
From now on, projects saved in earlier versions of tool will load into later versions of the tool.

v2.8
Adds an entry in the "Options" menu that will force the file's format_version attribute to be set to 3.
(Realm Works 254 appears to generate export files with format_version of 4, but can't read such files during import.)

v2.7
User can now choose which sheet inside an Excel document to be used as the data source (only one can be used).
Provide warning about unsaved work when attempting to load a new project, load a new data file, load a new structure file, or choosing a different worksheet in the current excel data set.
Ensure that the data mappings are shown when loading a project.

v2.6.1
This is a bug fix release.
It fixes the problem where a cell that is not HTML which contains any < or > or & characters does not import properly.

v2.6
A "*" is put in the window title bar if modifications have been made since the last load/save of a project.
On quitting the application, a prompt is displayed if a project has not been saved since the last change.

v2.5
When a project has been saved or loaded, the name of the project is displayed in the window title.
Fixes a bug where "fixed text" in a field definition was not being displayed after changing categories or loading a saved project.
Fixes a bug where the credit box in the File Details windows was being populated with the Requirements text when the window was opened.

v2.4.2
This is a bug fix release. It is now possible to save projects that use column 14 from the table, and the saved project includes the parentage that you've set up.
No projects saved with an earlier version can be loaded into this version.

v2.4.1
This version is a bug fix release which gets the Save Project action to save data in a format that can be read properly by Load Project.

v2.4
The internal warnings and critical messages are now reported in an error window - e.g. when a source data cell doesn't match a known value for tag, then this is reported in the error window.
It is also now possible to configure a field to fill in the annotation for an asset snippet type without requiring you to also configure a field for that actual asset type (e.g. picture or smart image).

v2.3
When reading an Excel spreadsheet, the default cell formatting is used as a starting point when trying to determine what formatting should be applied to text within each cell.

v2.2
Annotations will now be produced on all snippet types (except multi-line and labeled text types).
Numeric fields will now be checked that they contain only characters allowed in numbers.

v2.1
When generating aliases, the tool now checks that the name of the alias does not match either the topic title or an alias within the same topic.
A warning is displayed, and the alias is not added to the RWexport file.
(This prevents a crash of Realm Works during import.)

v2.0
GM Directions can now be generated from the import data.
The tool is now a 64-bit tool (and uses the latest Qt 5.12.0).
The tool has been renamed to RWImporter since it handles more than only CSV files.

v1.49
This fixes a bug (introduced in 1.45) where a topic was being created more than once under different parents, where the name of parent is a shorter version of another parent.

v1.48
An error dialog will be displayed if the same ROW of the source data would generate more than one topic in the output file. An output file will still be generated, but Realm Works will fail to import the file.
A restriction was added when implementing relationships (in 1.46) which meant that each ROW of the source data can only generic one topic in the output file (otherwise it would be impossible to know with which of multiple topics a relationship was being created.)

v1.47
Remove Minion_To_Master and Offspring_To_Parent relationships, since they are ignored by Realm Works on import.
Create one relationship for each match between the KEY field and the SEARCH column.
Add placeholder text to the relationship field boxes.

v1.46
This version introduces relationships. This allows you to specify a particular column of the table which will contain a value that can be linked to another row of the table.
(Note: old saved projects will not load into this version.)
The names of the relationships are currently based on the actual relationship which will be put into the XML file, rather than the "human friendly" names which are used inside Realm Works.
There are two fields which need to be filled in with a column. The first field identifies which column in the current row provides a search value. The second field identifies the column which will be searched (across the entire dataset) for a matching value - if a row is found, then a relationship will be created to that other topic.
(Note that if you are using selective parents and don't the other topic doesn't end up in the output file, then RW will generate an error during import.)

v1.45
Don't try to treat a parent topic title as a regular expression. It will now be treated as a fixed string, so "special" characters can now be used in topic titles (such as parentheses).

v1.44
All categories are now shown in the category drop-down list (sub-categories were previously being skipped).
It is now possible to set attributes on the public name of the topic/article (e.g. auto-accept, case-matching, match-priority and show-in-nav-pane).
The name of the case-matching has been shortened from "Auto Correct" to "Correct" to make the field shorter.

v1.43
This fixes a bug where a blank element was being put into the contents section when an "overlay" was present in the topic's definition in the structure.

v1.42 (the Meaning of Life edition)
This fixes a bug in that the reveal button on "other" names was set to be exclusive across all names. This is no longer the case.

v1.41
This version simply presents Tag_Standard snippets the same way as Hybrid_Tag snippets, in that a CSV column can be assigned to choose the corresponding domain tag for the field.

v1.40.1
A bug-fix release so that Realm Works doesn't crash when importing a file generated from an XLSX file containing multiple paragraphs in a single cell. (Multiple paragraphs are separated by blank lines in the cell.)

v1.40
Fix XLSX file reading so that "<" are copied to Realm Works properly, as are cells which contain only HTML.

v1.39
If you have BOLD, ITALIC, UNDERLINE and/or STRIKE-THROUGH in your XLSX files, then this formatting will be imported into Realm Works.

v1.38
Provide the option to read XLSX files as an alternative to CSV files.

v1.37
You can now save your current project, and load it back later.
The names of the CSV and RWEXPORT files are saved as part of the project.
Your mappings within the categories is also saved (although if no field is mapped to the topic's name, then that category won't get saved).

v1.36
Fixes a bug where the import would fail if the category has child sub-categories (e.g. the "Location" category has "Settlement" and "Adventure Area" sub-categories; so any attempt to create "Location" topics would fail).

v1.35
Fixes a bug where the import would fail if the category definition has sub-sections within sections.

v1.34
It is now possible to create multiple content snippets in the main body of a section.
Individual snippets can't be duplicated, but the general "contents" snippet of the section can now generate multiple snippets.

v1.33
Allow a section to be marked as starting in the "collapsed" state.
Allow a CSV column to be used as the name of a section within a topic.
Allow a set of CSV columns to be used to create multiple sections within a topic.
See http://forums.wolflair.com/showpost....&postcount=295 for more details on how to use this functionality.

v1.32
A small update to add a menu option to choose what to use for the field separator in the CSV file. You can choose from comma, semi-colon, or the windows list separator.

v1.31
The default separator in CSV files is now read from the Windows Regional Settings (the "list separator").
If you don't want to use your language's default list separator, it can be changed by opening windows Settings (windows 10) and finding the "Region & Language" section. Click on the "Administrative language settings" under "related settings". Go to the "Formats" tab, and click "Additional Settings...". Here you will see "List separator:" where you can change the default separator.

v1.30
You can now define more than one category to be exported from the tool (e.g. spells and items).
Each category definition can have its own set of parents.
The new "key" icon in the title of the category allows you to specify which rows in the CSV table should be displayed in this category (based on a column having a specific "key" value in it).
Note that separate parents are created for each category, so for example it isn't possible to create some spells and some items and have them appear under the same parent topic/article in Realm Works.

v1.29
The same category can now be chosen for each level of parents.
In theory (untested), you could even select the same category that you've chosen for the main topics.
This removes the previous restriction that all parents had to be using a different category.

v1.28
For each snippet, it is now also possible to set the Veracity of that snippet (True, Partially True, Untrue).
The selection of snippet style has been merged into the new Options menu.

v1.27
Adds support for numeric snippets.

v1.26
The "Import Name" text box has been replaced with a button to open a File Details dialog in which the import name, as well as other details, can be entered.

v1.25
Individual "<" are now treated properly so that they don't cause an error on import.
However, any field which starts with "<" and finishes with ">" is left unchanged, since it is assumed to be HTML that you wish to have imported directly.

v1.24
Don't include "overlay" elements in the generated RWexport file.
(There is no v1.23)

v1.22
You can now enter the name of the RW import before generating the output file.
The installer has been changed from QTIF to NSIS.

v1.21
You can now set the Snippet Style on the snippets that you're going to be importing.

v1.20
It is now possible to specify a hierarchy of parents. This means you could have spells organised by sub-school under a more general school topic; or you could have all your traveller planets stored per sub-sector and then per sector.
At the moment, a different category must be chosen for each parent, as well as for the main topic display.
Icons have been added to buttons to make their actions clearer.

v1.19
When defining a parent category, you now have the choice of being able to have more than one parent in your output file.
By dragging one of the CSV columns into the parent's name field, then there will be one parent created for each unique string in that column of the CSV file. The rows of the CSV table which contain that same value will then appear under that parent.
(For example, you could copy the "school" column of a spell CSV into the parent box, and then the RWexport file will define one parent topic/article for each spell school, and will have the spells for that school as children.)

v1.18
(thread comment #166)
True Names and Other Names can now be imported from a CSV file.
You'll find a new "+Name" button at the end of the topic title's row.
Pressing the button once will add a line where you can define the True Name for the topic.
Pressing it again will add additional rows for defining Other Names for the topic.
As usual, the relevant CSV column should be dragged to define the True/Other name. If the CSV field is empty on a particular row, then no alias will be added for that topic.
Several Other Names can be added and the various Realm Works name flags can be set differently on each Other Name (revealed, Case Matching, Auto Accept, Shown in Nav, Priority). So depending on the contents of your CSV file, you could have one CSV column for "revealed" Other Names, and a different CSV column for "unrevealed + priority avoid" Other Names.

v1.17
When reading images from a web URL, follow web redirects which are issued by the web server.
A progress bar is also displayed when performing the GENERATE, so that you can see how quickly (or slowly) it is working.

v1.16
When loading an image (see V1.11) it is now possible to specify a web URL instead of just a local file name. Note that the URL must point directly to the image, not to a web page which contains an image somewhere inside it.
The GENERATE button will now be disabled when the file generation process is in progress. This helps you know when the generation is complete (accessing files using a web URL can be very slow).

v1.14
When importing blocks of text, convert URLs so that they appear in Realm Works as clickable links.

v1.13
A mistake with github meant that the wrong code was tagged, so ignore this version (typical for number 13 anyway!)

v1.12
This version fixes a bug in the CSV reader where a " at the end of a line was not being handled properly.

v1.11
Need to load HeroLab portfolio files for all your favourite NPCs?
This version extends the file loading to allow loading data for the following snippet types: Portfolio, Statblock, Rich_Text, PDF, Audio, Video, HTML, Foreign.

v1.10
Load images into the RWexport file...
A column is chosen in the CSV file to use for the Picture or Smart-Image snippet.
The text in the column is the name of the file to load as part of the GENERATE process.
The file can either be a full or relative path to the file; if no path is given then the file must be in the same directory as the CSV file.
If the file does not exist, then a warning will be generated during the GENERATE process. The resulting file can still be loaded into Realm Works, in which case the snippet will show the name of the file even though the image itself will remain blank.
(Also, upgraded from Qt 5.8 to Qt 5.9.2 - this should have no impact on users.)

v1.9
Multi-line quoted cells are now supported in the CSV file.

Also, when you one of these fields then a blank line is created as a paragraph break when the data is being put into a Realm Works field which is either a Multi_Line or Label_Text field (the two fields which allow more than one line of text).

v1.8
This release fixes an issue where the RWEXPORT file would contain one or more blank topics if the original Realm Works structure file contains a category defined as a child of another category. (E.g. Location in the basic Pathfinder category structure).

v1.7
A small update which ignores blank lines in the CSV file. This prevents an empty topic being created in the RWEXPORT file which caused Realm Works to reject the import.

v1.6
A minor update which remembers the last path you used for each of the three file dialogs (each is remembered separately).
A tidy-up of code means that the text in the structure section is never mixed up with text in the topics section.
Multi-line text is now possible.
The lovely (R) has been added to all uses of the term Realm Works(R).

v1.5
This release allows you to type in some free text into a snippet, or choose a tag for a choice field, instead of having CSV data appear in the field.
You can't mix CSV data and free text in the same snippet (yet) though.
It also fixes a bug where sometimes a partition in the structure part of the XML file would wrongly get text attached to it.

v1.4
The entire topic, as well as the optional containing topic, can now be marked as REVEALED to players (using the circular button to the left of topic name).
This updates the existing functionality where individual snippets could be marked as revealed.

v1.3.1
A minor update to add validation to the new optional parent. You must enter a name, and you must choose a different category for the parent to the one chosen for the CSV data.

v1.3
This release see the addition of an option to allow a parent containing topic/article to be specified for all the topics which are to be created from the CSV file.

v1.2
The installer now adds the application to the Start Menu.
Fixed a bug with the layout of elements when there was a blank snippet in the template.

v1.1
Blank cells in the CSV file don't produce unnecessary elements in the output file.
Ensure that the annotation child appears before the tag_assign child in relevant elements.
Ensure the name of the topic/article is assigned one of the CSV fields.
Remove the "GM only" button from each snippet, since it doesn't do anything.

v1.0.0
The top drop-down menu does nothing.


TO DO
Improve the CSV value => Domain mapping so that partial matches are possible (comma separated lists of multiple tags is already supported).

Won't Do
The following items should really be handled by setting up the required category definitions within Realm Works before exporting the structure:
Provide support for Labeled_Text fields.
Allow the addition of additional text (and other) snippets to an imported category.
Allow the addition of new sections and subsections to an imported category.

The following idea can be achieved by re-organising the contents of the CSV file:
Allow predefined text and CSV field names to be mixed together within certain text snippet types.

Bidmaron April 15th, 2017 04:50 AM

This looks like a very great addition to our toolkits. Can you elaborate on the instruction regarding how to do domain selection?
Any good CSv sources out there?

Farling April 15th, 2017 05:24 AM

Quote:

Originally Posted by Bidmaron (Post 248084)
This looks like a very great addition to our toolkits. Can you elaborate on the instruction regarding how to do domain selection?
Any good CSv sources out there?

When you have a snippet type which requires a tag, then my tool will take the text from the CSV column and try to find a matching value for the tag.

From the list of CSV fields in the left panel, drag the name of the field across onto the box that contains the black triangle (to the right of the field name).

For example, with the Pathfinder game, if you have a CSV for spells, then the column for the "spell subschool" might contain the text "scrying". My import tool would take the text "scrying" and find the correct tag to put into RealmWorks for this text.

Absintheminded78 April 16th, 2017 04:04 AM

I must be doing something wrong somewhere along the line as each time I go to import after I've generated the export file I get the message that the category does not exist, any advice? Tried with feats and also magic items.

Farling April 16th, 2017 08:44 AM

Quote:

Originally Posted by Absintheminded78 (Post 248147)
I must be doing something wrong somewhere along the line as each time I go to import after I've generated the export file I get the message that the category does not exist, any advice? Tried with feats and also magic items.

Hi, sorry to hear you're having problems.

Is the message about the category not existing coming from my tool or from Realm Works?

Which fields from a CSV are you trying to copy? It is best not to use fields which have lots of HTML mark-up in them since this will likely conflict with the HTML being added by the conversion tool, and also with what Realm Works might be expecting.

Absintheminded78 April 16th, 2017 01:08 PM

Quote:

Originally Posted by Farling (Post 248150)
Hi, sorry to hear you're having problems.

Is the message about the category not existing coming from my tool or from Realm Works?

Which fields from a CSV are you trying to copy? It is best not to use fields which have lots of HTML mark-up in them since this will likely conflict with the HTML being added by the conversion tool, and also with what Realm Works might be expecting.

From Realm Works, when I go to import the file generated by your tool it crashes somewhere between 3-5% and the error report states it's because category does not exist.
I'm just using basic text fields from the CSV files from http://www.pathfindercommunity.net/home/databases/, I have already managed to get the spells imported with EightBitz's script so I attempted to bring in the magic items and feats, both gave that error.

Farling April 16th, 2017 02:20 PM

Quote:

Originally Posted by Absintheminded78 (Post 248156)
From Realm Works, when I go to import the file generated by your tool it crashes somewhere between 3-5% and the error report states it's because category does not exist.
I'm just using basic text fields from the CSV files from http://www.pathfindercommunity.net/home/databases/, I have already managed to get the spells imported with EightBitz's script so I attempted to bring in the magic items and feats, both gave that error.

Is it possible for you to provide the RWEXPORT file generated by my tool so that I can have a look at its output? I'd like to see what is happening with the tool.

Absintheminded78 April 16th, 2017 02:28 PM

Quote:

Originally Posted by Farling (Post 248158)
Is it possible for you to provide the RWEXPORT file generated by my tool so that I can have a look at its output? I'd like to see what is happening with the tool.

Sure, thanks for having a look :D

https://drive.google.com/drive/folde...Vk?usp=sharing

Farling April 17th, 2017 04:12 AM

1 Attachment(s)
Absinthe,

I'm trying to import your "feat text.rwexport" file at the moment into a newly created "pathfinder" realm. My attempt is currently at 23% and still going. What type of realm are you trying to import into?

The first improvement to the import would be to drag the feat type across to the "selector" in the feat template. The selector is the box containing the triangle. (It will detect multiple tags for the same field in the CSV if they are separated by commas within the CSV field.) You can have the same CSV column appear more than once in the template. See the attached picture

Absintheminded78 April 17th, 2017 11:55 AM

Hi Farling,

I'm importing into a new Pathfinder realm. My steps were:
  • Create a new Pathfinder Realm
  • Add copies of relevant categories (Feats, Magic Items)
  • Export that realm's structure
  • Create the .rwexport file with your tool
  • Import that back into the realm I created
How do I have more than one column appear in a field? e.g. with Feats you have Benefit but also Normal and Special. Is it possible to add more than one column to a field? Or is that part of the initial copying of the category? To add any missing fields before exporting the structure? (I think I know the answer to this already now after thinking about it :o)
Also, I couldn't see a picture in your post sorry.

Thanks for your help, I'll do another one using the selector now that I know what I'm doing haha.

Farling April 17th, 2017 02:16 PM

It is curious because I followed your steps above; except that I imported into a different realm. Maybe it is the re-import into the same realm that is the problem, I will try it out.

At the moment there is only one column per field.

Hopefully you can see the picture as an attachment now.

It is on the TODO to allow the creation of labelled_Text snippets, and also to intermix columns with entered text.

I'm trying to produce an updated version that fixes a slight error when you have both a snippet tag and an annotation in the field. I'm struggling with the installer (the code is already fixed). So in v1.0.0 either set up the tag or the annotation, but not both.

Absintheminded78 April 17th, 2017 03:10 PM

Thanks Farling, I can now see the picture too. I'll try again and create a new .rwexport file using the tags in the selector.

I'm starting to think that maybe there is an issue with my Realm Works as I had altered EightBitz's spell import script to do the magic items and received the same error about the category not existing but when I checked it in XML Notepad and compared it to my spells import file the structure and content were almost identical (except of course relevant category, tag and facet ID's, it was set to the correct one for Magic Items - category_4).
I'll do some more testing, did you manage to get my file to import ok?

Farling April 17th, 2017 03:26 PM

Importing into the same realm won't work at the moment. This is due to me using "Tag_1" as the tag for the new import, but Realm Works has already used this for the export of the structure which was performed.

I tried using a tag id of "MyImportTag" but this caused Realm Works to crash (reported), after reporting about an error during import.

I don't think it's anything that I can fix at my end for the moment.

ErinRigh April 17th, 2017 05:11 PM

Does this mean we are limited to one import with your tool?

davidp April 17th, 2017 05:25 PM

Quote:

Originally Posted by Farling (Post 248222)
Importing into the same realm won't work at the moment. This is due to me using "Tag_1" as the tag for the new import, but Realm Works has already used this for the export of the structure which was performed.

I tried using a tag id of "MyImportTag" but this caused Realm Works to crash (reported), after reporting about an error during import.

I don't think it's anything that I can fix at my end for the moment.

If you are referring to the tag_id values, they are only used during the import and as long as you don't duplicate them within the import file for different <tag> elements, it should work fine. Even if you are referring to the "name" on the tag in the import file, it should still work fine, although it would be confusing.

As far as the crash, if you haven't done so yet, please open a support ticket, also, so we can track it and contact you directly with more questions.

Farling April 17th, 2017 11:00 PM

Quote:

Originally Posted by davidp (Post 248234)
As far as the crash, if you haven't done so yet, please open a support ticket, also, so we can track it and contact you directly with more questions.

Hi David,
I have now raised a support ticket (two actually, one from my old email and one from my new email - Case 119499 and Case 119500).
It is curious that the import works into a new realm just not the original realm.

It all seems related to handling of the import_tag_id.

Farling April 17th, 2017 11:06 PM

Quote:

Originally Posted by ErinRigh (Post 248232)
Does this mean we are limited to one import with your tool?

Hello ErinRigh,

No, not at all, the import tool will create one file per category, so you would create one file for each of your different topic/article categories.

Each of these files can then be imported into the same realm.

I have a further tweak to make to the program where you can specify the name of the import (or have it automatically generated), rather than it always being called "Test Import".

The only limitation at the moment, which I've now identified in the first post, is that you can't import into the same realm from which you generated the structure-only export.

ErinRigh April 18th, 2017 06:06 AM

Hey farling, I got v1.0 to work and imported a file (YaY) but I tried to import another file made with v1.1 and got an error

The following errors were encountered during import:

Encountered validation error for the import XML: The element 'contents' in namespace 'urn:lonewolfdevel.com:realm-works-export' has invalid child element 'section' in namespace 'urn:lonewolfdevel.com:realm-works-export'. List of possible elements expected: 'topic, plot_group, view' in namespace 'urn:lonewolfdevel.com:realm-works-export'.

The following considerations arose during import...

Adding tag PFSpells (Structure Only) to domain Import


Here is a dropbox link with my files if you want to check through them;

https://www.dropbox.com/s/12ax8a7h2z...ells1.zip?dl=0

craghammer April 18th, 2017 11:07 AM

Getting a lot of errors with this tool, but did manage to get it working on occasion. I do see that when I generate a rwexport file, sometimes the XML structure get's jumbled (</export> tag appears after a <topic> tag). And then I get the same as ErinRigh.
Keep up the good work, this tool IS awesome, if it works :)

Farling April 18th, 2017 02:54 PM

Quote:

Originally Posted by craghammer (Post 248270)
Getting a lot of errors with this tool, but did manage to get it working on occasion. I do see that when I generate a rwexport file, sometimes the XML structure get's jumbled (</export> tag appears after a <topic> tag). And then I get the same as ErinRigh.
Keep up the good work, this tool IS awesome, if it works :)

Hi Craghammer, is it possible for you to send me a copy of the rwexport file and/or some lines from the original CSV and the rwstructure file?

I want to track down how the tags are getting out of step, since I didn't think it would be possible.

Farling April 18th, 2017 03:40 PM

V1.2 has just been posted which fixes a problem with the generation of the closing tags when a snippet was left blank.

The installer also adds the application to the Start Menu.

The Q.A. department has been sacked! Normal service will resume shortly.

Absintheminded78 April 18th, 2017 04:16 PM

Quote:

Originally Posted by Farling (Post 248278)
The Q.A. department has been sacked! Normal service will resume shortly.

Haha :D

I'll give v1.2 a go. Interestingly I managed to get a Feats export to work properly. I created a new Structure Realm and a new Test Realm and generated a feats .rwexport file that imported with no issues. I then tried with the magic items csv file and encountered the same issue as craghammer where the xml tags are out of order.
As I said I'll attempt with v1.2 and see how it goes.

Loving the tool btw

ErinRigh April 18th, 2017 05:04 PM

1.2 seems to have fixed my problem, thanks Farling, keep up the great work, mate

craghammer April 19th, 2017 12:30 AM

Quote:

Originally Posted by Farling (Post 248276)
Hi Craghammer, is it possible for you to send me a copy of the rwexport file and/or some lines from the original CSV and the rwstructure file?

I want to track down how the tags are getting out of step, since I didn't think it would be possible.

Hi Farling, looking into the CSV file, I've noticed it has a lot of special characters in the text, and that is probably messing up the import. Working through it and it looks better and better when importing.

Is it possible/easy/hackable, to add all imported items to a containing article? In my case, importing a few hundred items/articles, it would help having them in a container.

Farling April 19th, 2017 11:19 AM

Quote:

Originally Posted by craghammer (Post 248296)
Is it possible/easy/hackable, to add all imported items to a containing article? In my case, importing a few hundred items/articles, it would help having them in a container.

I could add it to the TODO list.

I already know the format of the XML file that would do this.

The difficult bit is designing how to present this to a non-technical user in a friendly manner. :-)

Farling April 19th, 2017 12:30 PM

Craghammer, your wish is my command, have a look at v1.3.1

Less than 90 minutes, thanks to the wonders of Qt :-)

With some proper validation entered so that people don't try to do silly things :-)

ErinRigh April 19th, 2017 01:58 PM

Quote:

Originally Posted by Farling (Post 248326)
Craghammer, your wish is my command, have a look at v1.3.1

Less than 90 minutes, thanks to the wonders of Qt :-)

With some proper validation entered so that people don't try to do silly things :-)

This is BRILLIANT, save me HOURS of work, you are the MAN

craghammer April 19th, 2017 09:53 PM

Farling, you rock! :D

Absintheminded78 April 19th, 2017 11:30 PM

I don't know what the hell is wrong with my one :mad:
Every time I do one now I get an error after about 10% import that the category doesn't exist. (Realm Works crashes at the error)
I've tried multiple combos of realm structure and import realm but still no luck. e.g. created two new realms - one for structure, one for import - tested - error.
Tried that multiple times with different export files, even doing a smaller amount of topics (just general feats) but always gives me grief :mad:
Anyone able to share a file they've created to see if I get the same error with your file?
Thanks

Farling April 19th, 2017 11:46 PM

Quote:

Originally Posted by Absintheminded78 (Post 248372)
I don't know what the hell is wrong with my one :mad:
Every time I do one now I get an error after about 10% import that the category doesn't exist. (Realm Works crashes at the error)
I've tried multiple combos of realm structure and import realm but still no luck. e.g. created two new realms - one for structure, one for import - tested - error.
Tried that multiple times with different export files, even doing a smaller amount of topics (just general feats) but always gives me grief :mad:
Anyone able to share a file they've created to see if I get the same error with your file?
Thanks

Hi Absinthe,
Is there any chance of sharing one of the files generated by the tool so that I can check the file to see if my tool is putting some rubbish into the file?

Generally there are two limitations at the moment:
- you have to create a copy of any predefined categories which you want to import data into.
- you have to export a structure from a different realm from the one into which you are importing the new data. (I created a blank realm merely to copy some predefined categories and then generate the structure-only export file.)

Absintheminded78 April 20th, 2017 12:01 AM

Quote:

Originally Posted by Farling (Post 248373)
Hi Absinthe,
Is there any chance of sharing one of the files generated by the tool so that I can check the file to see if my tool is putting some rubbish into the file?

Generally there are two limitations at the moment:
- you have to create a copy of any predefined categories which you want to import data into.
- you have to export a structure from a different realm from the one into which you are importing the new data. (I created a blank realm merely to copy some predefined categories and then generate the structure-only export file.)

Hey Farling,

Yeah, that's what I've been doing - New realm --> create duplicate of topics (in this case creature, feat and magic items) --> export structure --> create .rwexport file using your tool --> import into new blank realm --> receive error haha.

I've uploaded the structure file and also my latest attempt (Feats General Test) to my drive.
https://drive.google.com/drive/folde...HBSeThoaE1oaVk

Thanks for all your effort.

craghammer April 20th, 2017 02:38 AM

Farling and Absinthe;
A quick look at the rwexport xml gives me the impression that there is something afoul in the structure. There are some text fields that seem to be outside the xml tags, if the XML does not support inline linebreaks.
Also I see that topics are missing the public_name attribute value, this might cause a crash on import. Absinthe: Are you adding a value to the Name property in the CSV2RW tool? And verifying that the CVS columns contains values/text?

Absintheminded78 April 20th, 2017 02:56 AM

If that was in the Magic Items one, yeah, I noticed that too but the latest file is the Feats General one which definitely has the relevant fields filled.

I removed the previous uploads to reduce confusion.

ErinRigh April 20th, 2017 03:01 AM

1 Attachment(s)
Absinthe

Try this file

Absintheminded78 April 20th, 2017 03:05 AM

1 Attachment(s)
Quote:

Originally Posted by ErinRigh (Post 248384)
Absinthe

Try this file

OK, so the issue lies elsewhere it seems, got the error with your file too :mad:

ErinRigh April 20th, 2017 03:07 AM

Out of curiosity, are you using the latest build? I know that's a stupid question, but I gotta ask

Absintheminded78 April 20th, 2017 03:08 AM

Of Realm Works or the CSV tool? Yes to both.

ErinRigh April 20th, 2017 03:16 AM

Well attempting to import your test file gave the following;

The following errors were encountered during import:

Reference is still being edited


Then crash to desktop

ErinRigh April 20th, 2017 03:19 AM

just a thought but did you have the CSV open in a spreadsheet editor when you created the export?


All times are GMT -8. The time now is 01:47 PM.

Powered by vBulletin® - Copyright ©2000 - 2024, vBulletin Solutions, Inc.
wolflair.com copyright ©1998-2016 Lone Wolf Development, Inc. View our Privacy Policy here.