XML Transformation Script (PowerShell)

[EightBitz]

Originally Posted by davidp

The <contents> of a <snippet> should all be HTML and that would be the only place that the lists, tables, etc. would be present. You should be able to take the <contents> and just treat it as HTML and be done with it. Any more and you are making life hard on yourself.

The only slight gotcha is that we use encoding of the contents so that it isn't interpreted as XML tags, but that should be straight-forward to deal with.

Yep. For the HTML version, I'll be taking advantage of that. Right now, I just wanted a plain text version. No HTML or anything.

[EightBitz]

OK, I'm not going to keep cluttering up the forum, but here's a much better version of the plain text export. No HTML or anything yet, just plain text.

I'll set up a DropBox or something soon.

I sorted out a few bugs and oversights, and added some functionality:

Version 0.5a
Added a -Prefix switch to optionally include a topic's prefix.
Added a -Suffix switch to optionally include a topic's suffix.
Added a -Sort option to sort topics by:
1 = Name
2 = Prefix, Name **Default**
3 = Category, Name
4 = Category, Prefix, Name

Choosing options 2 or 4 will sort by prefix, regardless of whether or not
the -Prefix switch is specified. Likewise, choosing options 1 or 3 will
sort by name, regardless of whether or not the -Prefix switch is specified.

Added parentage for topics.
Made the topic suffix parenthetical so it's consistent with the display in RW.

Better (I hope) parsing of snippets, and support for more types.

Supported snippet types include:
Text (Including lists and tables, but NOT including any formatting such as bold, italic, highlights, etc.)
GM Directions
Labeled Text
Tags
Calendar Date
Calendar Date Range
Numeric Value
Tags (MultiDomain)

Unsupported snippet types include:
Picture (Simple)
Smart Image (Map)
Statblock
Hero Lab Portfolio
Foreign Object
Any snippet types listed under "Documents and Media"

It now responds to the Get-Help cmdlet.

Get-Help RWExport-To-Text.ps1

Have fun!

# RWExport-To-Text.ps1 (formerly Format-RWExport.ps1)
# by EightBitz
# Version 0.5a
# 1/16/2017, 10:10 AM CST
# 
# Copy this full length of code into a text file, and save it as Format-RWExport.ps1
#
# Open a Powershell window, and CD to where you saved it.
# For example, if you saved in C:\Users\EightBitz\Desktop, then in the Powershell window, you would type:
#    CD \Users\EightBitz\Desktop
# 
# Then run the powershell script, specifying the source file (the compact export you created)
#    and the new destination file you wish to create or overwrite. Example:
# .\Format-RWExport.ps1 -Source "C:\Users\EightBitz\desktop\Export test.rwoutput" -Destination C:\Users\EightBitz\desktop\TransformedExportTest.txt
# 
# 
# CSS categories:
#    <title>
#    <topic>
#    <section>
#    <snippet>
#
# Track parentage and levels of topics and sections
#
# Sort topics
# 

<# 
.SYNOPSIS
RWExport-To-Text.ps1 transforms a RealmWorks export file into a plain text file.

By: EightBitz 
Date: 1/16/2017
Version 0.5a

.DESCRIPTION 
RWExport-To-Text.ps1 loads the XML file exported from Realm Works and transforms it into plain text so it can be printed or imported/pasted into other programs. The export from Realm Works must done with the Compact Output option.
.PARAMETER Source 
Enter the full path and filename for the source file (the Realm Works export file).
.PARAMETER Destination 
Enter the full path and filename for the destination file (the plain text output).
.PARAMETER Sort
Choose your preferred sort order for exported topics.
   1 = Name
   2 = Prefix, Name **Default**
   3 = Category, Name
   4 = Category, Prefix, Name
.PARAMETER Prefix
Include this parameter to display the prefix for each topic.
.PARAMETER Suffix
Include this parameter to display the suffix for each topic.
.EXAMPLE 
RWExport-To-Text.ps1 -Source MyExport.rwoutput -Destination MyPlainTextFile.txt 
.EXAMPLE 
RWExport-To-Text.ps1 -Source MyExport.rwoutput -Destination MyPlainTextFile.txt -Sort 1
.EXAMPLE
RWExport-To-Text.ps1 -Source MyExport.rwoutput -Destination MyPlainTextFile.txt -Prefix -Suffix
.NOTES
The specified sort order will occur regardless of whether or not prefixes are included.
If you sort by prefix, but do not include the -Prefix parameter, your topics will still be sorted by prefix, even though the prefix won't be displayed.
Likewise, if you sort by Name, but do include the -Prefix parameter, your topics will still be sorted by name, even though the prefix will be displayed.

It appears that topics will always be sorted under their containers. In other words, topics that are not in containers will be sorted relative to each other. Contained topics will be sorted relative to their peers within that container.

Everything outputs in plain text. I do want to work on an HTML/CSS version, but first I had to get all of this done. This is my foundation for building a version with formatted output.
#> 

param (
    # Source file path and name.
    [Parameter(Mandatory,Position=1)] 
    [string]$Source,

    # Destination file path and name.
    [Parameter(Mandatory,Position=2)] 
    [string]$Destination,

    # Sort topics by:
    #    1 = Name
    #    2 = Prefix, Name **Default**
    #    3 = Category, Name
    #    4 = Category, Prefix, Name
    #
    # Choosing options 2 or 4 will sort by prefix, regardless of whether or not
    #    the -Prefix switch is specified. Likewise, choosing options 1 or 3 will
    #    sort by name, regardless of whether or not the -Prefix switch is specified.
    [Parameter()] 
    [int]$Sort = 2,

    # Include prefix in topic name ($true or $false)
    [switch]$Prefix,

    # Include suffix in topic name ($true or $false)
    [switch]$Suffix

) # param

Function ParseTopic($PassedTopic,$Outputfile,$Sort,$Prefix,$Suffix,$Parent) {
   $TopicName = $PassedTopic.public_name
   if ($Prefix -and $PassedTopic.Prefix) {$TopicName = $PassedTopic.Prefix + " - " + $TopicName}
   if ($Suffix -and $PassedTopic.Suffix) {$TopicName = $TopicName + " (" + $PassedTopic.Suffix + ")"}
   $TopicName = "Topic: $TopicName"
   $ParentName = "Parent Topic: $Parent"
   $CategoryName = "Category: " + $PassedTopic.category_name

   Add-Content -Path $outputfile -Value $TopicName
   Add-Content -Path $outputfile -Value $ParentName
   Add-Content -Path $outputfile -Value $CategoryName
   If ($PassedTopic.tag_assign) {ParseTags $PassedTopic $Outputfile}
   If ($PassedTopic.linkage) {ParseLinkage $PassedTopic $outputfile}
   Add-Content -Path $outputfile -Value ""

   foreach ($Section in $PassedTopic.section) {
      ParseSection $Section $Outputfile
   } # foreach ($Section in $PassedTopic.section)

   Switch ($Sort) {
      1 {$TopicList = $PassedTopic.topic | Sort-Object public_name}
      2 {$TopicList = $PassedTopic.topic | Sort-Object prefix,public_name}
      3 {$TopicList = $PassedTopic.topic | Sort-Object category_name,public_name}
      4 {$TopicList = $PassedTopic.topic | Sort-Object category_name,prefix,public_name}
      default {$TopicList = $PassedTopic.topic}
   }

   $Parent = $Parent + $PassedTopic.Public_Name + "/"
   foreach ($Topic in $TopicList) {
      ParseTopic $Topic $Outputfile $Sort $Prefix $Suffix $Parent
   } # foreach ($Topic in $PassedTopic.topic)
} # Function ParseTopic($PassedTopic)

Function ParseSection ($PassedSection,$Outputfile) {
   $SectionName = "Section: " + $PassedSection.name
   Add-Content -Path $outputfile -Value $SectionName
   Add-Content -Path $outputfile -Value ""
   foreach ($Snippet in $PassedSection.snippet) {
      ParseSnippet $Snippet $Outputfile
   } # foreach ($Snippet in $PassedSection.snippet)

   foreach ($Section in $PassedSection.section) {
      ParseSection $Section $Outputfile
   } # foreach ($Section in $PassedSection.section)
} # Function ParseSection ($PassedSection)

Function ParseSnippet ($PassedSnippet,$Outputfile) {
   switch ($PassedSnippet.type) { 
      "Multi_Line" {
        if ($PassedSnippet.purpose -eq "directions_only") {
           $Text = ParseSnippetText $PassedSnippet.gm_directions
           $Text = "GM Directions: " + $Text
           Add-Content -Path $outputfile -Value $Text
        } elseif (($PassedSnippet.contents.contains("<ul")) -or ($PassedSnippet.contents.contains("<ol")) -or ($PassedSnippet.contents.contains("<table"))) {
           $Text = ParseFormattedStuff $PassedSnippet.contents
           Add-Content -Path $outputfile -Value $Text
        } else {
           $Text = ParseSnippetText $PassedSnippet.contents
           Add-Content -Path $outputfile -Value $Text
        } # if ($PassedSnippet.purpose -eq "directions_only")
        Add-Content -Path $outputfile -Value ""
      } # "Multi_Line"
      
      "Labeled_Text" {
         $Text = ParseSnippetText $PassedSnippet.contents
         $LabeledText = $PassedSnippet.Label + ": $Text"
         Add-Content -Path $outputfile -Value $LabeledText
         Add-Content -Path $outputfile -Value ""
      } # "Labeled_Text"
       
      "Tag_Standard" {
         ParseTags $PassedSnippet $outputfile
         Add-Content -Path $outputfile -Value ""
      } # "Tag_Standard"

      "Numeric" {
         $annotation = ParseSnippetText $PassedSnippet.annotation
         $Numeric = $PassedSnippet.contents + ", $annotation"
         Add-Content -Path $outputfile -Value $Numeric
         Add-Content -Path $outputfile -Value ""
      } # "Numeric"
      
      "Tag_Multi_Domain" {
         ParseTags $PassedSnippet $outputfile
         Add-Content -Path $outputfile -Value ""
      } # "Tag_Multi_Domain"

      "Date_Game" {
         $DateText = ParseSnippetText $PassedSnippet.annotation
         $DateText = $PassedSnippet.game_date.display + ", " + $DateText
         Add-Content -Path $outputfile -Value $DateText
         Add-Content -Path $outputfile -Value ""
      } # Date_Game

      "Date_Range" {
         $DateRange = $PassedSnippet.date_range.display_start + " to " + $PassedSnippet.date_range.display_end
         $annotation = ParseSnippetText $PassedSnippet.annotation
         $DateText = "$DateRange, $annotation"
         Add-Content -Path $outputfile -Value $DateText
         Add-Content -Path $outputfile -Value ""
      } # Date_Range

      default {
         # Do nothing here.
      } # default
   } # switch ($PassedSnippet.type)
} # Function ParseSnippet

Function ParseSnippetText ($PassedText) {
   if ($PassedText -ne $null) {
      $gt = $PassedText.Indexof('>')
   } else {
      $gt = -1
   } # if ($PassedText -ne $null)
   $Text = $null
   While ($gt -ge 0) {
      # $gt = $PassedText.Indexof(">",$RWSnippet)
      $lt = $PassedText.Indexof("<",$gt)
      $length = $lt-$gt
      if ($length -gt 1) {$Text = $Text + $PassedText.substring($gt+1,$length-1)}
      if ($lt -lt $gt) {
         $gt = -1
      } else {
         $PassedText = $PassedText.substring($lt,$PassedText.length-$lt)
         $gt = $PassedText.Indexof('>')
      } # if ($lt -lt $gt)
   } # While ($gt -ge 0)
   if ($Text.contains("&nbsp;")) {$Text = $Text.replace("&nbsp;","`r`n`r`n")}
   Return $Text
} # Function ParseSnippetText

Function ParseTags ($PassedTags,$OuptutFile) {
   # This block of code assumes that tags of the same domain will always be grouped together.
   $domain = 0
   $tagdomains = $PassedTags.tag_assign.domain_name
   $tagdomain = $tagdomains[$domain]
   $tagline = ""

   foreach ($tag in $PassedTags.tag_assign) {
      if ($tag.domain_name -eq $tagdomain) {
         if ($tagline -eq "") {$tagline = $tagdomain + ": " + $tag.tag_name} else {$tagline = $tagline + ", " + $tag.tag_name}
         $domain++
      } else {
         $tagdomain = $tag.domain_name
         $tagline = $tagline + "`r`n" + $tagdomain + ": " + $tag.tag_name
         $domain++
      } # if ($tag.domain_name -eq $tagdomain)
   } # foreach ($tag in $PassedSnippet.tag_assign)

   Add-Content -Path $outputfile -Value $tagline
   if ($PassedTags.annotation) {
      $annotation = ParseSnippetText $PassedTags.annotation
      Add-Content -Path $outputfile -Value "Annotation: $annotation"
   } # foreach ($tag in $PassedTags.tag_assign)
} # Function ParseTags

Function ParseLinkage ($PassedTopic,$OutputFile) {
   $Linkage = $PassedTopic.linkage
   $LinkList = ""
   foreach ($link in $linkage) {
      if ($linklist -eq "") {$linklist = "Linkage: " + $link.target_name} else {$linklist = $linklist + ", " + $link.target_name}
   } # foreach ($link in $linkage)
   Add-Content -Path $OutputFile -Value $Linklist
   
} # Function ParseLinkage

Function ParseFormattedStuff ($PassedSnippet) {
   $TagLocations = GetTagLocations $PassedSnippet
   [array]$FullText = $null

   foreach ($tag in $TagLocations) {
      $tagtext = $PassedSnippet.substring($tag.start,$tag.end-$tag.start+1)
      Switch ($tag.tag) {
         "text" {
            $Text = ParseSnippetText $tagtext
            if ($Text) {
               if ($Text.contains("&#xd;")) {$Text = $Text.Replace("&#xd;"," ")}
               if ($Text.contains("&nbsp;")) {$Text = $Text.Replace("&nbsp;"," ")}
               $Text = $Text.Trim()
               Add-Content -Path $outputfile -Value $Text
            }
         } # "text"
            
         "ul" {
            $ordered = $false
            $Text = ParseList $tagtext $ordered
            Add-Content -Path $outputfile -Value $Text
         } # "ul"

         "ol" {
            $ordered = $true
            $Text = ParseList $tagtext $ordered
            Add-Content -Path $outputfile -Value $Text
         } # "ol"

         "table" {
            $table = ParseTable $tagtext
            $Text = $table | Out-String
            $Text = $Text.TrimEnd()
            Add-Content -Path $outputfile -Value $Text
         } # "table"

      } # Switch ($tag.tag)
   } # foreach ($tag in $TagLocations)
   Add-Content -Path $outputfile -Value ""
} # Function ParseFormattedStuff

Function GetTagLocations ($PassedSnippet) {
   $taglist = $null
   # Get the locations of all the start tags.
   $ul = $PassedSnippet.IndexOf("<ul")
   $ol = $PassedSnippet.IndexOf("<ol")
   $table = $PassedSnippet.IndexOf("<table")

   While (($ul -ge 0) -or ($ol -ge 0) -or ($table -ge 0)) {
      if ($ul -ge 0) {
         $properties = @{
            'Tag'='ul';
            'Start'=$ul;
            'End'=0
         } # $properties

         $TagLocation = New-Object –TypeName PSObject –Prop $properties
         [array]$Taglist = $Taglist + $TagLocation
      } # if ($ul -ge 0)

      if ($ol -ge 0) {
         $properties = @{
            'Tag'='ol';
            'Start'=$ol;
            'End'=0
         } # $properties

         $TagLocation = New-Object –TypeName PSObject –Prop $properties
         [array]$Taglist = $Taglist + $TagLocation
      } # if ($ol -ge 0)

      if ($table -ge 0) {
         $properties = @{
            'Tag'='table';
            'Start'=$table;
            'End'=0
         } # $properties

         $TagLocation = New-Object –TypeName PSObject –Prop $properties
         [array]$Taglist = $Taglist + $TagLocation
      } # if ($table -ge 0)

      $Taglist = $Taglist | Sort-Object Start
      $HighTag = $Taglist.Count-1
      $StartPosition = $Taglist[$HighTag].Start + 1

      $ul = $PassedSnippet.IndexOf("<ul",$StartPosition)
      $ol = $PassedSnippet.IndexOf("<ol",$StartPosition)
      $table = $PassedSnippet.IndexOf("<table",$StartPosition)
   } # While (($ul -ge 0) -or ($ol -ge 0) -or ($table -ge 0))

   # Get all the end tags.
   foreach ($tag in $Taglist) {
      $endtag = "</" + $tag.tag
      $endtag = $PassedSnippet.IndexOf($endtag,$tag.Start + 1)
      $Tag.end = $endtag+4
   } # foreach ($tag in $Taglist)

   # Fill in the gaps, if there are any.
   $firstitem = 0
   $firstpos = 0
   $lastitem = $Taglist.count -1
   for ($item=$firstitem; $item -le $lastitem; $item++) {
      if (($firstpos -lt $Taglist[$item].start)) {
         $endpos = $Taglist[$item].start - 1

         $properties = @{
            'Tag'='text';
            'Start'=$firstpos;
            'End'=$endpos
         } # $properties

         $TagLocation = New-Object –TypeName PSObject –Prop $properties
         [array]$gaplist = $gaplist + $TagLocation

      } # if (($firstpos -lt $Taglist[$item].start))

      if ($item -eq $lastitem) {
         $firstpos = $Taglist[$item].end + 1
         $endpos = $PassedSnippet.length - 1

         $properties = @{
            'Tag'='text';
            'Start'=$firstpos;
            'End'=$endpos
         } # $properties

         $TagLocation = New-Object –TypeName PSObject –Prop $properties
         [array]$gaplist = $gaplist + $TagLocation
      } # if ($item -eq $lastitem)
      $firstpos = $taglist[$item].end+1
   } # for ($item=$firstitem; $item -le $lastitem; $item++)

   $Taglist = $Taglist + $gaplist
   $Taglist = $Taglist | Sort-Object start
   Return $Taglist
} # GetTagLocations

Function ParseList ($PassedText,$Ordered) {
   if ($PassedText -ne $null) {
      $openli = $PassedText.Indexof('<li')
   } else {
      $openli = -1
   }
   [array]$List = $null
   $ItemNumber = 1
   While ($openli -ge 0) {
      $closeli = $PassedText.Indexof("</li",$openli)
      $length = $closeli-$openli
      if ($length -gt 1) {
         $LineItem = $PassedText.substring($openli+1,$length-1)
         $LineItem = ParseSnippetText $LineItem
         if ($Ordered) {$LineItem = $ItemNumber.ToString() + ". " + $LineItem} else {$LineItem = "* $LineItem"}
         [array]$List = [array]$List + $LineItem
      } # if ($length -gt 1)

      if ($closeli -lt $openli) {
         $openli = -1
      } else {
         $PassedText = $PassedText.substring($closeli,$PassedText.length-$closeli)
         $openli = $PassedText.Indexof('<li')
      } # if ($closeli -lt $openli)
      $ItemNumber++
   } # While ($openli -ge 0)
   Return $List
} # Funcion ParseList

Function ParseTable ($PassedText) {
   if ($PassedText -ne $null) {
      $opentr = $PassedText.Indexof('<tr')
   } else {
      $opentr = -1
   }
   [array]$RowList = $null
   While ($opentr -ge 0) {
      $closetr = $PassedText.Indexof("</tr",$opentr)
      $length = $closetr-$opentr
      if ($length -gt 1) {
         $Row = $PassedText.substring($opentr+1,$length-1)
         $Row = $Row.Replace("tr>&#xd;","")
         $Row = $Row.Trim()
         [array]$RowList = [array]$RowList + $Row
      }

      if ($closetr -lt $opentr) {
         $opentr = -1
      } else {
         $PassedText = $PassedText.substring($closetr,$PassedText.length-$closetr)
         $opentr = $PassedText.Indexof('<tr')
      }
   } # While ($opentr -ge 0)

   [array]$Table = $null
   foreach ($Row in $RowList) {
      if ($Row -ne $null) {
         $opentd = $Row.Indexof('<td')
      } else {
         $opentd = -1
      }
      $ItemList = new-object psobject
      $ItemNumber = 1
      While ($opentd -ge 0) {
         $closetd = $Row.Indexof("</td",$opentd)
         $length = $closetd-$opentd
         if ($length -gt 1) {
            $Item = $Row.substring($opentd+1,$length-1)
            $Item = ParseSnippetText $Item
            $ItemList | Add-Member -Name "Item $ItemNumber" -Type noteproperty -value $Item
         }

         if ($closetd -lt $opentd) {
            $opentd = -1
         } else {
            $Row = $Row.substring($closetd,$Row.length-$closetd)
            $opentd = $Row.Indexof('<td')
         }
         $ItemNumber++
      } # While ($opentd -ge 0)
      [array]$Table = [array]$Table + $ItemList
   } # foreach ($Row in $RowList)
   $Text = $Table | Format-Table -HideTableHeaders -AutoSize -Wrap
   Return $Text
} # Function ParseTable

# Main {
   $inputfile = $Source
   $outputfile = $Destination
   [xml]$Export = Get-Content -Path $inputfile
   $Title = "Title: " + $Export.Output.definition.details.name
   Set-Content -Path $outputfile -Value $Title
   Add-Content -Path $outputfile -Value ""
   $Contents = $Export.output.contents

   Switch ($Sort) {
      1 {$TopicList = $Contents.topic | Sort-Object public_name}
      2 {$TopicList = $Contents.topic | Sort-Object prefix,public_name}
      3 {$TopicList = $Contents.topic | Sort-Object category_name,public_name}
      4 {$TopicList = $Contents.topic | Sort-Object category_name,prefix,public_name}
      default {$TopicList = $Contents.topic}
   }

   $Parent = "/"
   foreach ($Topic in $TopicList) {
      ParseTopic $Topic $outputfile $Sort $Prefix $Suffix $Parent
   } # foreach ($Topic in $Contents.topic)
# } Main

[AEIOU]

You aren't cluttering. This is really interesting to see the project develop and it's a critical need for the community. Thank you for tackling this huge need.

And all the best in your job searching.

[davidp]

As far as a place to store code, github or gitlab could be good options. You can create free accounts on either and then check in code, getting versioning as part of it. git client is free. If someone wants to grab code but not install git, they can do so from the web interface. Anyone who wants to contribute to it can also submit changes that you could review and incorporate into your code base pretty easily.

[rob]

Originally Posted by Vargr

@EightBitz:

Notice how we have been civil and not turned into an angry mob complete with touches and pitchforks - yet...

Nah, just kidding. :-)

I'm onto you guys. This is just the calm before the storm, when the pitchforks and torches are properly gathered and the villagers summoned from their homes to be properly equipped and whipped into a frenzy in the days ahead. Right?

[kbs666]

I'm way too old school D&D for simple pitchforks. Guisarmes and Glaives for the peasants or there will be no mob.

[rob]

Originally Posted by EightBitz

OK, I'm not going to keep cluttering up the forum, but here's a much better version of the plain text export. No HTML or anything yet, just plain text.

This is NOT clutter. Please keep posting updates as you evolve this tool. There are going to be plenty of users who will find it valuable.

[ShadowChemosh]

My only suggestion would be to keep the "First" page updated with the latest script. As this thread grows it can be hard for someone new to find the "latest" script.

This forum lets you edit any post you have ever done no matter how long ago.

[Silveras]

Originally Posted by kbs666

I'm way too old school D&D for simple pitchforks. Guisarmes and Glaives for the peasants or there will be no mob.

Be careful of the Speed Factor on those...

[daplunk]

Put the script in a GitHub. It's worth it as you can track changes and people can submit bug reports etc. Update the front page with a link to the GitHub.