Limit on hex character display in label?

[TCArknight]

Hi all!

I am trying to get a special character in the middle of a word to display correctly in a description label, but it seems to always be displaying a diamond with a question mark instead.

description="You studied strategy, horsemanship, and soldiering at one of Th<hex value 233>ah's many military academies. When you make a Risk using Athletics, Warfare, or Ride, all of your dice gain +1 to their value."

It should display as: Théah's

it seems like it happens with any ASCII value over 160 with any number between 127 and 159 producing an error.

Am I missing something?
Thanks!
TC

 

[jfrazierjr]

Using the wrong decimal codepoint...127 to 159 are not valid in iOS 8859-1 x.l documents. Nor are any higher then 254

 

[TCArknight]

Thats the issue though.... between 160 and 254, the display is the diamond with a question mark in it.

Do I need to change some setting in the dataset or something else I can do to get these other characters to show up?

 

[Mathias]

Here's the wiki page on the character encoding set Hero Lab uses: http://hlkitwiki.wolflair.com/index.php5?title=XML_Character_Encoding_Set

 

[TCArknight]

Ah, that explains it. By default, the character I need isn't available.

I see mention of unofficial use of Windows-1252 set. Is it possible to use it instead by changing the encoding of the specific files containing it?

According to the wiki page though, ISO-8859-1 is considered the default. Looking at https://www.w3schools.com/charsets/ref_html_8859.asp for reference, it seems it should be able to support characters over 160. That's not the case though?

 

[jfrazierjr]

Here's the wiki page on the character encoding set Hero Lab uses: http://hlkitwiki.wolflair.com/index.php5?title=XML_Character_Encoding_Set

That seems to be SERIOUSLY out of date and thus incorrect. I checked the data files created by the editor for my own stuff,

<?xml version="1.0" encoding="UTF-8"?>

I then checked a few source files in savage/source and it's the same.

Then, for SnG's I took an existing item I had added as a TEST from my datafile, and changed the name to include some utf-8 characters. It shows up perfectly both in the application as well as in the print preview pdf.

  <thing
    id="miBALDuelB"
    name="  ß ¼ ɚ ˥ ط Baldimar Duelist’s Blade"
    compset="miMelee"
    description="This ornately worked rapier is very popular with duelists. It grants +2 to Fighting, +1 damage, and increases the wielder’s Parry by +2."
    summary="">
    <fieldval field="miAttack" value="2"/>
    <fieldval field="miDamage" value="1"/>
    <fieldval field="wpParry" value="1"/>
    <fieldval field="grCost" value="13150"/>
    <fieldval field="gearWeight" value="3"/>
    <usesource source="Fantasy"/>
    <tag group="WeaponDie" tag="2"/>
    <containerreq phase="Initialize" priority="2000"><![CDATA[
      source.Fantasy]]></containerreq>
    <child entity="fcMagItemP">
      <bootstrap thing="wpRapier"></bootstrap>
      <bootstrap thing="mieBlock2"></bootstrap>
      </child>
    </thing>

@TCArknight, NO, you do NOT want to utilize that crappy windows-1252 characterset. You will cause nothing but issues dealing with that.


So... the first thing I need to know, what editor are you using to modify your .user file? vanilla Notepad, Notepad++, TextPad, etc?

Second, what is the top line of your file as I noted above? What does the "encoding" attribute say?

For xml ,there are two critical parts of encoding, 1) the encoding declaration in the xml instrcutions(defaults to utf-8 if missing), and 2) the actual encoding on the FILE when you save. The latter is something most people miss unless they actually have good editor which makes it prominent as windows stupidly saves the files as windows-1252 in some very specific cases.

If using Notepad++, simply go to the Encoding menu and make sure it's set to utf-8 and save again AS WELL as change the xml prefix at the start of the file to utf-8 exactly like my example above. Also, when and HOW as the file created? ie, did you open a plain text file and start adding content? Did the editor create a stub file for you which you then edited? Did you copy from other sources and then throw away and rebuild? If this was copied from the editor made one, I assume it was done a long time ago or you might be using a very old baseline file(depending upon when they changed to support utf-8).

Let me know how that works.

 

[TCArknight]

Thanks!

I was using a prog called metapad, but it looks like I need to use Notepad++ to make sure I convert to UTF-8 for the save. Once I opened the file and did that, I was able to cut and paste Théah directly and it displayed properly everywhere.

The file had UTF-8 up at the encoding= bit, but it looks like Metapad was just saving as ASCII.

I do notice that even if I save it as UTF-8, if I then replace the ' é ' character with the 233 code (#233 after an ampersand, it throws it back to the black diamond. Is there an issue with the parser converting the actual code but not if it's a cut and paste?

 

[jfrazierjr]

Thanks!

I was using a prog called metapad, but it looks like I need to use Notepad++ to make sure I convert to UTF-8 for the save. Once I opened the file and did that, I was able to cut and paste Théah directly and it displayed properly everywhere.

The file had UTF-8 up at the encoding= bit, but it looks like Metapad was just saving as ASCII.

I do notice that even if I save it as UTF-8, if I then replace the ' é ' character with the 233 code (#233 after an ampersand, it throws it back to the black diamond. Is there an issue with the parser converting the actual code but not if it's a cut and paste?

the problem with #233; is that is typically for html notation. I am not sure one way or the other if html notation is supported, but I really rather doubt it. I say this since the tags for bold, italics, etc are BBCode tags(or their equivilent) as opposed to real html tags for the same.


There are various ways to embed characters within xml content, but you should typically stick with the straight up "typing" it in(ie, where it actually looks like the character you want to see).

If you REALLY want to check this stuff out, try the following, some or all of them may not work:

é
é

A quick test of Save Custom Output to html seems to output the correct declaration, but oddly enough, the magic items don't seem to output, so I can't verify. I assume this is just because the stylesheet does not know about the Fantasy Supplement stuff and have an output for it? Output Hero Stat Block DOES show the item in this case as well as the encoding declaration, but one issue someone might run into is if they COPY the html and put that into new file(as opposed to saving through the interface), they will likely NOT set the encoding correctly in their file since Notepad typically only does ANSI unless you explicitly override that to use utf-8 and then that person would be back where they started from.

encoding is a giant PITA.

EDIT: grr.. the forum is eating stuff. That FIRST example is the same as yours, but with a preceding & such as & #233; but without the extra space there.