d20 Dataset Project....

[ShadowChemosh]

So I have a little more time so I would like to get back to this again. We have several things to do to get an easier to use data set for d20. Here is what I am thinking.

1. The first thing is to decide what books should be in the core 3.5 data set. Maybe easier to work with what books should be removed. I am thinking the dragonlance stuff and anything that is for Lawful_g's home games. I saw at least one file that was for houserules. Need to ask Lawful_g if you kept everything in that one file or did you spread things around? Anyone else see any other books that should be removed?
2. We need to tag the last of the books with a Source type so that the player can decided what books are allowed to be used per character. In example the Races of the Wild things are not Source marked. Risner you said you would be able to do this? Is it coming along or do you need help?
3. We need to change all the custom .dat files into .user files so that we will have an easier time creating the export file. I talked to Colen at HL and he said they maybe able to do a modification to the Export Tool that will auto-select all the .user files in a future update. So if we change the .dat files to .user it will become very easy to create the export files. Right now its a little confusing.
4. Will need a new folder on Cheifweasel's web server to hold the .xml file that will be used for the Secondary Update URL feature. Or where ever you want to store it. Assuming you want that on your server as you will also need to a have a single place to put the export files also. Otherwise I can actually host this if you want instead off of my web server.
5. Create at least one stable Export File(.hl) that imports all the .user files into someones d20 data folder. Preferably having it show up inside HL's download list so it can be auto-downloaded and installed.

Anyone have any other things to add to the list?

Also anyone have any ideas how to keep the .user files in sync with different people making changes to it? I am thinking it may need a central person to take all the changes and incorporate them into a single location. I know I have seen a few things that I would like to improve upon or make changes to and wouldn't want it to get over written with the wrong version of the same file.

Lawful_G it looks like you change the name of files when you change them giving them a new date? This seems that it could and does cause issues as the files won't be overwritten then when doing an import. Even when manually dropping them into the d20 folder it leaves duplications of the same Things then. So was wondering if using the version info built into the editor now would be easier instead of changing the file name?

Anyone have any other ideas or thoughts to share?

 

[chiefweasel]

for #3, I can change the file names to end in .user instead of .dat.

for #4, I can create a new folder on the website. What xml files would go in there?

I also think one person needs to be a focal point for source documents. A versioning system might help as well, that way we will know what the latest files are. There should also be a test team. and all changes for our sites go through them and get approved by them first. that way is any errors exist they will be spotted and can be corrected before the files go on to the user community.

 

[ShadowChemosh]

for #4, I can create a new folder on the website. What xml files would go in there?

It is actually just one file and it is what will let Hero Lab know where the import files are located when using the Secondary Update URL(Settings->Basic Settings->Specify Secondary Updates URL....) This easy to use feature built into HL will cause a new Download option to appear from within HL. So that the .hl file gets auto downloaded and installed.

Here is an example of the xml file would look like:

<?xml version="1.0"?>
<document signature="Lone Wolf Updates" version="1">
<update
    name="d20 Community 10/4/2010"
    date="2010/10/04:1319"
    size="900"
    description="First release of Lawful_g's awesome d20 data set" uniqueid="0" filetype="3"
download="http://www.cheeseweasel.net/herolab/d20_20101004.hl"
    website="http://www.cheeseweasel.net/"
    companyurl="http://www.cheeseweasel.net/"
    priority="99" major="1" minor="0" patch="1" 
    folder="Pathfinder" special="" dynamic=""    
    />
</document>

So the Name/Description line is what a person would see form within HL. The download tag tells HL where to try and download the .hl file from. The website link tell HL where to open the web browser to if the user wants to download the file manually instead. The companyurl tells HL where to open a web browser to if the user asks to view company info.

I also think one person needs to be a focal point for source documents. A versioning system might help as well, that way we will know what the latest files are. There should also be a test team. and all changes for our sites go through them and get approved by them first. that way is any errors exist they will be spotted and can be corrected before the files go on to the user community.

I agree with this of course. Having some type of QA involved would be really nice. Two people said yes before in the Pathfinder forums. Will need to ask them if they want to help with this also. To do testing they would just need to run d20 in demo mode as the editor still works and one can get stuff to compile. They really don't care that they can't print or save characters for this.

Be good if they are interested to get them in on the ground floor.

 

[ShadowChemosh]

Also anyone have any ideas how to keep the .user files in sync with different people making changes to it?

Something else I was looking at is a free piece of software from https://www.dropbox.com/

It allows the auto-syncing of documents between computers and allows for a shared folder to multiple people to collaborate on a project. If nothing else me and Cheifweasel may want to use this as it would allow an easier method of maybe syncing files across our two sites.

Just something else to think about and free for 2gigs is pretty nice.

 

[chiefweasel]

I have a few folks who wanted to test the files as well, maybe we can have them combine the files as they are tested.

 

[ShadowChemosh]

Great bring them on. Help is always welcome.

So I spent sometime last night going over the d20 files. I think I have everything changed over to .user instead of .dat. Though I tried to remove extra files and have just the 3.5 books. Yea that is not going to happen. The stuff is SO interconnected including referencing 3.0 items, classes, and lots of references to Dragonlance stuff.

I mean it could be done but it would be a major change to many scripts and probably not worth the effort. So the idea of having one 3.0, 3.5, and dragonlance data set is pretty much out the window for now. I also found that Lawful_g started to implement some of the same stuff one sees in the pathfinder files. Like he wrote his own Darkvision that was not 60ft hard coded and instead is Tag or script set. Exactly like what Pathfinder data set uses. That is really a big help to anyone creating d20 stuff.

Our best bet for now is to simply have everything source marked to the right book so that player can pick. But we will have to leave everything together including 3.0, 3.5 and dragonlance. What I will probably do is add new Source Headers so that at least it will be easier break down by 3.0, 3.5, and dragonlance. So this way all the 3.5 book Sources will be listed under the 3.5 Header and so on.

Still this will be a good amount of work......

The last thing is reading some of Lawful_g's comments he has in the scripts and files made me laugh several times last night.

 

[chiefweasel]

Soundfs good, shoot me a copy of the d20 files when you have them ready to go.

 

[Lawful_g]

The darkvision thing was a pain because I had to make a replacement copy of every race in the core files that has darkvision.

Shoot me a copy as well.

Also, always happy to amuse folks. I was originally inspired by Lone Wolf's zombie script to set Con to 0 "because we are undead, and above that sort of petty nonsense."

 

[ShadowChemosh]

Yep don't worry everyone will get a copy. Speaking of copies are you still changing stuff while I am doing this Lawful_g? I guess I had assumed that it would stay in a frozen state. If you are changing stuff can you keep track of each change so I can sync them back into these files. Or would you rather wait until I finish and then you can make the changes directly and we can simply release a new version?

Also another question. At the end of the files you have a what appears to be a date with either a C or U. What does that mean? I am changing all those files to remove that and I am putting that info into the File History section of the XML now.

As people will want to put in new versions of this data set we can't have the files names changing. Otherwise they won't overwrite the existing file and instead the person would have duplicates of the same thing. So if they use the same name the new version will simply overlay the old file.

Otherwise its going along pretty well actually. I am down to the CUSTOM files last night. So I have the file names changed, the file history updated, and making sure everything is SOURCE marked. Plus I added the three new Source Options(3.5 D&D, 3.0 D&D, and Dragonlance Setting).

Last question which is sort of for everyone. What system of version ID is easier? In example should we use the Date we release as the Version Number or better to call this v1.0 and go from their. Thoughts?

 

[bodrin]

I'd prefer the version 1.0 moniker tagged onto the HL import file. Then it's easier to maintain and update the core program.

Once you've tagged the sources correctly Shadow any chance of sending me the updated files please. I'm still missing sources on a few of the custom data. Thanks.

 

[ShadowChemosh]

Once you've tagged the sources correctly Shadow any chance of sending me the updated files please. I'm still missing sources on a few of the custom data. Thanks.

Yep no problem. Though not necessary to ask. We will be first putting this up as a download from Cheifweasel's site. We are also hoping to start using a feature called the Secondary Update URL. This would allow people to directly download the file from Cheifweasel's site and have it automatically install itself directly through Hero Lab. Very similar to how you install official updates now.

So yes the main goal of this project is to have an easier and complete d20 data set for the community and show off all the GREAT work that Lawful_g has done. So no fear I have no plans to keep this for myself. Heck I have little need for this myself actually as my games are all Pathfinder now.

 

[Lawful_g]

I haven't been working on my data files much in the last few weeks. Do you remember the date I sent you that batch, I could check to see if I have changed much. Current project is the Draconomicon feats, which will entail replacing the Dragon Disciple's and Half dragon's breath weapon to forward a user tag for pre req purposes. Also have to mod the dragon races I made to do the same.

C stands for Completed (or near enough), U stands for Uncompleted (needs more work), NM stands for Not Mine (a file I have not changed significantly).

I think the 1.X type is fine, though I am curious about how multiple people will collaborate on making files and updating. You mentioned a website or tool but I can't recall the name.

 

[ShadowChemosh]

I haven't been working on my data files much in the last few weeks. Do you remember the date I sent you that batch, I could check to see if I have changed much.

You sent them to me on 9/19/2010 at 4:41pm CST. Yea just keep track of what changed so we can sync them back in is all. We can consider that 1.1 changes.

Current project is the Draconomicon feats, which will entail replacing the Dragon Disciple's and Half dragon's breath weapon to forward a user tag for pre req purposes. Also have to mod the dragon races I made to do the same.

Nice. Is the Dragon Disc and half-dragon in the files I have or is that base HL?

C stands for Completed (or near enough), U stands for Uncompleted (needs more work), NM stands for Not Mine (a file I have not changed significantly).

I think the 1.X type is fine, though I am curious about how multiple people will collaborate on making files and updating. You mentioned a website or tool but I can't recall the name.

Thanks for the info. I had assumed, but wanted to make sure. I had sort of mentioned a few ideas to help out. Not sure what will be the easiest actually. As you will always be doing the most it may work best to have you in charge of it or in control of the CORE software.

Another idea was I could host an FTP server that different people could access and use some type of check in /check out process from it. I had recently found a new software that is free called DropBox. It allows people to actually collaborate by having a central shared folder that auto syncs it self to each persons computer.

Now it mentions that if I changed a document and you changed the same document that it would merge them back together. I would want to see that in action to see if it was useful or not.

So if we went this route we would each get a free account at DropBox. Install this little software and share a single folder to all our IDs. Then I would drop in all the d20 files when I am done and the software would simply sync a copy to a folder on each of your computers. Then if I changed a file it would be updated back to everyone else and vise versa. This is just an idea is all.

If someone has a simpler method I am all ears. Maybe just emailing changed stuff to Lawful_G to correctly incorporate into the date set? Personally I won't be do much new stuff in d20 actually. My group is moving towards 100% pathfinder and that is where I need to focus on when this is done. So while I have the Healer 99% done I doubt I will have many other additions actually. Plus once done with the d20 I want to move on to the pathfinder files and do something similar.

 

[ShadowChemosh]

So I am getting allot done. I was able to create a Regular Expression that does a pretty good job of putting in Sources with about a 90% accuracy. So that is helping speed things along. Takes some time to figure out the Dragonlanc book as I am not familiar with them but in this case Google is my friend.

With the different books showing up in the sources section of the Configure Your Hero it really shows how many books Lawful_g has actually done. I mean wow!

 

[chiefweasel]

All sounds great.

 

[ShadowChemosh]

Don't have much progress to report right now. Been swamped at work and had to DM over the weekend so have not gotten anything done in days now.

 

[ShadowChemosh]

• Finished adding sources to everything down and through the P .user files. So about 85% done now actually.
• All file extensions where changed from .dat to .user.
• Removed all dates that where part of the file name and moved into the File Version section of the XML.
• Enhanced the Warlock Eldritch Blast script to use the Value from itself when setting the d6 values instead of the local Variable Bonus. This allows other scripts to adjust the value and have the new adjusted value display on the character sheet.
• Healer class from the Miniatures Handbook added.
• Added new Conjuration(Healing) CL Thing to handle any magic item and feats that give a caster level bonus to that specific school of magic. Located in Custom - SpecificSchoolCL.user file.
• All healer Cure spells calculate themselves with a generic script that has its values changes by setting Tags on the spell.
• Added the wavecrest gnome from Stormwrack to the Stormwrack races .user file.
• Added the Warlock Scepter to the MIC magic items .user file.
• Added scripts to several Magic Items and Feats to adjust the Value of the new Conjuration(Healing) CL Thing so that the correct CL is used for calculating the the Cure Spells for the healer.

So Lots are done and it looks really good to be able to just pick the books you want to use from inside of HL and just have those Things show up. Wish I could say that I will be done this week, but super busy again this week and I have to DM all day on Saturday.

So will see what I can finish but I am still thinking two weeks to finish this up. Then we will need to have a few people test this data set to make sure I didn't miss anything. Then we can go ahead and release this bad boy to the community. Should make allot of people happy as there is allot of good stuff done in here from Lawful_g and it will be easier for everyone to see that when they see the Books actually listed in the Configure Your Hero window.

Later

 

[ShadowChemosh]

So I finally finished all the changes last night. I have all the .user files marked up with a their book Sources. I have also created a new hl import file and wrote a xml file for my website. I have the import file loaded to my site just for now for testing. So looking for those Testers now to try this out and see if they find any issues.

Some things to keep in mind. If you already have Lawful_g .user files or files from Cheifweasel's site in your d20 data folder they will need to be removed. This is VERY important those files MUST be removed. If you don't they will cause duplicate Thing errors.

This is only required for this first install ALL future installs will take care of this problem by overlaying the files with new versions. So this only has to be done this one time. If you are just downloading the Hero Lab d20 data set for the first time you will NOT need to do this step.

Steps to get this to work:

• Remove old Lawful_g .user files. You don't have to remove any .user files you have created your self.
• At the top Menu enter the Secondary Update URL(Settings->Basic Settings->Specify Secondary Updates URL....) with the following:

  http://www.shadowsoftware.net/shadowgameworld/pathfinder/herolab/d20.xml

• Press ok and then ok again for the message about restarting HL.
• Restart HL and at the game selection window click on the blue button on the right to Get Downloads.
• Look in the list for 'Lawful_G d20(v1.0)' and click the blue button Download.
• After it finishes downloading(its almost 4 megs) you will be prompted with a new window that will list what is going to be imported and that it will be imported to your d20 data folder. Click the blue button marked Start Import.
• After the install you will be able to select the 3.5 d20 Game system and on the Configure your Hero window you will have several new sections under the User Data branch. These are 3.0 Books, 3.5 Books, Dragonlance Setting, Forgotten Realms, and Eberron Campaign Setting.

Please feel free to test this process out. Once working the stuff will be switched over to Cheifweasel's site. Also check to make sure that their are no errors when just selecting a single source. Keep an eye open for any thing marked to the wrong book. I think I corrected all the ones I found but their could still be more.

Please post any issues or questions to this thread.

Thanks

 

[chiefweasel]

Shoot me a copy. I'll create a test environemnt for it and test it.

 

[ShadowChemosh]

Shoot me a copy. I'll create a test environment for it and test it.

The instructions above will get you that copy. I would like to test the whole process to see if it all works.

In your case when you do the import their is a button to change the folder location you are going into. So instead of going to d20 you can put it into say d20TEST or what ever you call your test environment folder. The folder name must exist in the herolab\data location.

I don't have any access right now to the .user files but when we get this finished I can send them in a zip file to Lawful_g so he can start work then on v1.1 by incorporating his latest changes into the data set.

You can also follow this LINK to download the .hl file manually. Though then you will have to double click on it to have it start HL and start the import then.