PDA

View Full Version : Problems getting data


GravenImage
July 22nd, 2010, 07:57 AM
So I ran ddidownloader.exe, I choose to get data with my insider account, it connects, retrieves everything, downloads backgrounds, processes deity index, then attempts to process skill index and errors out. Pasting output below..


Connecting to server...
Login succeeded.
Retrieving deity index... done.
Retrieving skill index... done.
Retrieving ritual index... done.
Retrieving item index... done.
Retrieving class index... done.
Retrieving race index... done.
Retrieving paragonpath index... done.
Retrieving epicdestiny index... done.
Retrieving feat index... done.
Retrieving power index... (page 1/5) done.
Retrieving power index... (page 2/5) done.
Retrieving power index... (page 3/5) done.
Retrieving power index... (page 4/5) done.
Retrieving power index... (page 5/5) done.
Downloading backgrounds... done.
Processing deity index... done.
Processing skill index... Could not parse XML document 'C:\Users\Ryan\AppData\Lo
cal\Temp\ddicrawler\skills_index_0.xml'. skill import could not continue.


Press any key to quit.


Any help appreciated. Running Win7 64-bit, brand new DL of Hero Lab.

digitaldrake
July 22nd, 2010, 01:18 PM
Tried both with the insider account and without. The downloader runs, retrieves everything, but when it begins processing the skills index, it quits.


Connecting to server...
Retrieving deity index... done.
Retrieving skill index... done.
Retrieving ritual index... done.
Retrieving item index... done.
Retrieving class index... done.
Retrieving race index... done.
Retrieving paragonpath index... done.
Retrieving epicdestiny index... done.
Retrieving feat index... done.
Retrieving power index... (page 1/5) done.
Retrieving power index... (page 2/5) done.
Retrieving power index... (page 3/5) done.
Retrieving power index... (page 4/5) done.
Retrieving power index... (page 5/5) done.
Downloading backgrounds... done.
Processing deity index... done.
Processing skill index... Could not parse XML document 'C:\users\james\Temp\ddic
rawler\skills_index_0.xml'. skill import could not continue.


Press any key to quit.

I would buy a license if this could be fixed.

WinXP SP3. Fresh Download of Hero Lab and the 4e.hl

GravenImage
July 22nd, 2010, 01:21 PM
Glad to see someone else is having problems. Is it possible that something was done different with this month's Insider content that broke it?

GravenImage
July 22nd, 2010, 07:59 PM
So, I went and looked over the source, and once I got a feel for the process, I checked the XML files that were being DLed while the act was still in progress. It's possible I missed somewhere in the source where some form of decoding or decrypting is used on the file, but the data definitely looked encrypted. Attached the first few lines from skills_index_0.xml as DLed by ddidownloader..


PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz 4NCjxEYXRhPg0KICA8UmVz

dWx0cz4NCiAgICA8U2tpbGw+DQogICAgICA8SUQ+MTwvSUQ+DQ ogICAgICA8TmFtZT5BY3Jv

YmF0aWNzPC9OYW1lPg0KICAgICAgPFNvdXJjZUJvb2s+UGxheW VyJ3MgSGFuZGJvb2s8L1Nv


Thoughts? If it IS encrypted, which would likely make the Insider content inaccessible, is it possible to get a refund? I only got my license for that reason. :(

digitaldrake
July 22nd, 2010, 08:21 PM
I've been looking over the source myself a bit and can't find any reason for it. I did the same thing earlier and looked at the output .xml files the downloader created. It looked encrypted.

I'm no aspx code monkey in the least and I don't want to start now. I'll hold my thoughts on it though until someone from the Lab can respond.

digitaldrake
July 24th, 2010, 08:49 AM
So I've gotten a little farther, but still no luck in importing any real data.

I downloaded the ddidownloader.exe from the google code site: http://code.google.com/p/ddidownloader/downloads/detail?name=ddidownloader.exe just to see if it would act any differently. Much to my surprise, it did.

No errors whatsoever. However there was no data imported. Open up any ddI_*.dat file and no data is there. Open up the output.txt file and there are a whole host of errors presented.

I think your initial conclusion was correct Graven, the content was changed so much by DDI that the downloader is not currently functional. Not that this really helps you, Graven, but I thought I would update you on what's really going on.

Below is the text from output.txt:

Logging in...
Connected to server...
Login succeeded.

>>> Downloading skills...
Downloading page 1... completed.
No <WizTableSkills> tag found in document.
Downloading page 2... category completed.

This repeats for all other categories until it gets to powers

>>> Downloading powers...
Downloading page 1... completed.
No <WizTablePowers> tag found in document.
Downloading page 2... completed.
No <WizTablePowers> tag found in document.
Downloading page 3... completed.
No <WizTablePowers> tag found in document.
Downloading page 4... completed.
No <WizTablePowers> tag found in document.
Downloading page 5... completed.
No <WizTablePowers> tag found in document.
Downloading page 6... category completed.
Could not find node 'skInsight' to overwrite.[...]Could not find node 'mgAnkBra19' to overwrite.Could not parse XML document ddidownloader\ddi_classes.dat.
22 - Encountered unknown element tag 'assignval'Could not parse XML document ddidownloader\ddi_races.dat.
421 - Encountered unknown element tag 'assignval'Could not find node 'fParArcUnd' to overwrite.[runs through all the skill tags]Could not find node 'fEpcStrby' to overwrite.Could not parse XML document ddidownloader\ddi_feats.dat.
328 - Unrecognized attribute 'uniqueness' in element 'thing'Could not find node 'pRacDarkfi' to overwrite. [runs through all the skill tags] Could not find node 'pClrHeaMer' to overwrite.Could not find node 'pArtHeaShi' to overwrite.Could not parse XML document ddidownloader\ddi_languages.dat.
18 - Unrecognized attribute 'uniqueness' in element 'thing'Could not find node 'dtyAmaunat' to overwrite.Could not find node 'dtyAsmoFor' to overwrite.Could not find node 'dtyBahaFor' to overwrite.Could not find node 'dtyBaneFor' to overwrite.Could not find node 'dtyGond' to overwrite.Could not find node 'dtyGruuFor' to overwrite.Could not find node 'dtyKelemvo' to overwrite.Could not find node 'dtyLoltFor' to overwrite.Could not find node 'dtyOghma' to overwrite.Could not find node 'dtySelune' to overwrite.Could not find node 'dtySilvanus' to overwrite.Could not find node 'dtyTiamFor' to overwrite.Could not find node 'dtyTempus' to overwrite.Could not find node 'dtyTymora' to overwrite.

Import complete. Reload the game system to see the new content.

I also attached my output.txt file in case someone from Support wants to see it.

neowolf
July 24th, 2010, 10:56 AM
Getting the same problem here, I suppose the compendium update caused some issues?

GravenImage
July 24th, 2010, 11:33 PM
Thanks for the update, digital. Hopefully we'll hear some official word soon. I'm hoping it can be resolved as I was really looking forward to finally having a good way to maintain sheets that needed house rules.

Colen
July 26th, 2010, 09:44 AM
Yes, Wizards did a compendium update 2 weeks later than usual - they usually do it on the first Tuesday of the month, so I looked for it then. When it didn't happen, I assumed they were skipping the update for this month. Now it turns out they did it 2 weeks later than normal, *and* they changed the structure of some of the data. No rest for the wicked, I guess :)

I'll have an updated downloader out ASAP. Sorry for the delay!

GravenImage
July 26th, 2010, 10:20 AM
Woot, glad to hear that! Looking forward to it. And no sweat about the delay, I know how it goes, especially in software development. Heh.

tscottt
August 7th, 2010, 11:46 AM
The downloader is progressing for me these days. But on the 8177 item entries, it is printing out a lot of "!" instead of the occasional "." marks. Is this indicative of a problem processing an entry (or a bunch of them)?

-Scott

Colen
August 12th, 2010, 12:32 PM
An ! during the download means that the downloader encountered a problem downloading those entries. Once everything else is downloaded, it goes back to the "problem" entries and tries again; generally things work the second time around.

If the second attempt fails, the downloader stops and asks you whether you want to try again. In that case, I recommend waiting a few hours (maybe overnight) before saying "yes"; the D&D Compendium occasionally goes wrong for no clear reason, and stops sending the right pages out. If that happens, it usually fixes itself within a few hours, so everything should work when you retry the next day.

(The download problems aren't unique to the downloader - if you go to the downloader in your web browser and log in, you'll get error pages returned for the same items that the downloader is having problems with.)

Hope this helps!

tscottt
August 12th, 2010, 04:41 PM
Thank you for the advice. The importer has been running for about 5 days on a quad-core, 8GB computer, and it's still going. I see mostly errors "!" I've attached a screen shot image.

I will try again, and perhaps the next week of downloads will work. :)

-Scott

Colen
August 13th, 2010, 04:06 PM
Whoa, that looks bad - it certainly shouldn't take more than a few hours to download content, let alone multiple days.

If you're getting that many errors, try going to the D&D Compendium web site:

http://www.wizards.com/dndinsider/compendium/database.aspx

Log in, and try looking at maybe half a dozen items or powers. Do you get the correct information, or an error page (or a "you must subscribe" page)?

If the downloader has been running for more than a day or so, I'd advise closing it and just starting over, since there's obviously something wrong. Restart the downloader, re-enter your login information, and try again - does it work any better this time?

tscottt
August 15th, 2010, 06:56 PM
Well, I ran it again and everything worked. Mystery bugs...

ryntak4536
August 22nd, 2010, 04:24 AM
I am on my 4th attempt now, and still get the !!!!!! as it tries to download the powers. Every time.....

What format does it get saved to when it is downloaded and before it is put into herolab? Does it download the data as text files?

Gord
August 22nd, 2010, 05:40 PM
I have a somewhat different problem. The ddi files seem to load ok but all the data for the base weapons and armor have no value. For example, except for the Unarmed strike, all the weapons show zero damage and armor has a zero modifier.

Could this be because I am testing the software using a corporate key for my store?

Colen
September 7th, 2010, 02:03 PM
I have a somewhat different problem. The ddi files seem to load ok but all the data for the base weapons and armor have no value. For example, except for the Unarmed strike, all the weapons show zero damage and armor has a zero modifier.

Could this be because I am testing the software using a corporate key for my store?

Did you log in with a D&D insider account? If not, the downloader can't access many of the mechanics (like damage values, etc) from the compendium. They used to give you all the content from the core rules, like weapons etc, so I'm surprised to hear those aren't being included any more...

Colen
September 7th, 2010, 02:04 PM
I am on my 4th attempt now, and still get the !!!!!! as it tries to download the powers. Every time.....

What format does it get saved to when it is downloaded and before it is put into herolab? Does it download the data as text files?

There should be a file named "output.txt" in your HeroLab\data\4e folder that records the status of the download. Please email it to me and I'll see if I can work out what's going on.