Much Needed Status Update

[rob]

@snow crash:
All of these setbacks have been disappointing for us, as well. Having an employee do what we experienced was incredibly painful. I wish I could just snap my fingers to fix things overnight, but I can't. The only thing we can do is work to steadily dig ourselves out of the hole that we got into, which is exactly what we've been doing. And it's gonna take time for us to complete that process.

As to whether you should continue using HLO while we undertake this effort, that's a question only you can answer. HLO absolutely offers a long and growing list of great features for SR6, albeit without the latest supplements. Some groups will be content to hold off introducing those supplements into their games until we can get them added, while others will require all the new shiny bits now and explore other options. Whatever you choose on your end, we're gonna continue making headway on our end - slower than everyone wishes were possible, ourselves included, but headway nonetheless.

[markelphoenix]

Originally Posted by rob

@snow crash:
All of these setbacks have been disappointing for us, as well. Having an employee do what we experienced was incredibly painful. I wish I could just snap my fingers to fix things overnight, but I can't. The only thing we can do is work to steadily dig ourselves out of the hole that we got into, which is exactly what we've been doing. And it's gonna take time for us to complete that process.

As to whether you should continue using HLO while we undertake this effort, that's a question only you can answer. HLO absolutely offers a long and growing list of great features for SR6, albeit without the latest supplements. Some groups will be content to hold off introducing those supplements into their games until we can get them added, while others will require all the new shiny bits now and explore other options. Whatever you choose on your end, we're gonna continue making headway on our end - slower than everyone wishes were possible, ourselves included, but headway nonetheless.

Wait, employee did what? If y'all got done dirty that buys a lot of forgiveness in my eyes.

[Duggan]

Originally Posted by markelphoenix

Wait, employee did what? If y'all got done dirty that buys a lot of forgiveness in my eyes.

I don't think they've elaborated on it too much, but the impression I got was that the server code for Hero Lab Online (and maybe as far back as RealmWorks) wasn't written properly, which is part of the reason for all of the outages. And it sounds so far like it wasn't anything malicious, just an employee who didn't quite make the bar (and boy am I familiar with that situation...), but when your flagship product keeps going down, it's not good.

Honestly, it's one of those situations where being nice can really come back to bite you, since an underperforming developer (or worse, one who looks to be putting out good output only for it to turn out their code is buggy and needs to be rewritten) not only puts you in the hole for their wages, but also for the cost of fixing their work. You want to give people a chance, but you have your company to look out for.

[dacoobob]

Originally Posted by Duggan

Honestly, it's one of those situations where being nice can really come back to bite you, since an underperforming developer (or worse, one who looks to be putting out good output only for it to turn out their code is buggy and needs to be rewritten) not only puts you in the hole for their wages, but also for the cost of fixing their work.

There's a big difference between "being nice" and failing to implement an effective QA process. My boss is a super nice guy, but he also reviews my work before we send it out-- which i appreciate.

[markelphoenix]

Kind of curious if they use Test Driven Development with gated deploys around a CI/CD pipeline. May be harder for a smaller outfit as well, but pair programming creates interesting environment of collaboration and skill sharing. Also reduces doom states by making sure that a sole developer can't continue to spiral down a pit of doom, given that they always have a co-pilot to step on the breaks. Earlier, the better.

[rob]

I have avoided going into too many details, since that's just airing dirty laundry, but I guess it's important for context on how we got here.

About 3.5 years ago, I nearly died, and the recovery was protracted. I was on truly nasty meds that both clouded my brain AND made it so I could only sleep about 3 hours out of every 24, which just compounded the brain fog. I talked regularly with the developer that messed up, and I provided what guidance I could in that mental state. This was a VERY senior developer, and I beseeched him to bring his "A" game, since I was seriously gimped. Alas, it turns out he was going through issues outside of work, and that massively impacted his work performance. So I wasn't able to provide the necessary oversight, and he brought his "D" game (the server limped with lots of issues). It wasn't malicious on his part, but it certainly was bad timing, and I was in no state to recognize the problems as they were unfolding.

After finally getting off the meds, it took me a full 18 months to finally flush them all out of my system. I began regaining a little clarity last spring (7-8 months after stopping the meds), and I finally was able to actually get a good night's sleep (read: 6 hours) this past April. Having some semblance of clarity last summer, it became apparent things were messed up with the server code, but I assumed it was a handful of issues that could be addressed and worked with the developer to address them. When that merely uncovered new issues, I launched a detailed code review and discovered the extent of the problems - they were fundamental design failures. At that point, all I could do was accept the reality of the situation and figure out how to salvage things, which is why the server code has now been rewritten.

That doesn't change the fact that we are now more than a year behind where we should be in terms of progress. And all the frustrations of users due to these setbacks is a reality we're slowing working to repair.

As for instituting things like pair programming, that's not really feasible with a development team of only 4, especially when spread across a range of dissimilar technologies (JS-client, C#, C++, etc.) and domains (client, server, HL engine, licensing/sales). We utilize peer-review of all checkins within the client, where we have two very capable developers with that expertise. However, we don't do that on the servers (C#) and in the HL engine (C++), where only a single developer has good domain knowledge right now.

We also employ peer-review of all checkins within the data file team, where we have a team of 6 working on those.

FWIW, we do use a CI/CD pipeline for the client-side development. And we have a dedicated QA person.

I'm not sure if that answers all the questions that people have, but that ought to at least provide some useful context.

[markelphoenix]

Originally Posted by rob

I have avoided going into too many details, since that's just airing dirty laundry, but I guess it's important for context on how we got here.

About 3.5 years ago, I nearly died, and the recovery was protracted. I was on truly nasty meds that both clouded my brain AND made it so I could only sleep about 3 hours out of every 24, which just compounded the brain fog. I talked regularly with the developer that messed up, and I provided what guidance I could in that mental state. This was a VERY senior developer, and I beseeched him to bring his "A" game, since I was seriously gimped. Alas, it turns out he was going through issues outside of work, and that massively impacted his work performance. So I wasn't able to provide the necessary oversight, and he brought his "D" game (the server limped with lots of issues). It wasn't malicious on his part, but it certainly was bad timing, and I was in no state to recognize the problems as they were unfolding.

After finally getting off the meds, it took me a full 18 months to finally flush them all out of my system. I began regaining a little clarity last spring (7-8 months after stopping the meds), and I finally was able to actually get a good night's sleep (read: 6 hours) this past April. Having some semblance of clarity last summer, it became apparent things were messed up with the server code, but I assumed it was a handful of issues that could be addressed and worked with the developer to address them. When that merely uncovered new issues, I launched a detailed code review and discovered the extent of the problems - they were fundamental design failures. At that point, all I could do was accept the reality of the situation and figure out how to salvage things, which is why the server code has now been rewritten.

That doesn't change the fact that we are now more than a year behind where we should be in terms of progress. And all the frustrations of users due to these setbacks is a reality we're slowing working to repair.

As for instituting things like pair programming, that's not really feasible with a development team of only 4, especially when spread across a range of dissimilar technologies (JS-client, C#, C++, etc.) and domains (client, server, HL engine, licensing/sales). We utilize peer-review of all checkins within the client, where we have two very capable developers with that expertise. However, we don't do that on the servers (C#) and in the HL engine (C++), where only a single developer has good domain knowledge right now.

We also employ peer-review of all checkins within the data file team, where we have a team of 6 working on those.

FWIW, we do use a CI/CD pipeline for the client-side development. And we have a dedicated QA person.

I'm not sure if that answers all the questions that people have, but that ought to at least provide some useful context.

It answered a lot. Thank you for the honesty, vulnerability, and clarity. LoneWolf will continue to have my business and support.

Given that my day job is in Enterprise Software Engineering, I am not sure what if any support I can provide given my contractual obligations, but if there is ever any volunteer opportunities, I'm pretty sure volunteer passion projects done on personal hardware would not count as moon lighting.

[thaX]

Thank you, Rob, for your very candid insights on the background of company woes.

I am glad that you are better and hope that you are treating yourself and your time with care.

The whole situation seems like one big problem that the crew you have and yourself is tackling one piece at a time, and I have seen the improvements with the App over time. With the influx of revenue as the new features have been implemented, I hope you can get a couple of more people in to help out with the catch up.

[markelphoenix]

Rob, is there any way to open source the Shadowrun source material, or allow volunteer efforts?

Right now, I know of Genesis Character Generator and an Excel Character Generator that is being maintained by community. If given the opportunity, I would be surprised if people did not contribute.

[VaultScrolls]

You still have my support, both rob & LW! If I can provide any help with the data file work just let me know! I use HLO about everyday in some way with PC management for playing Starfinder. I couldn't imagine not having it!