Is it possible to stop a thrown weapon from being a tracked resource?
 

A custom weapon with the "Thrown Weapon" category automatically gets added to the tracked resources list. Is there any way I can make the weapon not appear here, but otherwise still function the same?

In the Community Pack there is an Other Adjustment for "Equipment, Remove Trackable". You can use that, or look at it to see how it works.