logo logo

 Back to main page

The NWNX Community Forum

 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
 
Resman stability and dynamic .jrl file
Goto page Previous  1, 2, 3, 4, 5, 6  Next
 
Post new topic   Reply to topic    nwnx.org Forum Index -> Linux development
View previous topic :: View next topic  
Author Message
leo_x



Joined: 25 Aug 2010
Posts: 75

PostPosted: Fri Sep 26, 2014 16:22    Post subject: Reply with quote

If it ever points to anything different than fileInfo.data it would be to a script loaded by resman and stored in one of the KeyTable's internal CRes structs. So, supposing that's the case -- and it might not be -- then at least you'd have an upper bound on what you could leak: scripts added to physfs after they'd already been loaded by CExoResMan would leak once.

I'll take a look again this weekend, but that it works without crashing is probably the most important thing. Laughing
_________________
the awakening (PW Action)


Last edited by leo_x on Fri Sep 26, 2014 16:26; edited 2 times in total
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Fri Sep 26, 2014 16:24    Post subject: Reply with quote

leo_x wrote:
If it ever points to anything different than fileInfo.data it would be a script loaded by resman and stored in the internal CRes stored in a keytables before it was add into a physfs or whatever. So, supposing that's the case -- and it might not be -- then at least you'd have an upper bound on what you could leak: scripts added to physfs after they'd already been loaded by CExoResMan would leak once.

I'll take a look again this weekend, but that it works without crashing is probably the most important thing. :lol:


Alright, that explains it! Thanks.
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Sun Sep 28, 2014 13:25    Post subject: Reply with quote

In case you decide to work on this some, there's another issue.

Sometimes, it stops serving script files over physfs (didn't test anywhere else).

Quite suddenly, spell scripts stop working, no other scripts are being called. SOME still work though, like event handlers for for nwnx_events or _chat.

This here should have ExecuteScript()'d another, but it never did. What stumps me is the all-zero fields.

Code:

File: msg_chat.ncs, Exists?: 1, mtime: 1411754134
ID: -1: msg_chat
Skipping msg_chat.ncs... Data: 0x11c72e10
Calling CRes*::OnResourceServiced: 0826a784
Resulting Structure:
- m_nDemands = 1
- m_nRequests = 0
- m_nID = 00000000
- m_pResource = 11c72e10
- m_nSize = 30150
- flags = 4
- m_status = 2
- m_pKeyEntry = 00000000


But I don't know yet what it means. :)

Edit: I guess it happens SOMETIMES just after updating/replacing/deleting scripts in a physfs-directory, although I cannot imagine it is nwnx_physfs which is having problems (except maybe physfs_sint64 overflowing time_t). Will watch this closely.

Edit 2: dbl. free()s have stopped since I changed it as shown above. Server memleaks, though I wouldn't know if it is because of my change.
Back to top
View user's profile Send private message
farhorizon



Joined: 07 Dec 2012
Posts: 12

PostPosted: Mon Sep 29, 2014 14:59    Post subject: Reply with quote

Sorry for slight off topic, but is it safe to use the resman so from earlier in this thread to address crashing issues in the standard resman plugin?

Based on my reading, it wouldn't require the physfs so, correct?
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Mon Sep 29, 2014 17:15    Post subject: Reply with quote

I have been using some older version of resman for a long time with no crashes and minimal memleaking, yes. Not sure if it is the one posted earlier.
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Tue Sep 30, 2014 15:44    Post subject: Reply with quote

@problem above: Original resman has the same issue.
Back to top
View user's profile Send private message
leo_x



Joined: 25 Aug 2010
Posts: 75

PostPosted: Tue Sep 30, 2014 17:11    Post subject: Reply with quote

Weird, is the size right? I noticed when I updated that I'd sometimes get script error... invalid bytecode. But that it would resolve itself maybe when the file size changed. The directory part of physfs is just a light wrapper of the normal OS filesystem operations.

m_nID being zero means it's not indexed in a key table, I guess it's normally a hash of the resref, which explains m_pKeyEntry being 0 because that's a NULL pointer that'd point back to the the actual key table entry had it existed.

How much memory is the server leaking?

@farhorizon if you're not serving scripts from resman that aren't somewhere in your module, hak, or a bif, I think the one in the master repository works fine. I wouldn't use the one I posted earlier, because it's never going to be worked on again. I'll edit that post.
_________________
the awakening (PW Action)
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Tue Sep 30, 2014 18:02    Post subject: Reply with quote

It's a really weird thing, goes like this, and it happens both with upstream resman and nwnx_resources+physfs:

I can click-spam store "A" which works exactly 7 times - loading dlg, utm, and some utis. The 8th time, resman doesn't even see the request for the .dlg anymore, and the dialogue doesn't open. At the same time, most scripts stop working.

Meanwhile, in the same situation, i can click-spam store "B" and suddenly "A" stops working, while B still does. Note that stores are re-created on each click, so they're not left in the area.

Most cached scripts still work (like NWNX event handlers), but new scripts are not found (and resman does not see the request to handle them either).

I've tested this with only resman and no external providers, it still fails.

This behaviour didn't happen before, even though I am running original upstream code. The only thing changed is that I load areas from resman now, and all game resources are either in resman directly, or in redis which talks to resman via pluginlink.

*scratchy-heady*

Returned file sizes are correct in both accounts. I got the script errors too, and they might be related to the internal nwserver cache. Not sure. They never were a problem that looked like this, only on first invocation after updating.

Memory leaks are not what I'm worried about right now - considering I've switched back to the upstream resman. It looks like it's about 10MB/h.
Back to top
View user's profile Send private message
leo_x



Joined: 25 Aug 2010
Posts: 75

PostPosted: Tue Sep 30, 2014 18:50    Post subject: Reply with quote

Honestly, it's this kind of random crap that put a damper on my enthusiasm for the project and pretty much any NWNX stuff except my Lua plugin.

The virtual machine does bail out when when the script recursion level is great than 7 right at the start of CVirtualMachine::ReadScriptFile... but why on earth that would be happening because of resman? And why would it only affect some scripts?
_________________
the awakening (PW Action)
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Tue Sep 30, 2014 19:03    Post subject: Reply with quote

I can understand. As I said, it's not script files as such, it's ANY ref that is loaded after the fact.

I'll see about getting a testcase/env up ..
Back to top
View user's profile Send private message
leo_x



Joined: 25 Aug 2010
Posts: 75

PostPosted: Tue Sep 30, 2014 19:28    Post subject: Reply with quote

Oh, I just assumed it was the onconv script blocking the BeginConversation from loading the dlg. Say, if you relogged with a different toon would the UTI for the equipment be bypassed?
_________________
the awakening (PW Action)
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Tue Sep 30, 2014 20:12    Post subject: Reply with quote

leo_x wrote:
Oh, I just assumed it was the onconv script blocking the BeginConversation from loading the dlg. Say, if you relogged with a different toon would the UTI for the equipment be bypassed?


No, it's not the script blocking. The script doesn't get requested either; no resman requests seem to work (I think).

I don't understand. What do you mean by "bypassed"? Inventories are embedded in the charfile, they are not loaded via resman.
Back to top
View user's profile Send private message
leo_x



Joined: 25 Aug 2010
Posts: 75

PostPosted: Tue Sep 30, 2014 20:45    Post subject: Reply with quote

Well the script wouldn't get demanded if it's the recursion level issue.

No clue what I was thinking there, sorry. Embarassed
_________________
the awakening (PW Action)
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Tue Sep 30, 2014 20:48    Post subject: Reply with quote

Oh, get it.

Though I don't see how I can even reach 7, assuming it works the way I think.

* nwnx_events calls msg_chat

* msg_chat does some parsing on "/*" and calls cmd_*.ncs.

That'd be recursion level 0->1, right?

I don't see how clicking on a NPC to trigger a simple conversation be at 7 unless it wouldn't reset for some reason.

And, similarily, why this would fail on NPC A, but then (even after triggering the issue) still work on B (where nw_c9_conv or whatever still fires and runs the dlg + merchant) - more than 7 times, too.

This sucks.
Back to top
View user's profile Send private message
leo_x



Joined: 25 Aug 2010
Posts: 75

PostPosted: Tue Sep 30, 2014 21:21    Post subject: Reply with quote

Yea, no clue why it would happen it doesn't make sense either way Laughing . Recursion level would be bumped everytime ExecuteScript is called, so really I have no idea why that would cause anything, but the return value of that function would be an easy place to start.

I don't know that it would be an issue with that per se, but maybe memory corruption?

Maybe I'm missing it, but I honestly can't see any other way a script wouldn't hit the resman hook.
_________________
the awakening (PW Action)
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    nwnx.org Forum Index -> Linux development All times are GMT + 2 Hours
Goto page Previous  1, 2, 3, 4, 5, 6  Next
Page 4 of 6

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group