NWScript Accelerator plugin release thread

[Skywing]

Posted release 6.

This release is a major feature release and includes the capability to write scripts in CLR languages like C#, C++/CLI, etc. CLR scripts can access threading, native sockets, and other programming capabilities above and beyond that accessible to NWScript. They can also seamlessly call nwscript.nss engine functions from native CLR code, including first-class DelayCommand support (no clunky wrappers required!).

Additionally, it is even possible to call functions written in NWScript from CLR script code, which means that you can reuse portions of your NWScript-based script libraries without having to rewrite them from scratch.

This thread has details, including an SDK with several sample C#-based CLR scripts, of how to get started with creating your own CLR scripts. To enable CLR scripts to run, you must edit AuroraServerNWScript.ini and set AllowManagedScripts=1.

Other features in this release include:

- You can now turn off execution guards for generated scripts (disabling the infinite loop and infinite recursion checks). This is not recommended.. Check out the included example AuroraServerNWScript.ini for details.

- You can now customize the maximum loop iterations and maximum call depth (recursion check) for NWScript scripts. Check out the included example AuroraServerNWScript.ini for details. It is recommended that you use the default settings for these optiongs.

- You can now save the generated assemblies that the NWScript JIT engine creates by setting the CodeGenOutputDirectory= option in AuroraServerNWScript.ini. This can be useful if you want to examine the generated assemblies in ILDasm or other .NET introspection tools. Useful for CLR script developers that would like to call a NWScript-based function from CLR script code.


In addition, an important bug fix is included in this release:

- When encountering an invalid UTF-8 code point, the NWScript JIT engine would previously map the code point to U+FFFD (the Unicode REPLACEMENT CHARACTER). This caused some scripts which treat UTF-8 strings as raw byte strings to become stuck into infinite loops in some circumstances. Now, invalid code points are mapped to U+003F (the Unicode QUESTION MARK character) which does not cause this problem.

[0100010]

Already taken a look through the SDK code (though not for release 6 if that source was updated)


Quick questions, why can't NWScriptActions, NWScriptConstants, and NWScriptSupport (the CLRScriptBase class etc under namespace CLRScriptFramework) be into its own dll refernece by just the project that has to use CLRScript and StandardEntryPoint?

Was there a reason to keep them as files that have to be copied into each individual script project that I am just overlooking? Wouldn't it be simpler to carve them off if they are not going to be changed by the scripter?

Alos if the methods inside StandardEntryPoint are intended to be the same across all CLRScriptBase child classes why not have them in CLRScriptBase to begin with? You could carve the entry point file off as well the same and just have the single CLRScript.cs templated file in the script project.

Lastly.. due to the repeated steps of setting up references choose the CLR20 vs 4 and adding the post build step, etc. I would suggest creating a project template that can get dropped into VS also to make it even easier.

[Skywing]

Thank you for taking a look. Let me know if you have further questions as I'm happy to answer them.

The SDK sample didn't change, provided you had downloaded them in the past week or so. There had been some bugfixes for problems reported a little bit before that.


The organization of the SDK could still be improved. I am absolutely open to suggestions on how to do that as there may be ways to make it even cleaner to use - more below:


You are correct in that it would be nicer if some of the code could live in a common assembly. I didn't do this for a couple of reasons:

1) I wanted to make it easy for scripts to be as standalone as possible, without having to install a mass of dependent assemblies in a well-known location (or even the GAC). This would significantly complicate rollout of CLR scripts. With them being standalone as they are, you can literally include a CLR script in a .hak file shared with multiple modules without issue (or in the module proper, as you desire). One of our earlier adopters uses this capability to distribute CLR scripts to multiple servers with a common hak file.

2) There are some tricky aspects as to how a dependent assembly would be resolved that I wanted to shield script authors from. (Of course, since you have the full CLR API available, you can do the work yourself, but I didn't want to involve a bunch of complicated steps to get a dependent assembly to work that were *required* to get one's first script up and running.)

Specifically, dependent assemblies would be search for in the GAC and in the game installation directory by the default search rules (unless a custom AssemblyResolve event hook is provided to locate the dependencies via customized means). Requiring assemblies to be placed in the game directory interferes with the goal of making them easily hakpak-compatible, and it's a bit of a maintenance headache. You can do it if you want to, of course, but it seemed like enough of a hurdle that it was worth avoiding.

[As a side note: Indeed, some users of the CLR script support have written the code to do the AssemblyResolve hooking for having common reference assemblies. But this entailed some of the issues and downsides I outlined above. And, of course, the main script type couldn't reference anything that would prevent it from being instantiated before it could run to set up the custom AssemblyResolve hook.]


If you have a suggestion on how to organize this better without making it a headache to manage the installation and loading of the reference assembly, I'd certainly like to eliminate the need to go and add the source files to each new project.



With respect to the inclusion of code in StandardEntrypoints, this relates to how the NWNScriptJIT loader locates the main script type in your CLR script assembly. For that, a bit of background knowledge is helpful.

The IGeneratedScriptProgram interface and most of the underlying constructs used by the CLR script support were originally engineered for use by the dynamic assemblies emitted by NWNScriptJIT when it JIT's a compiled NWScript script to MSIL code. (Indeed, every such script JIT'd by NWNScriptJIT implements IGeneratedScriptProgram.)

In the original scenario, NWNScriptJIT knew exactly which type it wanted to instantiate from a given dynamic assembly as the 'main' script type for that assembly, because it simply remembered the type as it constructed the assembly itself.

This doesn't map directly to CLR scripts for which NWNScriptJIT doesn't know a-priori which type within the assembly is the 'main' script type. Thus, I selected the following mechanism for NWNScriptJIT to identify which type to instantiate as the 'main' script tyoe for a CLR script assembly:

- Enumerate all of the types in the assembly, looking for public types
- Exclude all types that do not implement IGeneratedScriptProgram
- Take the first type found

This means that a base class can't, for example, implement some of the boilerplate methods for IGeneratedScriptProgram and just call directly to your ScriptMain, unfortunately. (Because otherwise, NWNScriptJIT would try to create an object of the base class type and not the most derived class.) One could imagine schemes of trying to find the most derived type in a hierarchy involving IGeneratedScriptProgram, but this seemed error-prone and fragile, versus a simple rule of 'there must be exactly one type in an assembly that implements the interface, and it should be the main type for the assembly's communication with the JIT interface'.



An additional note: You may be wondering, with all of the ado above about resolving dependent assemblies, as to how NWScriptManagedInterface.dll is pulled in, as the plugin package doesn't ship the DLL. Indeed, there is no such DLL on disk in the runtime environment; the assembly is a 'virtual' one, and if you look at the assembly in the SDK in ILDasm, you'll see that the NWScript interface methods are all stubs without any code in them.

Internally, NWNScriptJIT creates an assembly compatible with NWScriptManagedInterface.dll, but with a unique name, during startup. This assembly contains code to directly call into the host for engine functions in a way that is specific to the memory layout of the process (for a slight improvement of performance).

When NWNScriptJIT loads a CLR script assembly, it itself hooks the AssemblyResolve event and manually wires the dynamically generated NWScriptManagedInterface-compatible assembly up when the caller references NWScriptManagedInterface.dll. This, as an aside, also implies that the main script type for a CLR script has to have at least one reference to NWScriptManagedInterface to trigger it to be 'wired up' at the appropriate time.

If you want, you can set CodeGenOutputDirectory= in the configuration file for the plugin, and then examine the NWScriptManagedInterface_<pointer>.dll in ILDasm to see what the 'real' NWScriptManagedInterface assemblies are like.


Furthermore, the reason why I didn't include some of the common code here in NWScriptManagedInterface.dll itself is that this would make future compatibility with existing compiled CLR-based scripts difficult (and all of the code in NWScriptManagedInterface.dll is dynamically generated at runtime). As it is now, you should be able to upgrade to a future version of the NWScript Accelerator plugin without breaking your scripts. This is because the interface in NWScriptManagedInterface.dll is expected to remain stable (at least for the current NWN2 patch level), as it only exports the raw engine functions without any other logic.

Indeed, you can run a CLR script built with the current SDK against both the beta and the release 6 plugin versions. While changes can be made to the SDK framework in the future, this won't break existing compiled scripts; it's a priority for the project that upgrading the core plugin (such as for a bugfix) won't necessitate a painful and disruptive requirement to rebuild all CLR scripts, re-stage haks, etc.

[0100010]

[Skywing]

The problem is that someone has to know how to find those assemblies and connect them in the AssemblyResolve event (often before any of the user supplied code even runs, if the main class incorporated any references to types in the dependant assembly).

[luna]

Hi Skywing. We deployed this at the Baldurs Gate PW again using release 6.

I'm seeing some not so good behavior with our nwn2server process memory usage.

Normally I would say our peak memory usage was maybe 2.5GB.

After installing the script accelerator, I've seen our memory go up to 2.9GB which is getting close to crashing due to crossing over the 32-bit process memory usage of 3GB.
And we have seen 2 phantom crashes with no error log in the nwn2 directory, so we think possibly we hit 3GB and the server process crashed.

One other thing is it just seems that over time memory usage keeps slowly going up (this was not happening when we didn't run the script accelerator).


Is it possible the script accelerator has a memory leak?
Or does it just consume a lot of memory?
Or any other thoughts on what might be going on with the memory usage?

Thanks

[Skywing]

I'd strongly recommend upgrading to a 64-bit host OS which would let you use 4GB of VA space instead of just 3GB. Second, upgrade to the latest xp_bugfix version which, if you were using an old version, might reclaim up to 700MB of wasted memory by repairing some problems with the campaign database code.

The plugin will consume additional memory because scripts are kept in memory, in their JIT'd form, once they are first referenced. This is necessary to provide execution speed improvements. Once most scripts have been executed for the first time, memory usage reaches a stable state. I'm not aware of any true memory leaks in the code today.

There is a configurable memory usage limit in AuroraServerNWScript.ini that you can tweak to set a minimum amount of free address space at which the plugin will stop JIT'ing scripts. In addition, there's an option to avoid JIT'ing small scripts to conserve memory.

If you post a snippet of the plugin log since the last startup cycle of one of the cases where you were running out of memory, it may be possible to help with tuning these parameters appropriately.

[luna]

We are running a 64-bit O/S.
So my assumption then is probably wrong that we phantom crashed due to breaching the 3GB limit if it can go to 4GB.

Let me continue to monitor things then over the next week or so and see if and how many times we crash.
It sounds like I'm freaking out over nothing about the memory usage.

I'm running the latest xp_bugfix as well but I think we set out DB Object count to 8196.
So it went back to using a lot of memory on that front.


This is how we have things set right now.
It's very conservative.
If we don't get any crashes in the next day or two I'll ramp up the MinScript size so it compiles more of our scripts.

Thanks



[2012-01-19 08:56:18] ServerNWScript plugin - built Dec 5 2011 23:05:03
This release is paired with game build 1765.
[2012-01-19 08:56:18] Plugin starting up.
[2012-01-19 08:56:18] DebugLevel set to 1.
[2012-01-19 08:56:18] UseReferenceVM set to 0.
[2012-01-19 08:56:18] MinFreeMemoryToJIT set to 402653184.
[2012-01-19 08:56:18] MinScriptSizeToJIT set to 42000.
[2012-01-19 08:56:18] OptimizeIR set to 0.
[2012-01-19 08:56:18] LoadDebugSymbols set to 1.

[Skywing]

In particular, do you have the log section listing all of the scripts that were compiled up to when the server restarted?

[luna]

Not from the first day we ran it (which is when the crash and phantom reset happened).

We had an issue with one of our scripts that got executed often so the log was pretty bloated with the error below repeating, so I cleared the log.

[Skywing]

This is probably a pre-existing bug in the script, as the server is not very good at reporting TMIs by default.

Build the script with debug symbols (*.ndb) and make sure that debug symbols loading is enabled in the plugin INI. That will give you a stack trace of which NWScript functions were active when the TMI was hit.

I believe the toolset option is 'Generate debug info' in the options screen.

(If the script was JIT'd, the TMI limit is substantially higher.)

[Skywing]

New beta release Jan 20 2012 19:38:49 available. This release contains performance improvements.

http://valera-ext.nynaeve.net/~skywing/nwn2/AuroraServerNWScript.zip

Be sure to upgrade ALL of the DLLs. There is a new configuration file option, OptimizeActionServiceHandlers. You can manually add it to your configuration files if desired, although the option is on by default.

Changes:

- Added support for specially optimizing certain engine functions ("action service handlers") by replacing them entirely with versions internal to the JIT backend instead of calling the game's versions. This optimization improves performance by avoiding thunking costs between the game's functions and the JIT'd code. Most actions do not have significant thunking costs, but string manipulation functions are a worst case scenario that this release improves upon.

The following engine functions are now handled internally if OptimizeActionServiceHandlers is set to 1. They will see higher performance with this release; performance improvements are most likely to be seen in code with heavy string processing, such as string tokenization code. The change is only effective if you are using the JIT backend.

* GetStringLength
* GetStringLeft
* GetStringRight
* IntToString
* GetSubString

Future releases may provide accelerated implementation of other engine functions if performance analysis shows benefits.

New configuration directive text:

;
; Certain action service handler calls, otherwise known as nwscript.nss engine
; functions, can be replaced with optimized versions when the JIT backend is
; active. The optimized versions provide equivalent functionality but with
; faster performance. For example, some string functions such as GetStringLeft
; can be accelerated in this fashion.
;
; The optimized action service handlers, however, will not log their arguments
; if the debug level is increased. For debugging, it may be advantageous to
; disable optimized action service handlers if it is desired to trace all of
; the action service calls made by a script.
;
; Default is 1 (substitute built-in implementations for certain action service
; handler calls).
;

OptimizeActionServiceHandlers=1

[luna]

I will try to get this deployed tomorrow skywing.
Still no crashes after that first day we deployed release 6.
The server is also running a lot smoother when we reach our player capacity to. This appears to be working great.

Thanks

[luna]

Skywing, we have had a few reports of an incident where the in game clock just started going really fast in the sense time just started going really fast.
Peopel's buffs came off to (whcih is how I think they noticed it).

We had not seen this type of problem before the script accelerator was applied.

Is this something you think can be caused by the script accelerator?

Here is a thread on our forum to if you have any doubts.
http://www.bgtscc1.com/forum/viewtopic.php?f=25&t=23528

It had been reported a few times besides this as well.
But just as an example the first time it was reported it was so vague we thought maybe a DM advanced the time using DMFI.

Anyhow, any thoughts?

Thanks

[Skywing]

The plugin shouldn't be able to cause this as it does not change how timekeeping works in the game.

If you are hosting your PW in a VirtualBox-based VM, that may be the culprit. There are serious flaws in VirtualBox's timekeeping handling that will result in erratic behavior of nwn2server when running NWN2 in a VirtualBox VM.

Hyper-V and VMware both have functional timekeeping support, unlike VirtualBox.