JIT compiler for NWScript

[Skywing]

Hi,

The latest public_nwn2dev snapshot I've put up at http://www.nynaeve.net/Skywing/nwn2/nwn2dev/public_nwn2dev.zip includes a just-in-time compiler for NWScript. The just in time compiler takes compiled NWScript files (*.ncs) and turns them into MSIL, when is turned into native code. No modification to the *.ncs files is necessary; the JIT system is intended to be integrated with a script host.

The JIT system is structured in such a way so as to make it possible to write a NWNX plugin to patch the JIT engine into the stock nwserver/nwn2server.exe. While I have yet to start such a project (instead I have integrated the JIT system into my ground-up nwn2server replacement), all the pieces are available now.

Computationally-bound scripts show on the order of a 22x speed improvement with the MSIL JIT (versus the interpretive VM). Scripts that use mostly engine calls see more modest improvements (a typical lower bound would be 2-3x faster, with further speed improvements depending on the ratio of work done in NWScript versus work done in engine code). Note that these measurements were taken against my NWScript VM and not the stock game's VM.

In order to create a NWNX plugin to support the JIT system, the following would need to be done (should someone find this a sufficiently interesting project idea):

- Hook CVirtualMachine::StackPush* and CVirtualMacine::StackPop* to detour to the NWScriptStack::StackPush* and NWScriptStack::StackPop* implementations in NWNScriptLib in the public_nwn2dev drop.
- Hook CVirtualMachine::StackPopCommand_Internal to create a saved state object using NWScriptSaveState (or the NWScriptJITLib wrapper).
- Hook CVirtualMachine::RunScript to JIT code for the script (if it has not been done already) via NWScriptGenerateCode. Once JIT'd code is available, invoke NWScriptExecuteScript to run the script.
- Hook CVirtualMachine::RunScriptSituation to invoke NWScriptScriptSituation. The NWSCRIPT_JITRESUME handle that a previous call to CVirtualMachine::StackPopCommand_Internal created should be used. The best way to do this would be to stuff the handle into the script situation data returned by StackPopCommand_Internal (as we would be taking over all usage of the script situation object anyway)
- Implement INWScriptStack to call into the appropriate CVirtualMachine::StackPush*/StackPop* functions for each StackPush*/StackPop* API. This places parameters on the internal CVirtualMachine stack so that called engine functions can use them.
- Implement INWScriptActions to call the engine function dispatcher (CNWVirtualMachineCommands::ExecuteCommand).
- Implement INWScriptActions to call into CNWVirtualMachineCommands::CreateGameDefinedStructure, CNWVirtualMachineCommands::DestroyGameDefinedStructure and CNWVirtualMachineCommands::GetEqualGameDefinedStructure to manage engine structures according to the EngineStructure/INWScriptActions interface.

This architecture is designed such that scripts push data onto the game's native CVirtualMachine stack when calling engine functions. The calling convention that the JIT system uses for this is compatible with that used by the stock server.

[Nob]

Will this work using Mono under Linux?