logo logo

 Back to main page

The NWNX Community Forum

 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
 
nwnx_jvm - native JVM integration
Goto page 1, 2, 3, 4, 5, 6, 7, 8, 9  Next
 
Post new topic   Reply to topic    nwnx.org Forum Index -> Linux development
View previous topic :: View next topic  
Author Message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Mon Dec 07, 2009 16:56    Post subject: nwnx_jvm - native JVM integration Reply with quote

Hi,

another side project of mine for those looking to embed java directly in nwserver. Anyone wanting to give it a try is welcome to do so.

It provides the usual goodies:
- direct NWScript->JVM event calls
- direct NWScript calls from inside the JVM (benchmarked to be around the many millions per second, virtually the same as native nwscript)
- all features and libraries available for the JVM

Note: This is beta and there are a few things left to do to get it production-ready - though I have OSGi R4 running quite successfully on it.

This not implemented yet (due to lack of time and nothing else):
* transparent configuration of which NW* classes to use
* common NWN constants as enum types to tighten up the API
* a great README

Quickstart:
- put the .so in your nwserver directory
- you need to prefix LD_LIBRARY_PATH=/path/to/libjvm.so to your nwserver start script, where the LD_PRELOAD to nwnx2.so is too.
- Note: do NOT use openoverride.so together with nwnx_jvm.
- edit nwnx2.ini to include the given example file
- include the provided .jar file in your classpath (see nwnx2.ini)
- write some code, compile it, put it in your class path and set your init/event/token listeners in nwnx2.ini accordingly, or use the included TestRunner class to see things in action.
- Note: a sample tokenlistener is shipped with the bundled .jar file, you do not need to implement it yourself.
- Also, there are example .java source files in the same directory which outline the API required.
- Add inc_jvm.nss to your module, and place some JVM_EVENT("name") in event scripts throughout your module. Each call done such will yield context do nwnx_jvm (and call event(..) in your configured eventListener).
- add jvm_token.nss to your module and compile it. It is used to change OBJECT_SELF for Assign*() and does not otherwise need configuring or tweaking by you.

Other pointers:
- AssignCommand() is not implemented natively; instead, a round-trip is made to nwscript to change OBJECT_SELF and a stored anonymous function is executed, containing your code. The provided org.nwnx.nwnx2.jvm.Scheduler class has helpers to do that for you (Yes, Java is missing closures ..). This can be hidden inside a more abstract API later on, though. The scheduler is intelligent enough to group assigns per object and execute them all at once.
Code:

Scheduler.assign(objSelf, new Anon() {
   public void e() {
      NWScript.speakString("hi", 0);
   }
});


Sources of the provided Java classes are packaged inside the .jar.

Direct questions as usual to this thread or directly to me.

Have fun. :)

===========

Most up-to-date download information:

You can get a precompiled binary .so and all needed other files from the CI project linked as a sticky in this forum. Sources are in svn, in the core-2.8 branch. The given binary will run on any modern Linux and requires a recent JVM installed. 32bit, of course.


Last edited by elven on Tue May 01, 2012 23:23; edited 5 times in total
Back to top
View user's profile Send private message
PlasmaJohn



Joined: 04 Mar 2005
Posts: 70
Location: The Garage

PostPosted: Tue Dec 08, 2009 21:00    Post subject: Reply with quote

Shocked ... as if I already didn't have enough toys to play with ...
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Tue Dec 08, 2009 21:46    Post subject: Reply with quote

The cool thing is, with OSGi and your module code properly encapsulated, you can load/unload modules runtime and on-the-fly without effecting other systems. With some cool sco/rco wrapping (and resman, and nwnx_areas), you can even embed related resource files inside OSGi bundles instead of having them in the module itself.
And embed a webserver that displays a live map with players, or a chat-interface, or a place to organise ingame events. Or a RMI graphical interface that can directly inspect and invoke methods on ingame objects. Or anything, really. :)
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Fri Dec 11, 2009 14:02    Post subject: Reply with quote

The sources are available in svn.

You will need ruby1.8.x, a recent JDK, and a recent NWNX2 svn to build. You will also tweak the Makefile for your JVM pathes - I can't be bothered to wrestle with autoconf and integrate properly.


Last edited by elven on Mon Jul 11, 2011 19:42; edited 1 time in total
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Fri Dec 11, 2009 17:15    Post subject: Reply with quote

New Features:

- renamed to nwnx_jvm
- added wrappers for getFirst/getNext stuff, do not export those anymore
- simple TestRunner class to test nwnx_jvm without having to write some code
- various bugfixes
- updated first post


Last edited by elven on Tue May 01, 2012 23:18; edited 1 time in total
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Sun Dec 13, 2009 12:37    Post subject: Reply with quote

Updates! Again!

- fix not having nwn context in NWObject.apply() while yielding a event to the JVM
- fix TestRunner having no .class file
- NWLocation NWVector are case classes with final local fields now (= .equals works)
- a handler in NWObject that can be used to register own classes to map NWObjects onto a object hierarchy (this will be documented in detail later on)

Please note that the last item requires you to call NWObject.registerDefaultHandler(); in your eventHandler init() method or it will throw an Exception "can't find object class" or some such. I have updated the example code to reflect this.


Last edited by elven on Tue May 01, 2012 23:18; edited 1 time in total
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Mon Dec 14, 2009 13:29    Post subject: Reply with quote

Bugfixes:

- all iterator-collector functions now work as advertised (GetPCs, GetObjectsInArea, etc)
- NWObject.oid is now a int instead of long
- NWObject now case class (overrides equals to match oid() == oid())
- NWEffect/NWItemProperty should not mismatch anymore
- JNI exceptions are more verbose in their error text

Changed files at the usual location: the .so, the .jar
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Mon Dec 14, 2009 14:57    Post subject: Reply with quote

Changes:

- iterator-collectors are now more efficient
- the .jar now contains pre-generated class files for all constants, grouped by prefix.

.so and .jar updated.
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Tue Dec 15, 2009 20:48    Post subject: Reply with quote

- removed getModule; use NWObject.MODULE instead (faster)
- added NWObject.INVALID constant
- Scheduler.delay() now takes a long in milliseconds, as does NWObject.delay()
- NWScript and Scheduler are now singletons

Updated files: jar, .so
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Thu Dec 17, 2009 23:12    Post subject: Reply with quote

- fixed jarfile packaging
- don't export badBadReplaceMeThisDoesNothing()
- re-add @Override annots for core classes
- passing in null for a NWObject parameter no longer throws NullPointerException, it gets instead handled as OBJECT_INVALID
- NWObject.oid now has a ceiling of 0x7fffffff
- NWLocation, NWVector implement Serializable
- NWVector.ORIGIN points at (0, 0, 0)
- Scheduler has a SchedulerListener interface to act on token()
- NWObject has a ApplyListener interface to act on successful applies
- NWObject.classListener is now a sorted List instead of a Set (booo!)
- NWScript fun header generated with proper @param tags

jar and so updated at the usual place.
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Sat Dec 19, 2009 15:41    Post subject: Reply with quote

Just java changes for now:

- ApplyListeners receive newObject()
- ApplyListeners send out preApply()
- SchedulerListener sends out missedToken(), postFlushQueues()
- NWItemProperty, NWEffect now have classHandlers too
- Evaluation for all classHandlers is in REVERSED order now (core classes registered first, then all other matching ones)
- generated constants: don't miss constants with // comments at the end
- generated constants: generate static class files for constants down to 3 (previous: 8) - now DurationType et al included
- NWObject.registerDefaultHandlers() is gone now


Last edited by elven on Tue May 01, 2012 23:18; edited 1 time in total
Back to top
View user's profile Send private message
motu99



Joined: 24 Sep 2007
Posts: 16

PostPosted: Thu Dec 31, 2009 13:52    Post subject: Reply with quote

Nice plugin.

So basically we have all of NWScript's basic functionality at our fingertips?! (e.g. all functions in nwscript.nss; plus some nice additions by yourself, such as get* functions that return arrays - like NWEffect[] getEffects)

This means that from now on we can do everything that NWScript can do (although often very awkwardly and inefficient, because NWScript doesn't even have arrays, not to mention references) in java!

Mmh.

On the one hand that is great news, because I much prefer programming in an object oriented language - not necessarily Java, C# are C++ are great as well, although not as portable. What bothered me alot with NWScript is the lack of references and arrays. And then, for the larger projects, the (pre) C age #include "management" (or rather no management) becomes quite cumbersome. So basically, with nwnx_jvm I could write any complicated nwscript in java?! The ideal situation being the replacement of all code in all nwscript main() functions by one single call (including the passing of all necessary parameters) to the JVM.

On the other hand, replacing virtually all nwscript code with Java code just for a few scripts doesn't seem worth the effort. Meaning the JVN plugin would be of more interest to the really large projects. But here is the catch: If your project relies on a lot of includes (not only Bioware's, but a bunch of custom made includes), the JVM wouldn't know about these, meaning all includes would have to be ported to Java. Doable, but quite a hassle for a large existing project. For a new project this wouldn't be such a bother, although so far the functions provided by the Bioware includes haven't yet found their way into your NWScript.java implementation...

Bottom line. I am intrigued, but not yet fully convinced.

Just one question: Can I run the other plugin-functions (such as acaos nwnx_func) directly from the JVM, or would I have to go through the usual NWScript.setLocalString(Object.MODULE, "NWNX!****!****") routine?
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Thu Dec 31, 2009 13:59    Post subject: Reply with quote

motu99 wrote:
Bottom line. I am intrigued, but not yet fully convinced.


Was there a question? Because I think you summed it up quite well.

motu99 wrote:
Just one question: Can I run the other plugin-functions (such as acaos nwnx_func) directly from the JVM, or would I have to go through the usual NWScript.setLocalString(Object.MODULE, "NWNX!****!****") routine?


Running things directly from the JVM actually IS calling SetLocalString (which never touches the NWScript VM, since the indirection hooking NWNX!FUNCS! catches it. Integrating it into nwnxjvm itself is not feasible at the moment (though there are plans to unify a plugin API to make such a thing possible).

As to the complexity "problem": nwnxjvm isn't meant to support a rewrite of all script functionality, and I certainly agree that it makes the most sense for new projects. It is the big guns to implement in-server web interfaces and powerful AI, and whatnot.

Nothing though prevents you from writing small scripts in Java too, though, and it can be quite the pleasure to do it. In fact, I am working on a OSGi implementation (it runs, the API just isn't finished yet) where you can write small (or large) features/functionality facets (together with all neccessary GFF resources like items) in pure Java and pack them up into a OSGi bundle and load/unload it at runtime. Only sporadic updates and no support yet whatsoever, sorry.
Back to top
View user's profile Send private message
motu99



Joined: 24 Sep 2007
Posts: 16

PostPosted: Thu Dec 31, 2009 16:20    Post subject: Reply with quote

elven wrote:

Nothing though prevents you from writing small scripts in Java too, though, and it can be quite the pleasure to do it.


That's right. I have already been thinking about it. There are some scripts that cry out for a proper programming language with decent array (or more general: collection) handling capabilities. I already have a few scripts in mind; I was just wondering, how many of the includes would have to be rewritten.

For a large project, using nwnx_jvm would be beneficial in various ways:
- no code duplication in the various nwscript main(), considerably freeing up server memory [The PRCs compiled nwscript code has roughly 200 MB nwscript bytecode; 300 MB before they separated the pretty complex PreSpellCastCode out into a separate script now called via ExecuteScript(); twas inlined into every spell before]
- for complex scripts the JVM would probably be quite a bit faster for the "normal" (non-nwscript) code, e.g. "mundane" variable/array manipulation etc. - whereas it would be (almost) as fast on the "native" NWScript-calls. All in all I would expect a considerable gain in performance for any task that requires complex data manipulation (involving arrays and collections)
- for complex projects all of the benefits of a full fledged Java IDE (such as Eclipse or Netbeans) can be used

Maybe there are more advantages. For a new project the advantages clearly outway the hassle, in particular if nwnx has to be set up anyway (I can't imagine doing anything sophisticated without the nwnx_funcs, nwnx_structs and nwnx_events plugins).

Another question:
Do
- AssignCommand()
- DelayCommand()
- ActionDoCommand()
work from the JVM? I read that there were some problems (although solved now) in nwnx_ruby.

And a last question: Is there support for GFF-editing in nwnx_jvm? I know that the PRC has some java-based tools that can edit virtually all of the nwn files.
Back to top
View user's profile Send private message
elven



Joined: 28 Jul 2006
Posts: 259
Location: Germany

PostPosted: Thu Dec 31, 2009 17:04    Post subject: Reply with quote

Just to put this into context: You can actually attach Eclipse with a running remote debugger to the JVM inside nwserver, set breakpoints, and hotpatch running code. You have ALL of the Java world at your disposal, including full native multithreading, all available libraries and any native code you care to load. It's much more than "making scripts more concise" - but it certainly does that.

motu99 wrote:

Another question:
Do
- AssignCommand()
- DelayCommand()
- ActionDoCommand()
work from the JVM? I read that there were some problems (although solved now) in nwnx_ruby.

And a last question: Is there support for GFF-editing in nwnx_jvm? I know that the PRC has some java-based tools that can edit virtually all of the nwn files.


- AssignCommand, DelayCommand: same workaround as with nwnxruby, as described in the first post.
- ActionDoCommand: superfluous, not implemented (same as assign + closure with nwnxjvm)

There are GFF libraries, yes. (google for modpacker, pspeed)
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    nwnx.org Forum Index -> Linux development All times are GMT + 2 Hours
Goto page 1, 2, 3, 4, 5, 6, 7, 8, 9  Next
Page 1 of 9

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group