Passing c++ methods/pointers to C# Managed code

[Terra_777]

Well, since you didnt look at the stuff I linked I'll copy it over here:

In C++

[Baaleos]

Hi Terra,
I did read and examine the code on that page, but the reason I asked more questions was because I am not a c++ developer and I don't understand much of the syntax.

For instance - why provide arguments as void?

In the c++ code you are saying

[Terra_777]

Arugments are provided as void pointers because they can be anything. Void* is a DWORD holding the address to the value you're passing between C++ to C#. Its meant to be generic.

So say you passed an integer instead of a string/char* then the cast would look like this:

int myint = *(int*)someParam; <- if its a pointer to an integer in memory
int myint = (int)someParam; <- if the actual void* is the integer as void* and int happens to have the same size.

casting an int into a void would look like this:

void * myvoidptr = &myint; <- its now pointing towards the address of myint.
void * myvoidptr = (void*)myint; <- myvoidptr is now the value of int.

If you already know about pointers sorry about that. Good programming practice however states that you should NEVER store anything but addresses in void pointers and when they aren't valid anymore they should be set to NULL. So the first examples in the two different casts is "correct".

The biggest difference between C# and C++ is that C# is managed memory while C++ is unmanaged. Meaning C# garbage collects and you don't have to care about memory at all while C++ you have to tear down what you build up.

As for how the class in C++ is defined, i'm not entirely sure but I assume its just a wrapper object for C# to live in so it should probably be defined as a global or dynamically allocated so it doesn't go away. The structure should probably mimic the class that C# implements but I don't really know if thats the case.

I'm gonna go teach myself some C# in the coming days so I'll see if I can shed more light on it when my C# knowledge is more then "This looks like java and I hate java".
_________________
I dun have any signature, I'm happy anyway.

[Baaleos]

I did it in a slightly different way than you suggested.
I followed the guide at this site
http://blogs.msdn.com/b/jonathanswift/archive/2006/10/03/dynamically-calling-an-unmanaged-dll-from-.net-_2800_c_23002900_.aspx

First I made my method declared as extern (dll exporting)

Then in my C# Dll that is already loaded in nwnx through a nwnx_dotnet plugin I am able to use

[Baaleos]

Managed to create 2 new exported c++ methods

get/setData methods

Basically these methods will set or retrieve a char* (String) variable in the nwnx_cool context/workspace from the c# code.

The idea is that when the C# Code runs, now that its able to RunScripts - it makes sense to allow the C# code to provide arguments of some sort incase the script needs it.

So I've made it so the C# code can SetData to pass a string to the nwnx cool function, which is stored:
Then when a GetLocal on COOL!61 is called, it returns that string variable to the nwnserver, allowing the value to be accessible in nwscript.

One interesting thing to note - and im sure this is probably common knowledge to c++ developers.
But String variables in C# are not as compatible as they should be with c++, I don't believe they are treated as primitives such as int etc.
When passing a String from the c# domain to the c++, you need to convert it to bytes first - otherwise you get text such as ' .,,,,!¬' appearing.
C++ then interprets the byte information as the characters intended.

[Baaleos]

So, my code is working, although, I am seeing frequent crashes if I spam RunScript calls.
Surely if the function was implemented correctly, it should be Synchronous, and without error?

Im getting messages about corrupt memory etc.
I suspect it is around the Set/Get Data section of code.

@Terra - do you have any suggestions on better ways to pass data from the .net to the c++ environment?

the way I was doing it was having a single variable ( char* ) in the c++ nwnx_cool environment, that could have its value updated via SetData
A method that I exported to make available in the .net environment.

This works, but I guess the SetData calls must be overwriting data while It is still being used.

Because the .Net code is running on multiple threads (I've got it hooked up to a TCPListener):
Maybe I need to put in some sync functionality, to make sure that only one request is ever being dealt with at a time.
Eg: set an int to 1 while dealing with request, then set to 0 when done.
Then wait until that int is 0, before actually doing the SetData/RunScript calls

Thoughts?

[Terra_777]

Maybe is a threading issue? nwn is single threaded so calling RunScript from thread that isnt the main thread will most likely cause some serious issues.

You're going to have to establish a critical section if thats the case. IE:

C#: Set string to be ran and wait until the string var is empty.
C++: check if a string is set, run the script and empty the string var.

As for C++ you're going to have to hook the mainloop or a place where it runs every frame (or very often) for your critical exchange.
_________________
I dun have any signature, I'm happy anyway.

[Baaleos]

Not too keen on the idea of hooking mainloop - the nwnxbridge did that, and it caused similar errors.

The RunScript call itself seems to be working fine - leading me to think that multi-threading might be possible with the right architecture/planning in place.
I think its the SetData method that is making it fall over - which makes sense, the error is complaining about corrupt memory - its possible that the new call to SetData is overwriting data that was still being used from the previous thread running the RunScript for the previous request.

SetData and RunScript are always called in the same c# method.
So they both share a thread. So I just need to make that thread block subsequent requests, until they have returned successfully.

That way, nwnx is only dealing with one external request at anyone time, and is able to feed it into nwnserver in single file order.

I will experiment with it tonight, I've got my heart set on getting a threading model working.
This is more along the lines of thread injection, injecting from an external thread, into the nwnserver which resides on its own thread - so for all I know, it will be fraught with problems - but I like a challenge.

[Terra_777]

There are no critical sections around the sections that are critical in nwn. Meaning if there are two threads and one thread is fetching say localdata whilst the other is deleting, it'll kill itself because the array is getting resized and or even moved as it happens. Or if a script executes while you got a thread in runscript already, the virtual machine registers are going to get overwritten and it'll most definitely crash.

You can run as many threads as you like in nwns process but only the main thread should ever call nwns routines is what I'm getting at, because its a singlethreaded paradigm.

http://msdn.microsoft.com/en-us/library/windows/desktop/ms682530%28v=vs.85%29.aspx

Also nwnx bridge or whatever is a very bad example on that because its not a proper critical section either, its just me being dumb.
_________________
I dun have any signature, I'm happy anyway.

[Baaleos]

Hi Terra,
I've been thinking about what you said about Critical Section etc
and I was hoping to poke your brains about it.


I managed to eliminate the crashes caused by the injecting from one thread into the server process, by surrounding the offending code with c++ try and catch statements.

While hardly a fix, it does keep the server stable enough that it doesnt do a hard crash.


What I have noticed however, it that occasionally script errors will fire (STACK UNDERFLOW ETC), with the tags of seemingly random objects in the gameworld being reported as the offender - these typically coincide with me injecting into the gameworld.

eg: in c# I create a server class that is spawned from the nwnserver process via the nwnx_dotnet plugin.
This c# class has the ability to use the exported RunScript method from nwnx_cool.

Because this server exists on a thread outside of the Mainloop, I can assume that the reason random tags and script errors happen when I perform a shout from my website, is that as you said, the registers are in a state of constant flux, on the main thread.

Eg: ECX, EBX and so on are all shifting, while my secondary thread is trying to perform a RunScript command.

So - my question to you is this:

Can the mainloop be suspended, while my secondary thread is processing? (Allows me to keep my coding Synchronous)
Or
Should I pass the payload data to the mainloop, and have it perform the task at hand? (but then this gets more complex, as I need to make the mainLoop pass the return back to the caller - its almost like coding for asynchronous)

It occurs to me that the second choice would be the ideal one, while harder to do, but I was thinking that if the issue is merely that mainloop is doing things while my second thread is working, surely we could stop the mainloop from working temporarily, until the second thread has finished its task?

eg:

[Terra_777]

I wouldn't touch the mainthread at all. You're going to have to hook someplace. You don't want any other thread then the mainthread doing work on the nwn instructions. Only the mainthread should execute nwserver methods.

The errors you're seeing is likely the workerthread from C# modifying virtual machine values so the stack corrupts and the script ends.

I wrote you an example on how to do the exhange: http://pastebin.com/JW1nHDmc its prettymuch all done for you, all you have to do is the mainloop thread and the warper which are only demonstrated here.
_________________
I dun have any signature, I'm happy anyway.

[Baaleos]

Hey Terra,
Sorry for necro post-
Just wanted to update you with what I have done so far with regards to getting External triggering of internal nwn methods.



I am using something called 'WhiteMagic' - a .Net dll that allows the hooking of managed and unmanaged methods.

I kick this off with nwnx_dotnet (or rather my own flavor of it)

https://github.com/Baaleos/NWNDotNet/blob/master/Hooks/MainLoop.cs



So I am hooking MainLoop to process incoming messages - but its hopfully a little more robust than checking for messages every iteration of the loop.


The way I do it is via a TCPListener - much like your bridge.
When the bridge gets a message, it adds the message to an 'External List' of messages.
When the MainLoop is finished running, it moves any items in the ExternalList to the InternalList (its my way of avoiding 'Collection changed' exceptions)

When the MainLoop next runs, it then takes the contents of the Internal List, and acts on each item in the list in a switch/case statement.

Simple implementation I have done so far is just making the server 'shout' messages that I type into my admin console on my website.

But the good news is that I am not seeing any of the cross contamination I was seeing before, where it was causing stack underflow in other scripts etc

It seems by passing the workload to the MainLoop has solved the contamination issues, and so far no crashes.