NWNX ODBC2 object storage engine

[Papillon]

I am very happy to announce an exciting new feature and a premiere for Neverwinter Nights SQL Database connectivity.

Based on Earandel's research and demonstration module, the newest ODBC2 module offers what we have been waiting for: object storage with NWNX2 ! Finally the last domain where ODBC connectivity was lacking compared to the Bioware database has been significantly improved.

What is currently available for download (nwnx_odbc2-scorco_089.zip) is an alpha release, based on Jeroen's combined mysql and odbc module. The package includes the plugin and an updated aps_demo NWN module that demonstrates the new features.

Requirements:
1) MySQL database server. ODBC supports object storage as well, but you might need to tweak the create table statements in aps_demo.
2) NWNX Version 2.5.4 (important! it will not work with earlier version)

Installation:
1) Install NWNX 2.5.4 if you do not have it already
2) Put the plugin (nwnx_odbc.dll) into your NWN directory
3) Choose your connection method and copy the corresponding .ini file into your NWN directory.
4) Rename the .ini file to nwnx_odbc.ini and edit the connection parameters

Note: Since this is an alpha release, the API for object storage might still change a little. The complete functionality is there, though, and in my (limited) tests the module proved to be stable. The module has been tested with 1.64 and 1.65.
_________________
Papillon

[Papillon]

The current performance of the NWNX object storage engine is as following:

283 scrolls in a chest, table pwobjdata which is like pwdata, just with a blob column.

MySQL Data path:
283x SetPersistentObject(scroll) = 234ms
Loop over the resultset, creating 283x scrolls = 297ms

ODBC Data path:
283x SetPersistentObject(scroll) = 266ms
Loop over the resultset, creating 283x scrolls = 330ms
Size on disk: 513KB (using MyISAM storage engine)

Bioware DB:
283x StoreCampaignObject(scroll) = 625ms
283x RetrieveCampaignObject() = 313ms
Size on disk: 2,20 MB

No storage: (just the function calls without any data written)
283x StoreCampaignObject(scroll) = 125ms
283x RetrieveCampaignObject() = 0ms (nothing created)

Specs: Athlon 3200+, local MySQL 4.1.7-nt, local NWN Client (reduced priority), and local NWN Server.
_________________
Papillon

[Makzimia De Graf]

WOOT! way to go guys! now, I need to point addicted2rpg here and say look!!
_________________
Makzimia De Graf

DM/Creator Island of Fredian
fredian.game-host.org:5123
Forums at http://castille.us/fredian/Forums

[Papillon]

I would like to know what you think about this new feature !
Please post your opinions, experiences, and ideas below.
_________________
Papillon

[Liberty Valance]

so wait... i can store objects in the database now? so i could make an "item bank", like in World of Warcraft, where people can drop off a spare suit of armor that they want for later, but is currently just encumbering them (for a fee of course), or a couple of tradeskill items they've hauled in?

*drools*
_________________
A Persistent World West2 | APWW2 Forums
Wirehead Studios

[Papillon]

Yes. Cool, eh ?

The items do not have to be in the palette either - they could have been brought in by a DM.
_________________
Papillon

[Liberty Valance]

glorious!

and i'm assuming any unique properties of the item (stored local variables, other things like that) also get saved properly?

you guys rock
_________________
A Persistent World West2 | APWW2 Forums
Wirehead Studios

[Manuel]

[Blacksting]

Since we don't have the source code to look at I would like to ask a question for future functionality. Would it be easy to integrate "modifications" to the memo data being passed back and forth. I do not have a lot of experience with blob/memo data but from what I have read its not an easy nut to crack. How easy would it be for NWNX to modify the blob data before passing it back to RCO?

Right now I use letoscript to crack open a saved SCO file and modify data. That requires SCO, leto opening and resaving, and RCO (thus being able to rename creatures and items on the fly, etc.) It seems to me that now that NWNX has all the data in its paws already that it would be many times more efficient to just be able to edit that data.

In effect a CopyAndModify function could be written that does an SCO-modify-RCO routine without bothering with the DB write. What do you think? Is this kind of thing possible? Leto has been my holy grail and will still be necessary.... but this would be a very nice kludge reduction.

[Senalaya]

I'd say, the right person to get involved with that, would be dragonsong. He has already delivered a nice tool for modding GFF data with nwnx_leto and the 'blob' that SCO/RCO actually handle is nothing else than a GFF file (BIC for creatures and UTI for items).

[Blacksting]

Dragonsong hates C++ ... but Leto is open source so translating the GFF from Delphi won't be too much of a problem. The question I really want to know is.... is the blob as its passed through NWNX able to be modified. Doing the GFF modifications I know is possible.... and I think I am answering my own question here.... since the RCO data returned is valid it is just a uti/utc/bic file... which can be modified before its returned.

The infusion of Leto in to my crafting system has made it limited only to the limits of NWN but it has made me concerned about server efficiency with all of the SCO RCO. If modifications can be done with this idea without even the need for the DB write... then my efficiency worries will be no more..... and may even be more efficient than the official and very limited CopyAndModify which seems to be extremely unkind to the engine.

[Papillon]

The source will be released as soon the the scorco branch is merged back into the main odbc2 source, which will likely happen within the next week.

Blacksting: Indeed, you answered your own question. Since we have two functions in ODBC2, that have a pointer to the GFF data in memory, everything that leto can do on files could also be done on the pointer.
_________________
Papillon

[Senalaya]

A little question:

Do I asume corectly, that if the database and the variable name are NOT set to "-", SCO and RCO work just as usual (Bio DB)?

[Blacksting]

It looks like from the APS code that it is just the database being named "-" as there seems to be a few options that the variable name can have (like "ResumeFetch" or something like that I saw in the demo.)

[Senalaya]

That would be fine, too. As long as I can still use SCO/RCO to work with the Bio DBs, currently holding the data.