| View previous topic :: View next topic |
| Author |
Message |
Grinning Fool
Joined: 12 Feb 2005
Posts: 264
|
 Posted: Tue Jan 09, 2007 20:35 Post subject: Delayed write database inserts |
 |
|
Would anyone else see advantage to a plugin that accepts requests for database inserts, queues them up, and via a thread performs a 'lazy write'?
I ask this because I'm considering writing this for my own PW; since that the NWN2 server proc doesn't use the second CPU of a dual core at all, I see advantage to using a lazy write for non-critical inserts. (Non critical in this case meaning it's OK if they fail). For my use, I'll be using it for database tracking of character events and activities; and statistical data that I aggregate over time.
If this would get more widespread use, it might make sense to include the main SQL plugins. The additional consideration if this were done is that it would need its own DB connection (to avoid conflicting with the connection being used for normal DB activity).
_________________
Khalidine, a NWN2 persistent world
Looking for volunteers. |
|
| Back to top |
|
 |
Papillon
x-man
Joined: 28 Dec 2004
Posts: 1060
Location: Germany
|
| Posted: Tue Jan 09, 2007 21:59 Post subject: |
|
|
Would you support delayed SCORCO calls as well ? Since then, you would have to store not only the SQL string, but also the binary object data somewhere.
_________________
Papillon |
|
| Back to top |
|
|
Grinning Fool
Joined: 12 Feb 2005
Posts: 264
|
| Posted: Tue Jan 09, 2007 22:08 Post subject: |
|
|
If you'd be so kind as to explain SCORCO, then I could answer that better 
_________________
Khalidine, a NWN2 persistent world
Looking for volunteers. |
|
| Back to top |
|
|
Gryphyn
Joined: 20 Jan 2005
Posts: 431
|
| Posted: Tue Jan 09, 2007 23:35 Post subject: Re: Delayed write database inserts |
|
|
| Grinning Fool wrote: | Would anyone else see advantage to a plugin that accepts requests for database inserts, queues them up, and via a thread performs a 'lazy write'?
I ask this because I'm considering writing this for my own PW; since that the NWN2 server proc doesn't use the second CPU of a dual core at all, I see advantage to using a lazy write for non-critical inserts. (Non critical in this case meaning it's OK if they fail). For my use, I'll be using it for database tracking of character events and activities; and statistical data that I aggregate over time.
If this would get more widespread use, it might make sense to include the main SQL plugins. The additional consideration if this were done is that it would need its own DB connection (to avoid conflicting with the connection being used for normal DB activity). |
This would be a bit of a 'double-up'. Your database engine already does this with caching & transaction logging (for recovery).
However a plugin that manages a QUEUE is a good idea. (I want one)
You could queue non-critical* events to execute during 'slack' time or HOLD them until a signal** is raised. (Almost like a Service Broker).
*in this case non-critical means it doesn't matter if the info is lost on a server crash/shutdown.
** or a time-interval
Cheers
Gryphyn |
|
| Back to top |
|
|
xaltos
Joined: 03 Jun 2006
Posts: 31
Location: Germany
|
| Posted: Tue Jan 09, 2007 23:39 Post subject: |
|
|
this sounds very usefull.
We use a dual core as well and it could be a nice addon to store a lot unimportent data 
_________________
xaltos
NWN2_Audolo |
|
| Back to top |
|
|
virusman
Joined: 30 Jan 2005
Posts: 1020
Location: Russia
|
| Posted: Wed Jan 10, 2007 1:00 Post subject: |
|
|
| Consider using INSERT DELAYED. |
|
| Back to top |
|
|
Grinning Fool
Joined: 12 Feb 2005
Posts: 264
|
| Posted: Wed Jan 10, 2007 3:39 Post subject: |
|
|
insert delayed doesn't work with innodb, unfortunately.
_________________
Khalidine, a NWN2 persistent world
Looking for volunteers. |
|
| Back to top |
|
|
Papillon
x-man
Joined: 28 Dec 2004
Posts: 1060
Location: Germany
|
| Posted: Wed Jan 10, 2007 12:38 Post subject: |
|
|
SCORCO is short for StoreCampaignObject and RestoreCampaignObject.
_________________
Papillon |
|
| Back to top |
|
|
Grinning Fool
Joined: 12 Feb 2005
Posts: 264
|
| Posted: Wed Jan 10, 2007 23:49 Post subject: |
|
|
Hm - well, if we did a memcpy of the object it should be possible to do so - and I see a significant performance advantage for things like persistent storage if we did.
Is that a dedicated function in the plugin? Or is it just another SQL call? (I don't have access to the source from here.)
_________________
Khalidine, a NWN2 persistent world
Looking for volunteers. |
|
| Back to top |
|
|
virusman
Joined: 30 Jan 2005
Posts: 1020
Location: Russia
|
| Posted: Thu Jan 11, 2007 0:45 Post subject: |
|
|
| It's a hook to nwserver functions. |
|
| Back to top |
|
|
Grinning Fool
Joined: 12 Feb 2005
Posts: 264
|
| Posted: Thu Jan 11, 2007 23:56 Post subject: |
|
|
In that case, I don't see any reason it couldn't be supported, as long as a true copy of the object data was made.
_________________
Khalidine, a NWN2 persistent world
Looking for volunteers. |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
