logo logo

 Back to main page

The NWNX Community Forum

 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
 
Accent problem
Goto page Previous  1, 2, 3  Next
 
Post new topic   Reply to topic    nwnx.org Forum Index -> Development
View previous topic :: View next topic  
Author Message
Grinning Fool



Joined: 12 Feb 2005
Posts: 264
PostPosted: Fri Dec 22, 2006 23:21    Post subject: Reply with quote
Incidents like these should be sent to OE as bug reports; basically the font sets used to display information in-game are not complete.
_________________
Khalidine, a NWN2 persistent world

Looking for volunteers.
Back to top
View user's profile Send private message
amphiprion



Joined: 07 Nov 2006
Posts: 44
Location: Toulouse (France)
PostPosted: Thu Dec 28, 2006 12:10    Post subject: Reply with quote
i find a solution, if i insert data from nwn2/nwnx4 then when i read this information with nwn2/nwnx4, i can display it correctly in SendMessageToPC, in dialog, in GUI....

And if i look the database via a MySQL client, i see that accent character are encoded in the DB, i.e. the "ç" appear in the database as "ç"

Don't know if we can find a solution to have the good visualization in the database and in nwn2 at the same time. (May be an encode during the nwnx4 process ?)

For the moment i insert all my text from nwn2/nwnx4
Back to top
View user's profile Send private message
redils



Joined: 13 Jan 2005
Posts: 27
PostPosted: Fri Dec 29, 2006 0:01    Post subject: Reply with quote
Amphirion: It's look like UTF-8 caracters. So, try to reencode your db in UTF-8. Strings located into the dialog.tlk file are in UTF-8, so i guess it's the same. Didn't test it, but that could be a way of search..
Back to top
View user's profile Send private message
amphiprion



Joined: 07 Nov 2006
Posts: 44
Location: Toulouse (France)
PostPosted: Fri Dec 29, 2006 0:19    Post subject: Reply with quote
amphiprion wrote:
So, i have made many test with different character set (Latin1, Latin2,...Latin7, UTF8, and many others)
but none can display correcly the accent character in nwn2
Back to top
View user's profile Send private message
amphiprion



Joined: 07 Nov 2006
Posts: 44
Location: Toulouse (France)
PostPosted: Sun Dec 31, 2006 13:47    Post subject: Reply with quote
Here my solution while it's not fixed by nwnx itself

I have a Mysql matching table between accent character and encoded representation (can be completed for different language - i've created usefull french accent characters).

matching table:
Quote:
CREATE TABLE `util_conversion` (
`source` varchar(1) NOT NULL,
`dest` varchar(2) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

--
-- Contenu de la table `util_conversion`
--

INSERT INTO `util_conversion` (`source`, `dest`) VALUES
('é', 'é'),
('è', 'è'),
('ç', 'ç'),
('à', 'Ã '),
('ù', 'ù'),
('ë', 'ë'),
('û', 'û'),
('ê', 'ê');


the MySQL 5 stored function
Quote:

CREATE FUNCTION `EncodeNwN2`(texte VARCHAR(2000)) RETURNS varchar(2000) CHARSET utf8
BEGIN
DECLARE done INT DEFAULT 0;

declare sTmp VARCHAR(500) DEFAULT texte;
declare sSource VARCHAR(1);
declare sDest VARCHAR(2);

DECLARE cur1 CURSOR FOR SELECT source, dest FROM util_conversion;
DECLARE CONTINUE HANDLER FOR SQLSTATE '02000' SET done = 1;

open cur1;

Repeat
FETCH cur1 INTO sSource, sDest;
IF NOT done THEN
set sTmp := REPLACE(sTmp , sSource, sDest);
END IF;
until done END Repeat;

Close cur1;

RETURN sTmp;
END;



To use it for reading a data field
Quote:
Select EncodeNwN2(<field>) from <table> where ....
Back to top
View user's profile Send private message
caloup



Joined: 29 Sep 2006
Posts: 59
Location: albi (france)
PostPosted: Sun Dec 31, 2006 19:40    Post subject: problem with accent Reply with quote
text in french : alors ça marche bien ? si tu es d'accord je le mettrai dans la FAQ de la doc pour donner une première solution à ceux qui rencontrent le même problème.

text in english :

Does that work well ? So if you're agree i could integrate this in the FAQ of the documentation for nwnx4...
Back to top
View user's profile Send private message
Papillon
x-man


Joined: 28 Dec 2004
Posts: 1060
Location: Germany
PostPosted: Sun Dec 31, 2006 22:36    Post subject: Reply with quote
This solution might work, but it's highly specific: MySQL 5 only. If that's something for the FAQ, then only as a temporary fix (as amphiprion already suggested).

In my tests, I wrote "café is nice" into a string with the script editor, saved that via SQLExecuteDirect into a Latin-1 encoded table, and got exactly the same string in MySQL Query Browser (no unusual characters).

What do you get with a Latin-1 table ?
_________________
Papillon
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
amphiprion



Joined: 07 Nov 2006
Posts: 44
Location: Toulouse (France)
PostPosted: Mon Jan 01, 2007 1:10    Post subject: Reply with quote
now i'am in UTF8, but some time ago i had a MySQL latin1 DB and when i inserted "café" with MySQL client, nwn couldn't display it correctly, but at this period i haven't the idea to insert text using nwnx and see the result.

So MySQL 5.0 specific behaviour, or MySQL Client behaviour may be... but i'm certain that when i had the DB in Latin1, nwn2 couln't display correctly accent character entered manually (using a MySQL client)

PS: no problem for the FAQ, but specify that the DB charset is UTF8
Back to top
View user's profile Send private message
Gryphyn



Joined: 20 Jan 2005
Posts: 431
PostPosted: Mon Jan 01, 2007 1:32    Post subject: Reply with quote
amphiprion wrote:
Here my solution while it's not fixed by nwnx itself

I have a Mysql matching table between accent character and encoded representation (can be completed for different language - i've created usefull french accent characters).

matching table:
Quote:
CREATE TABLE `util_conversion` (
`source` varchar(1) NOT NULL,
`dest` varchar(2) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

--
-- Contenu de la table `util_conversion`
--

INSERT INTO `util_conversion` (`source`, `dest`) VALUES
('é', 'é'),
('è', 'è'),
('ç', 'ç'),
('à', 'Ã '),
('ù', 'ù'),
('ë', 'ë'),
('û', 'û'),
('ê', 'ê');


the MySQL 5 stored function
Quote:

CREATE FUNCTION `EncodeNwN2`(texte VARCHAR(2000)) RETURNS varchar(2000) CHARSET utf8
BEGIN
DECLARE done INT DEFAULT 0;

declare sTmp VARCHAR(500) DEFAULT texte;
declare sSource VARCHAR(1);
declare sDest VARCHAR(2);

DECLARE cur1 CURSOR FOR SELECT source, dest FROM util_conversion;
DECLARE CONTINUE HANDLER FOR SQLSTATE '02000' SET done = 1;

open cur1;

Repeat
FETCH cur1 INTO sSource, sDest;
IF NOT done THEN
set sTmp := REPLACE(sTmp , sSource, sDest);
END IF;
until done END Repeat;

Close cur1;

RETURN sTmp;
END;



To use it for reading a data field
Quote:
Select EncodeNwN2(<field>) from <table> where ....


MySQL already has this functionality...
CONVERT(<your string> USING utf8)
CAST(<your string> AS <type> CHARACTER SET utf8);

More detail can be found at
http://dev.mysql.com/doc/refman/5.1/en/charset.html
*NB: Chap 10 of help is available in other languages as well.

from what I've read changing CHARSET=latin1 to CHARSET=utf8
on table creation should solve the issue (with the following)

But the most important snippet is:

Character set issues affect data storage, but also communication between client programs and the MySQL server. If you want the client program to communicate with the server using a character set different from the default, you'll need to indicate which one. For example, to use the utf8 Unicode character set, issue these statements after connecting to the server:

SET NAMES 'utf8';
SET CHARACTER_SET utf8;

To change the connection charset permanently to UTF-8, add the following line in the [mysqld] section:
[mysqld]
init-connect='SET NAMES utf8'

Cheers
Gryphyn
Back to top
View user's profile Send private message
amphiprion



Joined: 07 Nov 2006
Posts: 44
Location: Toulouse (France)
PostPosted: Mon Jan 01, 2007 2:12    Post subject: Reply with quote
I try many many test after this post.

Set the DB to UTF8, the TABLE to UTF8, the field to UTF8, the ini file to UTF8, i try also to set all to Latin1, i try mixed configuration

none of them works to display correctly accent in nwn2, without using my stored function

i tried also
CONVERT(<your string> USING utf8)
CAST(<your string> AS <type> CHARACTER SET utf8);
to avoid using my stored function, but fix the accent bug.

If someone can display correctly accent data entered using a MySQL client and can display it correctly using SendMessageToPC, please let me know your configuration (database charset, table, field, ini)

Thanks alot
Back to top
View user's profile Send private message
caloup



Joined: 29 Sep 2006
Posts: 59
Location: albi (france)
PostPosted: Mon Jan 01, 2007 9:44    Post subject: Reply with quote
I'll try to make a test tomorrow and give you my answer Wink

Papillon, café is the only word n english that take an accent originally... could you try with another word ? What is your config ?
Back to top
View user's profile Send private message
Papillon
x-man


Joined: 28 Dec 2004
Posts: 1060
Location: Germany
PostPosted: Mon Jan 01, 2007 16:10    Post subject: Reply with quote
I haven't tested SendMessageToPC, though. Just wrote the string to the log file.

My comment was not aiming at providing a solution, just pointing out the fact that I do not get any special characters in my Latin-1 table...
_________________
Papillon
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
caloup



Joined: 29 Sep 2006
Posts: 59
Location: albi (france)
PostPosted: Tue Jan 02, 2007 21:32    Post subject: tested Reply with quote
I have tested and have no problem !

My test scripts :

implement a variable in a dialog box :

Quote:

/*Script de test de localstring avec accents*/

#include "nwnx_sql"

void main()
{
object oPlayer = GetPCSpeaker();
SetPersistentString(oPlayer,"test","tombé café bête lève à où noël");
SendMessageToPC(oPlayer,"message stocké");

}


next i try to show the variable in SendMessageToPC :
Quote:

/*Script de test de localstring avec accents*/

#include "nwnx_sql"

void main()
{
object oPlayer = GetPCSpeaker();
string sMess = GetPersistentString(oPlayer,"test");
if (sMess =="")
SendMessageToPC(oPlayer,"ne marche pas");
else
SendMessageToPC(oPlayer,sMess);

}


I have an Intel CORE 2 DUO 6400 + 2 Go DDRAM2 + french windows XP familly SP2

I use MySQL version 5.0 and NWNX4 version 1.06 (the last version)
+ NWN2 version 1.03

I use latin1 table...

So no problem for me...
Back to top
View user's profile Send private message
amphiprion



Joined: 07 Nov 2006
Posts: 44
Location: Toulouse (France)
PostPosted: Wed Jan 03, 2007 10:44    Post subject: Reply with quote
Like i said previously, the problem occurs for me (and others) ONLY if i insert data using a MySQL client (phpmyadmin, etc...)

But like you, i don't have any problem if i display a string previouly inserted by nwnx.
Back to top
View user's profile Send private message
caloup



Joined: 29 Sep 2006
Posts: 59
Location: albi (france)
PostPosted: Wed Jan 03, 2007 11:24    Post subject: Reply with quote
Sorry for the mistake...I tried with SQLYog to insert some accent in a variable manually and effectively i have the same problem than you...
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    nwnx.org Forum Index -> Development All times are GMT + 2 Hours
Goto page Previous  1, 2, 3  Next
Page 2 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group