Stack Overflow

I get a Google Drive permissions error when I try to follow your link. Anyway, do you get the same result with text saved to a Char field?


@MartynA, thank you for you answer. I create char(1000) field with utf encoding and then write text from memo in this field. Code is: ms : TMemoryStream; begin ms := TMemoryStream.Create; try memo1.lines.savetostream(ms, TEncoding.UTF8); ms.position := 0; Query1.Edit; Query1.FieldByName('CHAR1').AsString :=memo1.Lines.Text; Query1.Post; As a result: Text with georgian characters is saved succesfully.


"DB has utf-encoding." that is not how Firebird works. Each column of a character type has its own character set, the default character set only specifies the character set to apply when a column is created without a character set clause, and the default character set doesn't apply retroactively. The problem could be that the column has the wrong character set (or none). The problem could also be indicative of using character set NONE as the connection character set instead of explicitly connecting with a connection character set (UTF8 would probably be the best choice given the data)


Note that blobs are even more problematic especially when using text data in binary blobs and then trying to show them as text. I don't do Delphi myself, but I seem to recall Delphi uses UTF-16 by default, which may introduce another chance for conversion errors.


@MarkRotteveel, everywhere and everytime I use only utf encoding. Connection character set also is utf. Text blob field is created with utf. I delete it and recreate but nothing changes. I add Project and db files in my post above.


@MartynA, I edit my onedrive cloud permission. Now you are able download project and db files.

Not knowing Delphi, I can't really help you further, but to me a TFileStream suggests it reads binary, not text. Are you sure that is the proper course of action here?



I do not have Delphi 10.x nor do I have any older AnyDAC/FireDAC library with me, however.... program MyInfoRepository1; uses Vcl.Forms, uMain in 'D:\MyInfoRepository\uMain.pas' {fMain}; - there are two problems with your ZIP: 1) you do not put your main form sources into it, and 2) the very fact that DPR lists absolute file name instead of relative means that your form sources lie outside of your main project file. Sometimes it is intended, but in your case you clearly is dizzy about Delphi programs structure. So, I am ATM skeptical about ur claim Connection character set also is utf.


Perhaps you can put the texts of your forms - both .pas source texrt and .dfm source text - onto some service like Pastebin.com, or github.com


@MarkRotteveel TFileStream is a stream that reads files. How it's interpreted, as raw bytes or as text or whatever else, is an unrelated concern.


@basti also if you can work with TMemoryStream then perhaps you look for TBlobSteam or TField.CreateBlobStream or something of that kind, that your database-connecting library provides.


@Arioch'The, I update my onedrive folder. I simplify project and when I save with new name(save project as...) I forgotten create forms in it. :) As for being skeptical about my claim that Connection character set is utf. I update post and add image. please see.


did you try other firebird connection libraries? dbExpress, Unified Interbase, Interbase Express? Will they work differently w.r.t blob ?


Notepad's UNICODE is WindowsNT Unicode, which used to be UCS-2 in older NT versions and which is UTF16 now ( no a "core" subset USC-2 and UTF16 are the same). So your rightmost screenshot shows almost the same text file data, just with or without BOM


Query1.Params[0].AsStream := TFileStream.Create(OpenDialog1.FileName, fmOpenRead); - what happens here is TWO operations. 1) you create a binary (not-text) stream out of text file. 2) you push this binary (not-text) stream into blob. When you read files by Default Delphi uses "platform-specific" charset, which for Windows and Delphi2009+ is "WideChar" = UTF16. Then you push the charset-less non-text data into the blob parameter, so no conversion is possible. // try to think along TFile.ReadAllText(const Path: string; const Encoding: TEncoding) and maybe TFile.OpenText


your param type should be text blob, that is ftMemo - check these and compare wit hyour SELECT columns: docwiki.embarcadero.com/Libraries/Seattle/en/… docwiki.embarcadero.com/Libraries/Seattle/en/… docwiki.embarcadero.com/Libraries/Seattle/en/Data.DB.TMemoFi‌​eld also check value and description of TMemoField.Transliterate


So, as of now, I can see two obvious ways to put the text file into the blob field. First one makes double conversion UTF-8 -> UTF-16(UCS2) -> UTF-8. But doing so it implicitly makes data sanity checking on client. Second one would try to just push the data into the database bypassing checks. If you would try to do it with the malformed (non-UTF-8) file - results would differ. Also, Delphi's TEncoding class is weird beast, try to use TEncoding.UTF8 = TUTF8Encoding.Create object to convert non-UTF-8 (malformed) data to string,AFAIR it would return empty string instead of raising Exception


Also, in general I think a special explicit UPDATE OR INSERT SQL statement with using parameters is more safe and debugging-friendly than using implicit SELECT with Query1.Edit, YMMV. Then you can {TMemoField/TParam} . {AsString/AsWideString} := TFile.ReadAllText('myUTF8text.txt', TEncoding.UTF8); for double-conversion and {TMemoField/TParam} . AsBytes := TFile.ReadAllBytes('myUTF8text.txt'); bypassing checks (in particular, if future database versions would change the collation of the blob-text column the direct bypassing insertion maybe would not detect it, not sure).

As of Delphi XE2 though there is a bug in TFile that prevents it reading read-only files (like ones on CD-ROM media, or r/o ACL grants, or r/o file attribute), as even TFile.ReadXXXX methods try to open file for read-write access instead, like default Pascal reset procedure did. I outlined the potential future problems I can see, so check them proactively


@Arioch'The thank you for you attempt to help me. I will carefully read you comments. Today I realise some aspects: When I create txt-file in noteped++ in utf encoding it is utf without BOM. utf in noteped and utf in noteped++ arent the same. utf in noteped is equivalent to utf wirh BOM in noteped++. You may create file in noteped as utf and then open in noteped++ and see encoding in menu.


When I create txt-file in noteped++ not in simple utf encoding but in ucs-2 BE BOM or ucs-2 LE BOM they are saved in blobs correctly(both are 16-bite utfs). Maybe it is because it's difficult for delphi distinguish utf without BOM from Ansi. Both are without markers. You may see image from hex, I put it on onedrive, in my project folder. onedrive.live.com/…


utf encoding - no such thing. There is UTF-7, UTf-8, UTF-16. UTF-32 and maybe more. They are different. Then, UTF-8 (and perhaps UTF-7) are independent on "byte order" (google "big endian" or "intel/Motorola endian") while those >8 UTFs are dependent in proper bytestream aggregation, so they usually use BOM (byte-order marker), to make parser learn byte order of the generator, however it is not required, but a good habit to put it when passing texts across architectures, still a habit not requirement. UTF <= 8 don't need BOM, while it is allowed. BOM is just "invisible letter", one of.


UTF-nn is still UTF-nn with or without BOM. It is just a special letter. Text encoding does not change if you add or remove some letter. UCS2 (old Windows Unicode) is subset of UTF-16 (today Windows Unicode). Every UCS2 text is UTF-16, just like every ASCII7 text is UTF-8. Delphi's WideString is by definition Windows Unicode (it is COM B_STR datatype), UnicodeString is not B_STR but is Windows Unicode too. C# and Java strings are UTF-16. So, when you loose charset information and do plain bytes transfer - UTF-16 interpretation is what you probably would receive, by default.


BOM letter then MIGHT trigger reconsideration, if some Delphi library would do a fool-proofness check. But that is not granted. To be on safe side you should keep strings (texts) as strings and never switch to bytes, then you might blame libraries if they screw charset information. Alternatively, you can bypass libraries and work with bytes directly, then it would be your responsibility to track and check all the charsets through all the chain and to transcode where needed. When you did ftBlob and LoadFromFile you talked in bytes-not-letters terms. When you use select and db-aware grid


you talk in letters-not-bytes terms. In between there can be mismatches. Now, your database have it as text blob, the letters-not-bytes principle, then the mismatch is (least library bugs) when you pushing the data into the database, on the update or insert statement part for example. And in that step you have to decide you either would work with letter or with bytes, you either would delegate charset tracking and transcoding to libraries or do it yourself and preclude libraries. And those approaches I outlined above in .AsWideString := ... and .AsBytes := ... snippets


UTF-16 interpretation is what you probably would receive, by default - in Delphi 2009+, I mean. Delphi 2007 and prior, and libraries written back then, would rather default to 8-bits MBCS ("ANSI") interpretation of the very same binary data, if you convert bytes to text.