BRT Community

Please login or register.

Login with username, password and session length
Advanced search  

News:

Welcome to the Bridgetek Community!

Please read our Welcome Note

Technical Support enquires
please contact the team
@ Bridgetek Support

Please refer to our website for detailed information on all our products - Bridgetek - Bridging Technology

Pages: [1] 2

Author Topic: UTF-8 Encoding Error  (Read 17687 times)

Simon

  • Newbie
  • *
  • Posts: 11
    • View Profile
UTF-8 Encoding Error
« on: March 06, 2020, 10:22:54 AM »

Hi there,

I have an issue on the UTF-8 encoding used with the BT816-Chip. For the use of some special symbols i need font-elements that are adressed by UTF-8 encoding with four bytes. So I converted this font https://fontlibrary.org/de/font/symbola#Symbola-Regular that contains the symbols I want to use with EVE Asset Builder v1.2.0.
After progammming the font on my flash, I tried to adress i.e. the UTF-8 code F09FA183 to type a special arrow. The display didn't show anything.
Then I tried to use a symbol with a three byte UTF-8 encoding i.e. E2AF86 (a different arrow symbol) and everything worked fine.

I also tried different ways to use the symbol:
One try was about putting each byte into a text array by hand like shown below, what worked as describes up to 3-byte encoding.

Code: [Select]
switch (direction)

         {
            case 0://arrow up
            {

               arrow_x = x1+((x2-x1)/2)-20;
               arrow_y = y1+((y2-y1)/2)-12;

               EVE_cmd_setfont2(13, SPECIAL_FONT2_RAM_ADR,0); //special font

         //this arrow is displayed correctly
               LCD_FuncData.text_buffer[0] =0xE2;
               LCD_FuncData.text_buffer[1] =0xAF;
               LCD_FuncData.text_buffer[2] = 0x85;
               LCD_FuncData.text_buffer[3] = 0;
               break;
            }
            case 1://arrow down
            {
               arrow_x = x1+((x2-x1)/2)-20;
               arrow_y = y1+((y2-y1)/2)-12;

               EVE_cmd_setfont2(13, SPECIAL_FONT2_RAM_ADR,0); //special font

         //this arrow is not shown on display
               LCD_FuncData.text_buffer[0] =0xF0;
               LCD_FuncData.text_buffer[1] =0x9F;
               LCD_FuncData.text_buffer[2] = 0xA1;
               LCD_FuncData.text_buffer[3] = 0x83;
               LCD_FuncData.text_buffer[4] = 0;
               break;
            }
  }

  if (button_down)
        {
               EVE_cmd_button(x1,y1,x2-x1,y2-y1,13,EVE_OPT_FLAT); //in here the button and its label is generated by sending the text array content to the BT-Chip
        }
        else
        {
               EVE_cmd_button(x1,y1,x2-x1,y2-y1,13,0);
        }

Another try I started in the EVE Screen Editor with only a few lines:

Code: [Select]
CLEAR(1, 1, 1)
CMD_SETFONT2(13, 0,0)
CMD_TEXT(123, 117, 13, 0, "🡃?")

I opened the font data in the emulator and loaded my font. This time the display showed two questionmarks instead of the arrow. In the Inspector view i could see that the generated code actually adresses the same section as for the questionmark character.

So my question is if there is an availability to use the Symbols with four byte UTF-8 encoding or if the BT-Chip restricts the usage of them.

Hopefully somebody can answer me that question.

Regards,
Simon
Logged

BRT Community

  • Administrator
  • Hero Member
  • *****
  • Posts: 733
    • View Profile
Re: UTF-8 Encoding Error
« Reply #1 on: March 09, 2020, 02:47:24 PM »

Hello Simon,

I will pass this on to the R&D team for clarification on UTF-8 encoding support.

Best Regards,
BRT Community
Logged

BRT Community

  • Administrator
  • Hero Member
  • *****
  • Posts: 733
    • View Profile
Re: UTF-8 Encoding Error
« Reply #2 on: March 16, 2020, 11:29:55 AM »

Hello Simon,

We discovered an issue with EAB regarding 4-byte UTF-8 characters.
EAB v1.4 is scheduled at Mar 25 and will include a fix for this.

Best Regards,
BRT Community
« Last Edit: March 16, 2020, 11:32:29 AM by BRT Community »
Logged

Simon

  • Newbie
  • *
  • Posts: 11
    • View Profile
Re: UTF-8 Encoding Error
« Reply #3 on: March 16, 2020, 02:44:54 PM »

Hello BRT Community,

many thanks for your reply. I am very excited about the revised version and would like to thank you for the bug fix.

Regards,
Simon
Logged

Rudolph

  • Sr. Member
  • ****
  • Posts: 389
    • View Profile
Re: UTF-8 Encoding Error
« Reply #4 on: April 09, 2020, 10:16:43 AM »

As the 25th of march has passed now, any ETA for the new EAB?
I would not mind giving a beta a spin if it shows up in the "Test and Review Area".
Logged

BRT Community

  • Administrator
  • Hero Member
  • *****
  • Posts: 733
    • View Profile
Re: UTF-8 Encoding Error
« Reply #5 on: April 09, 2020, 01:58:54 PM »

Hello,

I will chase the web team up to see what's what regarding the latest release of EAB.

Best Regards,
BRT Community
Logged

Simon

  • Newbie
  • *
  • Posts: 11
    • View Profile
Re: UTF-8 Encoding Error
« Reply #6 on: April 28, 2020, 10:15:57 AM »

Hello,
I have noticed that the new version of EVE Asset Builder is available and I am happy to see that the UTF8 problem is fixed. I have been testing version 1.4 since yesterday and still have the same problem in my test scenarios as before.
Could you send me a flash file and an ESE example with which the fix works? Or is there a possibility that someone from the R&D team contacts me?

I look forward to your feedback.
Simon
Logged

BRT Community

  • Administrator
  • Hero Member
  • *****
  • Posts: 733
    • View Profile
Re: UTF-8 Encoding Error
« Reply #7 on: April 28, 2020, 04:13:59 PM »

Hello,

Just to clarify, after using the latest version of EAB, your special symbols are still not showing on screen when you attempt to use them in your code?

Best Regards,
BRT Community
Logged

Simon

  • Newbie
  • *
  • Posts: 11
    • View Profile
Re: UTF-8 Encoding Error
« Reply #8 on: April 29, 2020, 07:26:33 AM »

Yes exactly. With version 1.4 my icons are still not displayed correctly.
It is noticeable that the version of the EVE Font Conversion Utility in EAB version 1.2 has the same version number (v0.1.6) as in EAB version 1.4. Maybe it has something to do with this? Another thing that would suggest is that according to the release notes the compression for the UTF-8 code point setting has been fixed but does not seem to work either.

I would be glad about a feedback.

Regards Simon
Logged

BRT Community

  • Administrator
  • Hero Member
  • *****
  • Posts: 733
    • View Profile
Re: UTF-8 Encoding Error
« Reply #9 on: April 29, 2020, 11:58:02 AM »

Hello

I have asked the development team if the font convertor version number should have incremented for the latest release.

In the meantime, we have given it a go here, please see below for the ESE source and attached screenshot:
Font_112.zip

Best Regards,
BRT Community
Logged

Rudolph

  • Sr. Member
  • ****
  • Posts: 389
    • View Profile
Re: UTF-8 Encoding Error
« Reply #10 on: April 29, 2020, 06:11:36 PM »

When I try to convert Symbola.ttf with EAB 1.4 to extended format, a size of 12 and "Full Unicode Characters" I get this in the "Result" window:
"unknown freetype error"
According to Fontrdop Symbola has 7956 glyphs.

When I switch to a NotoSans-Regular.ttf with 3246 glyphs it takes a while but completes.
Two issues though.
The reported amount of characters in the .xfont file is always 0x00010000.
And the compression ratio when I zip the resulting NotoSans-Regular_12_ASTC.glyph is really high:
1473 kB -> 96 kB

Sure, this is not fair since zip can work with the complete set at once.
But when I open the NotoSans-Regular_12_ASTC.glyph in a hex-editor it looks like this:

0000 FC FD FF FF FF FF FF FF 00 00 00 00 00 00 FF FF ................
0010 FC FD FF FF FF FF FF FF 00 00 00 00 00 00 FF FF ................
0020 66 68 18 80 46 18 18 10 18 10 7F 7F 28 10 04 28 fh..F.......(..(
0030 07 0A 11 00 00 80 5F 5C 61 33 13 20 84 00 11 42 ......_\a3. ...B
0040 66 00 00 B6 01 00 00 40 00 00 00 00 00 00 00 00 f......@........
0050 66 00 00 30 00 00 00 00 00 00 00 C0 00 00 00 80 f..0............
0060 FC FD FF FF FF FF FF FF 00 00 00 00 00 00 FF FF ................
0070 FC FD FF FF FF FF FF FF 00 00 00 00 00 00 FF FF ................
0080 FC FD FF FF FF FF FF FF 00 00 00 00 00 00 FF FF ................
0090 FC FD FF FF FF FF FF FF 00 00 00 00 00 00 FF FF ................
00A0 FC FD FF FF FF FF FF FF 00 00 00 00 00 00 FF FF ................
00B0 FC FD FF FF FF FF FF FF 00 00 00 00 00 00 FF FF ................
00C0 FC FD FF FF FF FF FF FF 00 00 00 00 00 00 FF FF ................
00D0 FC FD FF FF FF FF FF FF 00 00 00 00 00 00 FF FF ................

Is this really compressed?


The example works, sort of, it displays the arrow.
But it deletes the Symbola_85_ASTC.glyph and Symbola_85_ASTC.xfont files in the resources dir when you start it.
Under "Content" it shows errors.




Logged

Simon

  • Newbie
  • *
  • Posts: 11
    • View Profile
Re: UTF-8 Encoding Error
« Reply #11 on: April 30, 2020, 12:31:09 PM »

The example also shows the arrow in my case. As flash content the fonts are displayed as .raw files. A self-generated .raw file brings the same result with the arrow. 
However, it does not work if I use my self-created .glyph or .xfont file, which is used to save the font data to the flash memory of the display.

Logged

BRT Community

  • Administrator
  • Hero Member
  • *****
  • Posts: 733
    • View Profile
Re: UTF-8 Encoding Error
« Reply #12 on: May 04, 2020, 04:41:08 PM »

Hello,

Can you check the settings of EAB font conversion?   
To use the glyph in RAM_G,  you need set it properly in EAB.  See attached image.

Best Regards,
BRT Community
Logged

Simon

  • Newbie
  • *
  • Posts: 11
    • View Profile
Re: UTF-8 Encoding Error
« Reply #13 on: May 05, 2020, 07:56:17 AM »

Hello,

thank you for your answer. I don't want to use the glyph file in RAM_G but save it on an external flash and only load the xfont file into RAM_G. With this file I can then address the graphic data. I convert the data for the flash with the EAB Flash Utilities and load it via the map file into the ESE, just like I would do it on the hardware or have done so far.
This worked without any problems for me so far, only with the mentioned symbols it still does not work.
I have attached my setting of the EAB and the use of the font in the ESE. To show that my approach works basically I added some symbols from lower code pages.
Because the used symbols maybe hard to see in the screenshot, here is the code in text form:
Code: [Select]
CLEAR(1, 1, 1)
CMD_SETFONT2(0, 0, 0)
CMD_TEXT(59, 41, 0, OPT_FORMAT, "%c%c BC", 129091, 65)
CMD_BUTTON(51, 104, 100, 30, 0, OPT_FORMAT, "%c", 129091)
CMD_TEXT(25, 160, 0, 0, "♪☣⛇∰⌚🡃")

Regards
Simon
Logged

pauljiao

  • Jr. Member
  • **
  • Posts: 53
    • View Profile
Re: UTF-8 Encoding Error
« Reply #14 on: May 08, 2020, 04:48:19 AM »

Simon, as far as I knew, rendering the glyph from flash needs
1) The glyph is based on ASTC.
2) The starting address in flash shall be 64 bytes aligned.  From the screenshot, you may see the address 10000 is not 64 bytes aligned.

Logged
Pages: [1] 2