EVE Asset Builder

[Rudolph]

I found more issues with EAB 1.4.0 beside the UTF-8 ones that are already under investigation.

The release notes state:
- Add the "Verify" function in flash utility

And yes, there is a checkmark-button for this in the EAB GUI.
However, when you activate the option and click "UPDATE", you only get the default output of "ProgFlash.exe"
in the Log window:
Welcome to BT81X Flash Programming Utility v1.0
Copyright(c) Bridgetek Pte Ltd All Rights Reserved

Usage: ProgFlash [command] [argument]

module       Select Program Module [ FT4222 (default) | MPSSE ]
chip         Select Program EVE Chip [ BT815 | BT816 | BT815A ]
erase        Erase all of flash
newblob      Install blob <file_name> in flash
write        Write <file_name> to flash (assumes flash is already erased)
update       Write <file_name> to flash, erasing if necessary
read         Read all of flash into <filename>

As if the "ProgFlash.exe" is supposed to have a verify command but the version shipped with the EAB package does not.
The "ProgFlash.exe" from EAB 1.2 has the exact same output, the newer version is 512 bytes longer though.
And I have not found a verify command in the .exe but the list above is missing the "detect" command.

Regarding "detect", it does work as a command from the command-line but fails from the GUI.
And the GUI suggests that there is a parameter to avoid measuring the writing speed, I wonder if that is there and what is.


When I use ProgFlash to erase a chip this is what happens:
---
D:\Chip_Test>ProgFlash module MPSSE erase
Information on channel number 0:
Flags=0x2
Type=0x8
ID=0x4036014
LocId=0x422
SerialNumber=
Description=Single RS232-HS
ftHandle=0x0

handle=0x7197f0 status=0x0
VC1 register ID after wake up 7c

reg_touch_rz =0x7fff
reg_touch_rzthresh =0xffff
reg_touch_tag_xy=0x80008000
reg_touch_tag=0x0

Switch flash status to FULL. Result: SUCCESS
Erasing Flash...
Progress ERASE 1
Progress ERASE 90
Switch flash status to BASIC. Result: SUCCESS
Progress ERASE 100
Erase Flash successfully
---

Why does it switch the flash to FULL in order to erase it?
There is no need to this, the chip-erase command will not work any faster.
What it does however is making it impossible to erase chips with EAB that do not work with the "unified.blob".
For example AT25SF641-SUB-T and AT25QF128A-SHB-T from Adesto.


On a side-note, what is a BT815A?

And why does EAB come with a "unified.blob", a "BT815.blob" and a "BT816.blob" now?
Which really is the same file with three different names.


- Replace 'numpy' by 'tinynumpy' to reduce distribution package size

No numpy to be found anywhere in the EVE Asset Builder installation.

[BRT Community]

Hello,

Thanks for you post.
I have passed this on to the development team for EAB for their consideration.

Best Regards,
BRT Community

[Rudolph]

Hello,

a month has passed and neither has EAB been updated or the questions been answered.
I really wonder why Bridgetek is moving so slowly.
Same applies to documentation.

Sure, quality control is important.
But especially EAB does not leave the impression with me that it has been thoroughly tested.
Let us help with that, put more stuff in the "Test and Review Area".

Best regards,
Rudolph

[BRT Community]

Hello,

I will chase the development team up regarding this.
As for the 3 separate blob files, I believe this is for future proofing in-case different versions are required for different ICs, but yes currently they are all the same file.

Best Regards,
BRT Community

[BRT Community]

Hello Rudolph,

There is now a new version of EAB available (1.6.0) which should address the points in this post:
www.brtcommunity.com/index.php?topic=153.0

I would be grateful if you could give it a try.

Best Regards,
BRT Community

[Rudolph]

I have a suggestion for a new tab in EAB.

What about adding a bin2c tool?
The output could be similar to the .rawh files that the image converter already generates.

But a few options would be sweet:
hex ouput
byte / word / long output
values per line configureable to upto four digits

Why? Conveniance.
I just converted two UTF-8 fonts, put them in a flash-image and compressed this flash-image.
Now I have to use some other tool to convert the .zlib file to a header file.

[BRT Community]

Hi Rudolph,

Thanks for your suggestion,

Yes we agree, it would be a very useful and convenient feature,

Our development team will add this in for a future release of EAB.

Best Regards, BRT Community

[Rudolph]

Hello,

I just stumbled over EAB v2.1.0 RC2: https://brtchip.com/eve-toolchains/#EVEAssetBuilder

-   Add new tool Bin2C to convert binary file to C array

Sweet. :-)

I have two minor issues with the converter.
The first is that it does not output leading zeros which make the array look asymetrical.
The second is that the array is generated without size.

uint8_t data [] = {
0x78, 0x9c, 0xed, 0x9d, 0xf, 0x7c, 0x14,

Yes, *minor* issues. :-)
This is pretty much exactly what I was looking for, good job. :-)


The "Custom Touch" module looks interesting.
The example given in the "Read Me" does not compile though, it is using i2c_read16() while
it needs to be i2c_read16le() or maybe i2c_read16be().

After a modification it compiles but the output is irritating:

touch.c compilation is in progress...
Microcode 'capcs.fs' uses 1100 bytes.
Loader command stream is 364 bytes
350 new bytes in firmware
EXCEPTION IN (custom_touch.py, LINE 52 ""): WindowsPath('D:/EAB_Test/touch.bin') and 'D:/EAB_Test\\touch.bin' are the same file

Compilation is done.

There is a touch.bin and a touch.load.h file in the output directory though.

When you do not use the same folder as output folder the result is just this:
touch.c compilation is in progress...
Microcode 'capcs.fs' uses 1100 bytes.
Loader command stream is 364 bytes
350 new bytes in firmware

Compilation is done.

And also only a touch.bin and a touch.load.h file in the output directory.

[Rudolph]

I have a bigger issue to report that persists with EAB 1.2.0 RC2.

It still does switch on the backlight and it looks like to full when using the "Flash Programmer" part.

Please change this, it only works with rather small displays that are not drawing much current for the backlight.
We are on the verge of getting the first 10.1" in our hands and this behaviour of EAB already is bad with 7" modules.
The current reported for the USB by the adapter is 500mA but a larger display easily will exceed this.
And all the USB/SPI adapters I know are using LDOs, with the VA800A-SPI even setting a bad example by using a puny
one in a SOT23 package.

There is no need to display anything for flash programming, or testing the speed of the chip.
Yet it displays a black screen with backlight on which is the worst it could do since the panel itself
draws the most current with all pixels switched to black.

[BRT Community]

Hello Rudolph,

Thank you for your valuable feedback, I will pass this onto the development team immediately.

Best Regards,
BRT Community

[Rudolph]

I was playing with EAB v2.1.0 RC2 conveting fonts and found
that the "Font Size" can not exceed 255.
That is a little inconveniant as I was trying to convert a numbers-only subset to as large as possible.
I used a numbers.txt file containing only " +-.0123456789" and with a "Font Size" of "only" 255,
the result in pixels for Funtype.ttf and these 14 chars is a width of 180 and a height of 192.

Also, that "Full Unicode Characters" is still not working exactly like it should.
For Funtype.ttf ( https://fontlibrary.org/de/font/funtype ) the preview states:
This font contains 364 glyphs.

And this is accurate.

The font conversion log however has this:
Format        : COMPRESSED_RGBA_ASTC_8x5_KHR
Compressed    : thorough
Layout Width  : 64
Layout Height : 4
Pixel Width   : 32
Pixel Height  : 20
Number of characters in xfont file     : 65536
Number of characters in user input file: 65536
   Success: 65536
   Fail   : 0

And when I use a charmap.txt file to limit the characters converted I get this:
Format        : COMPRESSED_RGBA_ASTC_8x5_KHR
Compressed    : thorough
Layout Width  : 32
Layout Height : 4
Pixel Width   : 16
Pixel Height  : 20
Number of characters in xfont file     : 8448
Number of characters in user input file: 106
   Success: 106
   Fail   : 0

The numbers reported do not match.

Also the Funtype_12_ASTC.glyph with 106 chars is 48384 bytes long.
But the Funtype_12_ASTC.glyph which should have 364 glyphs since the font does not contain more,
it has a size of 307.072 bytes.
That is almost twice the size that I would expect when going from 106 chars to 364 chars.

And talking about sizes, again, there appears to be a severe issue when converting fonts.
16x20x106 is 33920
So an uncompressed array of 16x20 pixel images in L8 format of 106 individual images would be smaller than
what the font converter currently presents as COMPRESSED_RGBA_ASTC_8x5_KHR.

8x5 is 3.2 bits per pixel and when I compress a single image of 16x20 from this font,
the result is 320 bytes in L8, and only 128 bytes as COMPRESSED_RGBA_ASTC_8x5_KHR -> works.

So the data for the glyhs should be around 13568 bytes but the resulting .glyph file
is over 3.5 times that size.

There also is the option to generate the font in L8 format.
The resulting Funtype_12_L8.glyph for 106 glyphs of 16x20 pixels has a size of 106560 bytes.
Why? That is 3.1 times the expected size.

I thought this might be an issue with alignment of images in FLASH but then the requirement is 32 bytes aligned.
128 bytes for COMPRESSED_RGBA_ASTC_8x5_KHR already is perfectly aligned.
Even adding 32 bytes to every glyph does not account for the size.
And when using RAM_G as a target for the ASTC compressed font the .glyph file has the exact same size.
So no, alignement can not be the issue.
Also L8 does not even allow the data to be in FLASH.

[BRT Community]

Hello,

I’ve had some clarification from the developers regarding Unicode font conversions:

• Only the font metrics table covers the maximum code point of the input characters.
• For the .glyph data,  only the input characters are converted. The characters whose code point are not present are not
  converted.

Then, number of characters is determined by: (max code point + 128 ) & ~ 127. For example if one input character set includes a character whos max code point is 382 (ž), then the .xfont will list 384 characters:

(382 + 128) & ~127 = 384

If another input character set in used which include for example the trademark symbol (â,,¢), its max code point would be 8482, and the .xfont would list 8576 characters:

(8482 + 128) & ~127 = 8576


The metric table (.xfont) will include the number relating to the max code point, and will include padding to the relevant code points for a characters width data.
However only the characters included in the input character set will be converted to the .glyph data.

I will need to investigate why you are seeing such large .glyph files when converting these fonts with differing character sets. Could you provide the input char set and font file you are using during conversion?

Best Regards,
BRT Community

[Rudolph]

Hello,

I’ve had some clarification from the developers regarding Unicode font conversions:

• Only the font metrics table covers the maximum code point of the input characters.
• For the .glyph data,  only the input characters are converted. The characters whose code point are not present are not
  converted.

This does not appear to be working and it might never did work so far.

In "full" unicode range the output always is 65536, regardless how many glyphs the font actually contains
and regardless of that the number of glyhs is successfully identified.

In the attached fonttest.zip you find font_metrics.txt with details about the different conversions in the set.
I used the included notosans-regular-webfont2.ttf for this test which is a reduced subset of NotoSans-Regular.ttf
from Google.
I converted it down to 135 glyphs and the highest glyph is 0x25fc / 9724.

Then, number of characters is determined by: (max code point + 128 ) & ~ 127. For example if one input character set includes a character whos max code point is 382 (ž), then the .xfont will list 384 characters:

(382 + 128) & ~127 = 384

If another input character set in used which include for example the trademark symbol (â,,¢), its max code point would be 8482, and the .xfont would list 8576 characters:

(8482 + 128) & ~127 = 8576

This is not what I got, I have a set which includes only a single char beyond 0x00fc and
that is 0x02da.
And the number of characters in the xfont file is reported as 256 for a total number of 103 chars in the set.
I would guess that there is at least one block of 128 for each code-block.

The full set of 135 glyphs for this font should report 5 * 128 = 640 as the chars are from five different blocks.

The metric table (.xfont) will include the number relating to the max code point, and will include padding to the relevant code points for a characters width data.
However only the characters included in the input character set will be converted to the .glyph data.

I will need to investigate why you are seeing such large .glyph files when converting these fonts with differing character sets. Could you provide the input char set and font file you are using during conversion?

Best Regards,
BRT Community

The sizes in this test set are all over the place.
In full the converter just appears to be broken.
It appears that it generates glyphs for 655536 codepoints although there are only 135 glyphs in this example.
And this happens with all fonts that I feed it this way.

I set the compression to ASTC and auto which got me 8x8 for all conversion and which is 2 bits per pixel.
full 135 glyphs: notosans-regular-webfont2_32_ASTC.glyph 377280 bytes, 48x48 pixel -> room for 655 glyphs
103 glyphs: notosans-regular-webfont2_32_ASTC.glyph 145472 bytes, 48x48 pixel -> room for 252,5 glyphs
95 glyphs: notosans-regular-webfont2_32_ASTC.glyph 73728 bytes, 48x48 pixel -> room for 128 glyphs
14 glyphs: notosans-regular-webfont2_32_ASTC.glyph 24576 bytes, 32x25 pixel -> room for 128 glyphs
10 glyphs: notosans-regular-webfont2_32_ASTC.glyph 24576 bytes, 32x25 pixel -> room for 128 glyphs

The sizes of the glyphs with ASTC 8x8 and their pixels perfectly align to 32 bytes.

As last exercise I just converted the notosans-regular-webfont2_32_ASTC.png that was generated
when conveting to  95 chars with ANSI_32_126.txt.
As ASTC 8x8 the file only has 54720 bytes and not 73728 bytes as the .glyph.
The notosans-regular-webfont2_32_ASTC.png has 48x4560 pixels.
I cut out a single glyph of 48x48 pixel. Converted to ASTC 8x8 the .raw file has 576 bytes.

Looking at the compressed data, the .glyph file looks suspicious, not exactly like the ASTC 8x8 raw
and the .glyph has a higher compression ratio when zipped as the .raw file has.

[BRT Community]

Hello Rudolph,

Thank you for your detailed analysis of this issue, I have passed this onto the development team for investigation.

Best Regards,
BRT Community.

[Rudolph]

I have annother one, in order to better understand this I converted only a single char of full NotoSans-Regular.ttf.
And then I went a step further and disected the .xfont files.

ASTC 8x8: 2 bits per pixel

mapped: !singlechar1.txt, Size: 32, ASTC auto, Compression Speed: thorough
" ê™± " - U+A671 - 0xea 0x99 0xb1
Format        : COMPRESSED_RGBA_ASTC_8x8_KHR
Compressed    : thorough
Layout Width  : 64
Layout Height : 5
Pixel Width   : 32
Pixel Height  : 40
Number of characters in xfont file     : 42624
Number of characters in user input file: 1
   Success: 1
   Fail   : 0

NotoSans-Regular_32_ASTC_1.png 32 x 40 - 1 glyph
NotoSans-Regular_32_ASTC_1.glyph 86720 bytes
32*40*2 = 2560 bits -> 320 bytes
86720 / 320 = 271

NotoSans-Regular_32_ASTC_1.xfont:
0   0x0100aaff Signature
4   0x00000b1f Total size of the font block, in bytes: 2847
8   0x000093b7 Bitmap format: COMPRESSED_RGBA_ASTC_8x8_KHR
12  0x0000024a Bitmap swizzle value: 10 0100 1010
16  0x00000040 Font bitmap line stride, in bytes: 64
20  0x00000005 Font bitmap height, in pixels: 5
24  0x00000020 Font screen width, in pixels: 32
28  0x00000028 Font screen height, in pixels: 40
32  0x00800080 Pointer to font graphic data in memory, including flash: address in flash, 4096 (128 * 32)
36  0x0000a680 Total number of characters in font(N), multiple of 128: 42624
40  0x0000b2c0 Offsets to glyph data - N/128 -> 42624 / 128 = 333
...
1348 0x0000b2c0 Offset to glyph data
1352 0x00000000 Offset to glyph data
1356 0x00000000 Offset to glyph data
1360 0x00000000 Offset to glyph data
1364 0x00000000 Offset to glyph data
1368 0x000012c0 Offset to glyph data
1372 0x00000a90 Offsets to width data - - N/128 -> 42624 / 128 = 333
...
2696 0x00000a90 Offset to width data
2700 0x00000a9f Offset to width data
2704 0x00000000 Width data, one byte per character
...
2828 0x00000000 Width data, one byte per character
2832 0x0000001E Width data, one byte per character
2836 0x00000000 Width data, one byte per character
2840 0x00000000 Width data, one byte per character
2844   0x000000 Width data, one byte per character



mapped: !singlechar2.txt, Size: 32, ASTC auto, Compression Speed: thorough
"Ω" - U+03A9 - 0xce 0xa9
Format        : COMPRESSED_RGBA_ASTC_8x8_KHR
Compressed    : thorough
Layout Width  : 64
Layout Height : 3
Pixel Width   : 32
Pixel Height  : 24
Number of characters in xfont file     : 1024
Number of characters in user input file: 1
   Success: 1
   Fail   : 0

NotoSans-Regular_32_ASTC_2.png 32 x 24 - 1 glyph
NotoSans-Regular_32_ASTC_2.glyph 32640 bytes
32*24*2 = 1536 bits -> 192 bytes
32640 / 192 = 170

NotoSans-Regular_32_ASTC_2.xfont:
0   0x0100aaff Signature
4   0x00000b1f Total size of the font block, in bytes: 2847
8   0x000093b7 Bitmap format: COMPRESSED_RGBA_ASTC_8x8_KHR
12  0x0000024a Bitmap swizzle value: 10 0100 1010
16  0x00000040 Font bitmap line stride, in bytes: 64
20  0x00000003 Font bitmap height, in pixels: 3
24  0x00000020 Font screen width, in pixels: 32
28  0x00000018 Font screen height, in pixels: 20
32  0x00800080 Pointer to font graphic data in memory, including flash: address in flash, 4096 (128 * 32)
36  0x00000400 Total number of characters in font(N), multiple of 128: 1024
40  0x00001f70 Offsets to glyph data - N/128 -> 1024 / 128 = 8
...
64  0x00001f70 Offset to glyph data
68  0x00000000 Offset to glyph data
72  0x0000092 Offsets to width data - - N/128 -> 1024 / 128 = 8
...
96  0x0000092 Offset to width data
100 0x0000068 Offset to width data
104 0x00000000 Width data, one byte per character
...
140 0x00000000 Width data, one byte per character
144 0x00001900 Width data, one byte per character
148 0x00000000 Width data, one byte per character
...
268 0x00000000 Width data, one byte per character
272     0x0000 Width data, one byte per character


And to check if the fonts are working I generated a flash image and fed it to ESE:

CLEAR(1, 1, 1)
CMD_SETFONT2(10, 0, 0)
CMD_SETFONT2(11,4096, 0)
CMD_TEXT(95, 213, 10, 0, "ê™±")
CMD_TEXT(140, 222, 11, 0, "Ω")

0   0x26000007   CLEAR(1, 1, 1)
1   0x0500000a   BITMAP_HANDLE(10)
2   0x28000000   BITMAP_LAYOUT_H(0, 0)
3   0x07f88005   BITMAP_LAYOUT(GLFORMAT, 64, 5)
4   0x2e0093b7   BITMAP_EXT_FORMAT(COMPRESSED_RGBA_ASTC_8x8_KHR)
5   0x2f00024a   BITMAP_SWIZZLE(ONE, ONE, ONE, RED)
6   0x29000000   BITMAP_SIZE_H(0, 0)
7   0x08004028   BITMAP_SIZE(NEAREST, BORDER, BORDER, 32, 40)
8   0x0500000b   BITMAP_HANDLE(11)
9   0x28000000   BITMAP_LAYOUT_H(0, 0)
10   0x07f88003   BITMAP_LAYOUT(GLFORMAT, 64, 3)
11   0x2e0093b7   BITMAP_EXT_FORMAT(COMPRESSED_RGBA_ASTC_8x8_KHR)
12   0x2f00024a   BITMAP_SWIZZLE(ONE, ONE, ONE, RED)
13   0x29000000   BITMAP_SIZE_H(0, 0)
14   0x08004018   BITMAP_SIZE(NEAREST, BORDER, BORDER, 32, 24)
15   0x22000000   SAVE_CONTEXT()
16   0x27000002   VERTEX_FORMAT(2)
17   0x06000000   CELL(0)
18   0x0500000a   BITMAP_HANDLE(10)
19   0x1f000001   BEGIN(BITMAPS)
20   0x01800580   BITMAP_SOURCE(0x800000 | 1408)
21   0x40be0354   VERTEX2F(380, 852)
22   0x23000000   RESTORE_CONTEXT()
23   0x22000000   SAVE_CONTEXT()
24   0x27000002   VERTEX_FORMAT(2)
25   0x06000000   CELL(0)
26   0x0500000b   BITMAP_HANDLE(11)
27   0x1f000001   BEGIN(BITMAPS)
28   0x01800c66   BITMAP_SOURCE(0x800000 | 3174)
29   0x41180378   VERTEX2F(560, 888)
30   0x23000000   RESTORE_CONTEXT()


Okay, there is a correlation between the number of chars in the .xfont file and the Unicode number of the char.
U+A671 -> 42609 -> 42609 / 128 = 332,8 -> 333 blocks of 128 needed
U+03A9 -> 937 -> 937 / 128 = 7,3 -> 8 blocks of 128 needed
So the higher the number the last char in the set is, the longer the .xfont file gets.

But how do we get to the glyph?

a671
1010 0110 0111 0001
101001100 1110001 -> block number 332, glyph number 113th ?
glyph offsets:
0-327: 0x0000b2c0
328-331: 0x00000000
332: 0x000012c0

And now I am lost.

This directly addresses the glyph:
17   0x06000000   CELL(0)
20   0x01800580   BITMAP_SOURCE(0x800000 | 1408)

The "1408" is in units of 32 bytes.
This works fine as well:
CMD_SETBITMAP(0x800000 | 1408, COMPRESSED_RGBA_ASTC_8x8_KHR, 32, 40)
VERTEX2F(1600, 100)

But how does the glyph-offset in the .xfont file lead to this address?


Regarding the sizes, it looks like there are actually three images in this .glyph file?
And the one that contains the actual glyph needs to be larger than the glyph itself since
its absolute address is calculated somehow from the lower 7 bits of its Unicode number?

That explains the size of the .glyph file, unclear math or not, the .xfont only has information about blocks
of 128 glyphs and calculates the absolute address of a glyph from the offset-address of the block, the lower 7 bits of the Unicode number and probably the character widths.
This means the .glyph file itself has to be bigger than the glyph to allow for that structure.

So adding more glyphs to a group that already is present does not make the .glyph bigger?

mapped: !CharMap2.txt, Size: 32, ASTC auto, Compression Speed: thorough
" !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ Â£Â°Â±Â²Â³ÂµÂ¼Â½Â¾Ã,,Ö×Üßäöü"
Format        : COMPRESSED_RGBA_ASTC_8x8_KHR
Compressed    : thorough
Layout Width  : 96
Layout Height : 6
Pixel Width   : 48
Pixel Height  : 48
Number of characters in xfont file     : 256
Number of characters in user input file: 113
   Success: 113
   Fail   : 0

NotoSans-Regular_32_ASTC.png 48 x 5424 - 113 glyphs
NotoSans-Regular_32_ASTC.glyph 145472 bytes
48*48*2 = 4608 bits -> 576 bytes
145472 / 576 = 252,5

That theory seems to be holding some water as the size of the .glyph is exactly the same as with the !Charmap.txt file from my earlier post that has ten glyphs less.
So the glyphs for the not existing codepoints are of the same width and height but empty?

The .xfont file also has the same size, the extra ten glyphs probably had no impact on the width data.


That leaves the potential issue that empty pages seem to take up quite some space in the .glyph file as well.
Oh yes, and that "Full Unicode Characters" does not convert the glyphs that are contained in the font but
generates a file with 65535 glyphs is odd as well.

In any case, working with a charmap-file works a lot better than stripping unicode-ranges from a given font and doing a full-convert on that stripped font.
And it is a lot faster.