Font "All Chars" Broke after updating?

Questions
1

I encountered a problem with the special characters on my font and can’t fix it so far. I have it so that a user can input alt-sequence codes to get a special character. This was working just last week, but now it’s broken. It displays the tilde as if it’s not covered in the font.

I seem to remember updating to Version 1.2.127 soon after pushing my most recent build, so I didn’t find out until a few days later.

if self.alt == true then
	local seq = string.match(action.text,"%d+")
	if seq ~= nil then
		-- if string.len(self.alt_seq) > 0 then
			-- local sp_char = string.len(self.chat)
			-- self.chat = string.sub(self.chat, 1, sp_char-1)
		-- end
		self.alt_seq = self.alt_seq .. seq
		self.chat = self.chat .. string.char(tonumber(self.alt_seq))
	end
...

I’ve tried multiple fonts each with “All Chars” enabled, and can see that the font contains them, yet the problem persists.

1 Like
2

This works for me:

local text = string.char(182)
label.set_text("#label", text)

It will set the label to ¶ if I have All Chars checked and ~ if I don’t.

Are you sure the font you are using includes the character you are trying to display?

3

I tried disabling it, saving, testing, enabling it, saving, and testing. It displays, with “All Chars” checkmarked, all the special characters such as “¶” in the engine. When testing, anything above the regular characters returned a tilde.

I’m using the built-in vera-mo-bd font for testing, although I’ve tried others. I tried testing with multiple different special characters, but none of them worked.

I even tried setting it straight to a number, instead of input, and got the same result.

Should I share the project with you?

4

Strangely enough, while doing a test using the non-numpad keys to enter the alt-sequence, I stumbled upon the special characters. The alt-sequence works on both, but the codes changed.

The numbers I was trying to use used to be within ASCII codes 130+, but now they skipped to 180+. Something changed… I am perplexed as to what happened, but at least it’s working.

5

Could you give me an example of one of the codes that aren’t working and what you expect it to output?

6

Absolutely. The code I used repeatedly when first testing was 130. It output “é” previously. This is even listed on many Alt-Code websites. I used one to test different special characters.

https://www.alt-codes.net/

7

Hmm, yes, there seems to be an issue with this now. @Erik_Angelin recently upgraded the font compiler in Editor 2. This is likely related to the problem you are seeing.

4 Likes
8

Ok, so it’s not the recent changes to the font compiler. Some additional information and a solution:

The site https://www.alt-codes.net/ does not show the same results as for instance https://www.rapidtables.com/code/text/ascii-table.html. It’s best to double check where you get the codes from. The latter one seems to be the one that is correct.

If you get length like below you might expect it to be 1 but it is in fact 2:

local s = "¶"
print(#s) -- 2

This indicates that there is more to this than simply doing like this:

local s = "\182"
print(#s) -- 1

Lua strings are a sequence of bytes, not encoded in any way. Strings can be used to represent any kind of binary data (a file, an image, a string of human readable characters).

In Defold we expect string passed to labels and text nodes to be UTF8 encoded. This means that if you want to use characters outside the ascii range (0-127) you need to ensure they are UTF8 encoded. This is done automatically if you set a string to “¶” but not if you do “\182”. There are many pure Lua UTF8 encoders out there, here’s one: https://gist.github.com/pygy/7154512#file-utf-8-lua-L37-L49

4 Likes
9

:face_with_raised_eyebrow: What could’ve caused the byte codes to change?

I actually encountered this when testing. It’s causing a number of issues, but I have a workaround in mind.

Ok, that’s fine for now. I’ll save the UTF8 encoder for the next step, if possible. Just to clarify: If I’m using the special characters as is (96-200+), the engine will have issues with the characters that go above the (0-127) range?

I tested with codes above 200 and everything appears to be working fine, besides the string length and one other issue. Here’s the other issue that I noticed:

%C3%A8%C3%A7%20error
%C3%A8%C3%A7%20error1200×675 92 KB

I was inputting “èç” repeatedly, and this is what rendered. As you can see, even in the debug log, there are two characters that were outputted incorrectly.

10

Are you sure something has changed? Download an old version of Defold and try. As far as I know the relevant code hasn’t changed in a long time.

This isn’t Defold specific. It’s how Lua works. Open a Lua command line interpreter and this is what you get:

$ lua
Lua 5.1  Copyright (C) 1994-2006 Lua.org, PUC-Rio
> local s = "¶" print(#s)
2
>

If you in the middle of a string add \182 (eg “foo\182bar”) and expect it to result in foo¶bar then yeah, that will not be the actual result, for reasons I explained in my previous post (Lua strings being a sequence of bytes and the label and text node expecting UTF8 encoded strings). For anything outside the 0-127 range where you want to add it as escaped bytes like above you need to ensure that what gets added to the string is UTF8 encoded.

2 Likes