Smooth Pixelart Scaling

Found this on reddit today and decided to adapt it for Defold. It would be useful for some kind of games where you want pixel art but don’t want the nearest neighbor jitter when scaling on non-powers of 2.

One change I had to make was sending the texture size as a constant (I think it would be better to send it as a vertex constant rather than a fragment constant?) as OpenGL ES 2.0 doesn’t have textureSize. So if you use this you’ll need to do this for each atlas. Or if all of your pixel art is in the same atlas then you can hard code it in your material as your atlas size changes.

SmoothPixelScaling.zip (3.8 KB)

https://www.pkeod.com/defold/SmoothPixelArtScaling/

I think Celeste uses a technique similar to this to make their pixels look smooth.

13 Likes

@britzl this could be added to the pixel art template as a rendering option. Render everything to a render target, then display it on a unitquad with this material and it should just work. It seems to be a viable alternative to forcing power of 2 scaling method.

2 Likes

@Pkeod How would it work? I render everything to the render target using nearest (enough to set this in game.project I think) and then when the quad is rendered I do that using the material/shader in your example?

1 Like

Yep!

I put it on a branch (https://github.com/britzl/lowrezjam-template/tree/upscale_materials) and used render.enable_material() to toggle material on a key press (key 1 and 2). I didn’t see any difference though, but that’s likely because the render script forces a square ratio?

1 Like

Yeah, it will only benefit from the free scaling sizes to smooth the nearest neighbor weirdness that can happen normally. What your example is doing currently is still very useful but what Celeste does is also attractive for certain projects which this seems to mimic. If you added ability to scale freely too as an option you would probably be able to see its effect. Also the texture size in the material needs to be updated in the render.constant_buffer after enabling. The hard coded values is 64x64 and needs to be the current size of the render target or atlas.

1 Like

Added some changes to the render script to add the constants and a snap toggle. You can see the impact the new material has by lowering the sharpening. You’ll need to add a key to toggle the snapping. I used F3 for testing it. If you disable the snapping while 1 is active you can see the jittery effects as you slowly scale the window.

I set the smoothing to 9.9 so it’s obvious that it works but it should be at 10.0 or higher. Maybe 20.0? Or less with some games maybe. When you scale the window with the snap toggled off and 2 active you can see where the nearest neighbor jittering is greatly smoothed out. You have to scale the window at pretty small increments to notice the jitter with this demo.

local WIDTH = 64
local HEIGHT = 64


local IDENTITY = vmath.matrix4()
local LOWREZ_PROJECTION = vmath.matrix4_orthographic(0, WIDTH, 0, HEIGHT, -1, 1)

function init(self)
	self.tile_pred = render.predicate({"tile"})
	self.gui_pred = render.predicate({"gui"})
	self.text_pred = render.predicate({"text"})
	self.particle_pred = render.predicate({"particle"})
	self.lowrez_pred = render.predicate({"lowrez"})
	self.controls_pred = render.predicate({"controls"})

	self.view = IDENTITY
	--self.projection = LOWREZ_PROJECTION

	-- render target buffer parameters
	local color_params = {
		format = render.FORMAT_RGBA,
		width = WIDTH,
		height = HEIGHT,
		min_filter = render.FILTER_NEAREST,
		mag_filter = render.FILTER_NEAREST,
		u_wrap = render.WRAP_CLAMP_TO_EDGE,
		v_wrap = render.WRAP_CLAMP_TO_EDGE
	}
	local depth_params = {
		format = render.FORMAT_DEPTH,
		width = WIDTH,
		height = HEIGHT,
		u_wrap = render.WRAP_CLAMP_TO_EDGE,
		v_wrap = render.WRAP_CLAMP_TO_EDGE
	}
	self.rt = render.render_target("lowrez", { [render.BUFFER_COLOR_BIT] = color_params, [render.BUFFER_DEPTH_BIT] = depth_params })

	local clear_color = vmath.vector4(0, 0, 0, 0)
	clear_color.x = sys.get_config("render.clear_color_red", 0)
	clear_color.y = sys.get_config("render.clear_color_green", 0)
	clear_color.z = sys.get_config("render.clear_color_blue", 0)
	clear_color.w = sys.get_config("render.clear_color_alpha", 0)
	self.clear_buffers = {
		[render.BUFFER_COLOR_BIT] = clear_color,
		[render.BUFFER_DEPTH_BIT] = 1,
		[render.BUFFER_STENCIL_BIT] = 0
	}

	self.upscale_material = hash("lowrez")
	self.scale_snap = true
end


local function clear(self, w, h)
	-- clear
	render.set_view(IDENTITY)
	render.set_projection(vmath.matrix4_orthographic(0, w, 0, h, -1, 1))
	render.set_depth_mask(true)
	render.set_stencil_mask(0xff)
	render.clear(self.clear_buffers)
end


local function draw_game(self)
	clear(self, render.get_window_width(), render.get_window_height())
	
	render.set_viewport(0, 0, WIDTH, HEIGHT)

	-- draw world (sprites, tiles, pfx etc)
	render.set_depth_mask(false)
	render.disable_state(render.STATE_DEPTH_TEST)
	render.disable_state(render.STATE_STENCIL_TEST)
	render.disable_state(render.STATE_CULL_FACE)
	render.enable_state(render.STATE_BLEND)
	render.set_blend_func(render.BLEND_SRC_ALPHA, render.BLEND_ONE_MINUS_SRC_ALPHA)
	render.set_view(self.view)
	render.set_projection(LOWREZ_PROJECTION)
	render.draw(self.tile_pred)
	render.draw(self.particle_pred)
	render.draw_debug3d()
	
	-- draw screen space gui
	render.set_view(vmath.matrix4())
	render.set_projection(LOWREZ_PROJECTION)
	render.enable_state(render.STATE_STENCIL_TEST)
	render.draw(self.gui_pred)
	render.draw(self.text_pred)
	render.disable_state(render.STATE_STENCIL_TEST)
end


local function draw_upscaled(self)
	-- calculate zoom
	local window_width = render.get_window_width()
	local window_height = render.get_window_height()
	local zoom = math.min(window_width / WIDTH, window_height / HEIGHT)
	if self.scale_snap then zoom = math.max(1, math.floor(zoom)) end

	-- positioning
	local width = WIDTH * zoom
	local height = HEIGHT * zoom
	local offsetx = (window_width - width) / 2
	local offsety = (window_height - height) / 2

	-- draw!
	render.set_viewport(offsetx, offsety, width, height)
	render.set_view(IDENTITY)
	render.set_projection(IDENTITY)
	render.enable_texture(0, self.rt, render.BUFFER_COLOR_BIT)
	local constants = render.constant_buffer()
	constants.sharpness = vmath.vector4(20.0, 1.0, 1.0, 1.0)
	constants.texture_size = vmath.vector4(WIDTH, HEIGHT, 1.0, 1.0)
	render.enable_material(self.upscale_material)
	render.draw(self.lowrez_pred, constants)
	render.disable_material()
	render.disable_texture(0, self.rt)
end


local function draw_controls(self)
	render.set_viewport(0, 0, render.get_window_width(), render.get_window_height())
	render.set_view(IDENTITY)
	render.set_projection(vmath.matrix4_orthographic(0, render.get_window_width(), 0, render.get_window_height(), -1, 1))

	render.enable_state(render.STATE_STENCIL_TEST)
	render.draw(self.controls_pred)
	--render.draw(self.text_pred)
	render.disable_state(render.STATE_STENCIL_TEST)
end

function update(self)
	clear(self,	render.get_window_width(), render.get_window_height())
	render.enable_render_target(self.rt)
	draw_game(self)
	render.disable_render_target(self.rt)
	draw_upscaled(self)
	draw_controls(self)
end

function on_message(self, message_id, message)
	if message_id == hash("clear_color") then
		self.clear_buffers[render.BUFFER_COLOR_BIT] = message.color
	elseif message_id == hash("set_view_projection") then
		self.view = message.view
	elseif message_id == hash("set_upscale_material") then
		print("set_upscale_material", message.material)
		self.upscale_material = message.material
	elseif message_id == hash("toggle_scale_snap") then
		if self.scale_snap then
			self.scale_snap = false
		else
			self.scale_snap = true
		end
	end
end
function on_input(self, action_id, action)
	if action_id and SET_MATERIAL_ACTIONS[action_id] then
		if action.pressed then
			print(SET_MATERIAL_ACTIONS[action_id])
			msg.post("@render:", "set_upscale_material", { material = SET_MATERIAL_ACTIONS[action_id] })
		end
	elseif action_id and (action.pressed or action.repeated) then
		self.actions[action_id] = true
	end
	if action_id == hash("key_f3") and action.released then
		msg.post("@render:", "toggle_scale_snap")
	end
end

A higher resolution than 64x64 may make the smoothness more noticeable as well. Celeste is 320x180.

Heya, was this work ever merged into the main lowrezjam template? I’m looking for a good Defold starter project for pixel art (coming from Pico 8 / fantasy console land, not quite wiling to give up the aesthetic just yet :stuck_out_tongue: )

Thanks!

2 Likes

The lowrezjam template is already great for pixel art games as is! There is a branch where this is implemented but you may not see much difference as by default the template snaps to good resolutions. This type of shader will make irregular pixel art scaling look good such as in games where you don’t snap all pixel art scaling to a grid/ you rotate pixel art.

1 Like

Just a heads up. I use the low res template in one of my projects and one thing that confused me first was that I couldn’t click things.

Because we scale the rendering of the game we also need to scale the input, I did this by creating a module (I simply called it input.lua)

input.lua

local screen_info = require "utils.screen_info"

local M = {}

local WIDTH = screen_info.RENDER_WIDTH
local HEIGHT = screen_info.RENDER_HEIGHT

function M.scale_action(action)
	local width = tonumber(sys.get_config("display.width"))
	local height = tonumber(sys.get_config("display.height"))
	action.x = (WIDTH / width) * action.x

	action.y = (HEIGHT / height) * action.y
	
	return action
end

return M

I put the values I wanted it to render at in a different module (I called it screen_info.lua)
screen_info.lua

local M = {}

M.RENDER_WIDTH = 144
M.RENDER_HEIGHT = 256

return M

I also used that module in the render script and changed the top lines

game.render_script

local screen_info = require "utils.screen_info"

local WIDTH = screen_info.RENDER_WIDTH
local HEIGHT = screen_info.RENDER_HEIGHT

Then if I want to do something with action.x or action.y I simply do
on_input example

function on_input(self, action_id, action)
	if action_id == hash("touch") then
		action = input.scale_action(action)
		button.on_input(action_id, action)
	end
end
6 Likes

I just wanted to ask, if there is any new, smarter way of getting a texture/atlas size that could be used in fp?

It might be possible to get texture size dynamically from resources but you still have to send it to the texture program.

Maybe it’s time to review upgrading GLES to a newer version too since iOS and Android requirements are being raised by the stores anyway. Many phones released since 2017 support 3.1 it seems like. Would have to look at market data too.

1 Like

I’m asking because I can’t figure out how to set a texture_size for a tilemap? :thinking:

Whatever the image size is of the image you set should be the size you use.

I mean, for sprites I use atlas size, but using size of a tilesource for tilemaps results in a blurry tilemap when I run the game and set that size, so I bet the size is wrong. The same result is when I try to calculate the tilemap size instead:

    if self.has_tilemap then
		if self.tilemap == msg.url() then self.tilemap = "#tilemap" end
		local x,y,w,h = tilemap.get_bounds(self.tilemap)
		local size = vmath.vector4(self.tiles_size * w, self.tiles_size * h, 0,0)
		print("SETTING TILEMAP", self.tilemap, size)
		tilemap.set_constant(self.tilemap, "texture_size", size)
	end

What’s an example of tile source image size? A square? It would probably be easier if it always was even if you have transparent space where no tile images are.

Tilesource is 320x192, there are transparent space tiles, calculated size of tilemap is 5168x656 (16x16 tiles * width and height respectively from get_bounds)

Edit: Read that wrong at first.

320x192 = 512x256

You should extend your image that much and use those sizes instead.

Edit2: Defold most likely is already extending your tiles texture to that size, since it needs to for options like extrusion. You could test assuming that size and see if it helps. A render debug / profiler too may be able to also tell you the exact size of the tiles texture that is actually being used.

1 Like

This could be a truth indeed, as assuming 512x256 resulted in a sharp image! Thanks! :slight_smile:

Going further, is it possible to apply one material for every sprite and material at once? For example, something like drawing everything on the other plane and apply just one material with that fp to this?

Yes, check this. https://github.com/britzl/lowrezjam-template/tree/upscale_materials

It is what Celeste does for smooth upscaling of its pixel art I think. Everything is rendered in low res, then upscaled with this kind of method by a render target material, then smooth distance field text is draw on top of that.

3 Likes