0

Description

"## Introduction\r\n\r\nSetting a compute shader can be a tedious task, to say the least. The following small code is meant to ease the process. Add the main script of this page as a Global script and name it `ShaderUtil` (or whatever you want, as long as it is not `ComputeShader` or `ShaderBuffer`).\r\n\r\n## Example\r\n\r\nLet's start with a random example. First, a compute shader.\r\n\r\n```glsl\r\n#[compute]\r\n#version 450\r\n\r\nlayout(local_size_x = 16, local_size_y = 16, local_size_z = 1) in;\r\n\r\nlayout(set = 0, binding = 0, std140) uniform Size {\r\n\tint width;\r\n\tint height;\r\n};\r\n\r\nlayout(set = 0, binding = 1, rgba32f) uniform restrict writeonly image2D occupancy_mask;\r\n\r\nvoid main() {\r\n\tif (gl_GlobalInvocationID.x >= width || gl_GlobalInvocationID.y >= height) {\r\n\t\treturn;\r\n\t}\r\n\r\n\timageStore(occupancy_mask, ivec2(gl_GlobalInvocationID.xy),\r\n\t\t\tvec4(\r\n\t\t\t\t0.,\r\n\t\t\t\t1. - (gl_GlobalInvocationID.x \/ float(width)),\r\n\t\t\t\t1. - (gl_GlobalInvocationID.y \/ float(height)),\r\n\t\t\t\t1.)\r\n\t);\r\n}\r\n```\r\n\r\nThis shader will simply put a gradient coloring on the texture via the blue and red channel.\r\n\r\nThe driving script using our utility classes is made as-is :\r\n\r\n```gdscript\r\nextends Node3D\r\n\r\n@export var floor_mesh : MeshInstance3D\r\n\r\n@export var occupancy_map_size : int = 2048\r\n\r\n@onready var mask_shader : ShaderUtil.ComputeShader\r\n\r\n@onready var rendering_device : RenderingDevice = RenderingServer.get_rendering_device()\r\n\r\n@onready var shader_texture : Texture2DRD = Texture2DRD.new()\r\n\r\n@onready var shader_size : ShaderUtil.ShaderBuffer\r\n\r\n@onready var occupancy_map : ShaderUtil.ShaderBuffer\r\n\r\nfunc _ready() -> void:\r\n\tassert(floor_mesh != null and floor_mesh.mesh != null and floor_mesh.mesh is PlaneMesh)\r\n\r\n\tshader_size = ShaderUtil.create_uniform_buffer(rendering_device, 2)\r\n\tshader_size.build_empty_cache()\r\n\tshader_size.set_parameter_i(0, occupancy_map_size)\r\n\tshader_size.set_parameter_i(1, occupancy_map_size)\r\n\tshader_size.sync(rendering_device) # Don't forget to sync to send info to GPU !!\r\n\r\n\toccupancy_map = ShaderUtil.create_hires_image_buffer(rendering_device, Vector2i(occupancy_map_size, occupancy_map_size))\r\n\r\n\tmask_shader = ShaderUtil.ComputeShader.new(\"Occupancy_Mask\", \"res:\/\/experiments\/experiment_1_occupancy_shader.glsl\", rendering_device, [ shader_size, occupancy_map ])\r\n\r\n\tassert((floor_mesh.mesh as PlaneMesh).material != null)\r\n\r\n\tmask_shader.run_deffered(Vector3i(occupancy_map_size \/ 16, occupancy_map_size \/ 16, 1))\r\n\r\n\tshader_texture.texture_rd_rid = occupancy_map.rid\r\n\r\n\t((floor_mesh.mesh as PlaneMesh).material as ShaderMaterial).set_shader_parameter(\"occupancy_texture\", shader_texture)\r\n```\r\n\r\nThe `floor_mesh` is just a `MeshInstance3D` with a `PlaneMesh` and a simple shader that take a `sampler2D` as uniform.\r\n```\r\nshader_type spatial;\r\n\r\nuniform sampler2D occupancy_texture;\r\n\r\nvoid fragment() {\r\n\tALBEDO = texture(occupancy_texture, UV).rgb;\r\n}\r\n```\r\n\r\nWhat have we done here ?\r\nFirst, we declared the shader buffers. One to convey information about the texture size. Using the utility function, we set the values before sending them to the GPU using the `sync` function (don't forget that ! That's one of the most common mistake).\r\n\r\nThen, we declare a texture. For now, the instanced textures are RGBA32F formatted. It's the most general purpose textures as they can convey a lot of information. But they are not the most efficient ones (as they consume a TONS of VRAM).\r\n\r\nOnce we got all our shader buffers, we can create the shader by specifying the path to the shader code, the `RenderingDevice` on which the computation will occur, and the ORDERED list of the shader buffers. If you look closely at the compute shader code, you will notice the `binding` keyword along with a number. This number is a binding identifier. In order to link the correct shader buffer to the declared data, you will need to fill the array with the created `ShaderBuffer` at the correct indexes.\r\n\r\n## Classes\r\n\r\nThe compute shader utility relies on the following classes.\r\n\r\n> [!NOTE]\r\n> This is a very basic use-case and should be adapted\/enhanced to your needs. A more complete set of utilities will be released in a near future.\r\n\r\n### ShaderBuffer\r\n\r\nA shader buffer is a VRAM space manipulated by a compute shader. It can be of multiple nature. In our case, we only deal with uniforms, buffers and textures.\r\nIt is advised to use the following function to create a `ShaderBuffer`.\r\n\r\n* `ShaderBuffer` **create_hires_image_buffer** (rd : `RenderingDevice`, sz : `Vector2i`) : Creates a shader buffer that is a texture of format `RGBA32F`. You'll need to specify a `RenderingDevice` and a size.\r\n* `ShaderBuffer` **create_uniform_buffer** (rd : `RenderingDevice`, parameters_count : `int`, bytes : `PackedByteArray = PackedByteArray()`) : Create a simple uniform list of parameters. In the shader itself, this uniform can have a structure, but it is not reflected here. We only take care of the number of parameters. It is advised to consider parameters as either integers of simple precision float (as long as the parameter is 32 bits long). An extra parameter can be provided to set the initial data to be sent\/stored to\/in the graphic card.\r\n* `ShaderBuffer` **create_storage_buffer** (rd : `RenderingDevice`, struct_size : `int`, element_count : `int`, bytes : `PackedByteArray = PackedByteArray()`) : A storage buffer can be considered as an array of uniform (see `create_uniform_buffer`). Specify the number of parameters\/members `struct_size` in the structure to be arrayed (of size `element_count`).\r\n\r\n> [!NOTE]\r\n> `ShaderBuffer` comes with handy information for further use, like the `rid` of the resource. In case of texture, this can be used to tie the texture to a `Texture2DRD`.\r\n\r\n> [!CAUTION]\r\n> When you modify the content of a shader buffer, never forget to invoke the `sync` method to send back the data to the graphic card.\r\n\r\n`ShaderBuffer` class comes with the following API:\r\n* **build_empty_cache** : Create an empty cache that can be used to receive and send data to the graphic card. By default, the cache is not created (for memory performance issue).\r\n* **set_cache** : Instead of setting an empty cache, you can specify a preallocated cache. This can be useful in some (dark magic) cases where you share cache between several shader buffer or perform some double\/triple buffering.\r\n* **sync(rd : `RenderingDevice`)** : Update the shader buffer by sending cached info to the graphic card.\r\n\r\n### ComputeShader\r\n\r\nThe `ComputeShader` class is the heart of the utility system. It will gather the declared `ShaderBuffer`, compile and link the compute shader. It will then be possible to invoke it.\r\n\r\nIts API is the following:\r\n\r\n* **ComputeShader(name : `String`, path_to_shader : `String`, rd : `RenderingDevice`, uniforms : `Array[ShaderBuffer]`)**\r\n\t* _name_ : A simple human readable identifier (not currently used)\r\n\t* _path_to_shader_ : Path to the shader file. Can be a resource file (`res:\/\/`) or any file on the file system.\r\n\t* _rd_ : The `RenderingDevice` on which the compute shader will run. Be careful to use the same `RenderingDevice` that you used for the `ShaderBuffer` you will incorporate into your compute shader definition.\r\n\t* _uniforms_ : Array of `ShaderBuffer`. As explained above, the order of the shader buffers is critical as it matches the `binding` in the compute shader.\r\n* **run(dispatch : `Vector3i`)** : Run the shader. `dispatch` is the number of invocations on the 3 axises. Check the [Godot Documentation](https:\/\/docs.godotengine.org\/fr\/4.x\/tutorials\/shaders\/compute_shaders.html) for more information."

Main code

extends Node

# Utility class to ease buffer registering
class ShaderBuffer:
	var rid : RID
	var type : RenderingDevice.UniformType
	var size : int
	var cache : PackedByteArray

	func _init(r : RID, t : RenderingDevice.UniformType, s : int, c : PackedByteArray = PackedByteArray()) -> void:
		rid = r
		type = t
		size = s
		cache = c
		if not cache.is_empty():
			assert(cache.size() == s)

	func set_parameter_vi(value : Vector2i) -> void:
		cache.encode_s32(0, value.x)
		cache.encode_s32(4, value.y)

	func set_parameter(idx : int, value : float) -> void:
		var pos : int = idx * 4
		assert(pos < cache.size())
		cache.encode_float(pos, value)

	func set_parameter_i(idx : int, value : int) -> void:
		var pos : int = idx * 4;
		assert(pos < cache.size())
		cache.encode_u32(pos, value)

	func copy_parameter(src : int, dst : int) -> void:
		var s : int = src * 4; var d : int = dst * 4
		assert(s < cache.size() and d < cache.size())
		cache[d] = cache[s]
		cache[d + 1] = cache[s + 1]
		cache[d + 2] = cache[s + 2]
		cache[d + 3] = cache[s + 3]

	func set_parameter_v(values : PackedFloat32Array) -> void:
		assert(values.size() <= cache.size() * 4)
		for i : int in range(values.size()):
			cache.encode_float(i * 4, values[i])

	func build_empty_cache() -> void:
		cache = PackedByteArray()
		cache.resize(size)
		cache.fill(0)

	func set_cache(c : PackedByteArray) -> void:
		assert(c.size() == size)
		cache = c

	func sync(rd : RenderingDevice) -> void:
		if not cache.is_empty():
			rd.buffer_update(rid, 0, cache.size(), cache)

	func clear(rd : RenderingDevice, clearing_buf : PackedByteArray) -> void:
		rd.buffer_update(rid, 0, size, clearing_buf)

	func print_content(count : int) -> void:
		if not cache.is_empty():
			for i : int in range(count):
				print("#%d : %f" % [i , cache.decode_float(i * 4)])

## Compute Shader General Purpose structure.
class ComputeShader:
	var ios : Array[ShaderBuffer]
	var shader : RID
	var uniform_set : RID
	var pipeline : RID

	var identifier : String
	var path_to_source : String

	var rendering_device : RenderingDevice

	func run_deffered(dispatch : Vector3i) -> void:
		RenderingServer.call_on_render_thread(run.bind(dispatch))

	func run(dispatch : Vector3i) -> void:
		assert(rendering_device != null)
		var render_list : int = rendering_device.compute_list_begin()
		assert(pipeline != null and pipeline.is_valid())
		rendering_device.compute_list_bind_compute_pipeline(render_list, pipeline)
		assert(uniform_set != null and uniform_set.is_valid())
		rendering_device.compute_list_bind_uniform_set(render_list, uniform_set, 0)
		@warning_ignore("integer_division")
		rendering_device.compute_list_dispatch(render_list, dispatch.x, dispatch.y, dispatch.z)
		rendering_device.compute_list_end()

	func _init(n : String, p : String, rd : RenderingDevice, uniforms : Array[ShaderBuffer]) -> void:
		identifier = n
		path_to_source = p
		ios = uniforms
		rendering_device = rd
		shader = _init_shader(path_to_source)
		_link_shader()

	func _init_shader(path : String) -> RID:
		var shader_file : RDShaderFile = load(path)
		assert(shader_file != null)
		var shader_spirv : RDShaderSPIRV = shader_file.get_spirv()
		if shader_spirv.compile_error_compute != "":
			push_error(shader_spirv.compile_error_compute)
			assert(false)
		return rendering_device.shader_create_from_spirv(shader_spirv)

	func _link_shader() -> void:
		var uniforms : Array[RDUniform] = []
		for i : int in range(ios.size()):
			_register_uniform_in_set(ios[i], i, uniforms)
		uniform_set = rendering_device.uniform_set_create(uniforms, shader, 0)
		pipeline = rendering_device.compute_pipeline_create(shader)

	func _register_uniform_in_set(buffer : ShaderBuffer, binding : int, uniform_list : Array[RDUniform]) -> void:
		var uniform : RDUniform = RDUniform.new()
		uniform.uniform_type = buffer.type
		uniform.binding = binding
		uniform.add_id(buffer.rid)
		uniform_list.append(uniform)

## Shader Buffer creation functions for several kind of buffers. Image, storage, uniform.

func create_hires_image_buffer(rd : RenderingDevice, sz : Vector2i) -> ShaderBuffer:
	var format : RDTextureFormat = RDTextureFormat.new()
	format.width = sz.x
	format.height = sz.y
	format.depth = 1
	format.format = RenderingDevice.DATA_FORMAT_R32G32B32A32_SFLOAT
	format.texture_type = RenderingDevice.TEXTURE_TYPE_2D
	format.array_layers = 1
	format.mipmaps = 1
	format.usage_bits = RenderingDevice.TEXTURE_USAGE_SAMPLING_BIT + RenderingDevice.TEXTURE_USAGE_COLOR_ATTACHMENT_BIT + RenderingDevice.TEXTURE_USAGE_STORAGE_BIT + RenderingDevice.TEXTURE_USAGE_CAN_UPDATE_BIT + RenderingDevice.TEXTURE_USAGE_CAN_COPY_TO_BIT
	var init_data : PackedFloat32Array = PackedFloat32Array()
	init_data.resize(sz.x * sz.y * 4)
	init_data.fill(0.0)
	# Debug fill. It should display a mix of red and green gradient depending on UV.
	for i : int in range(sz.y):
		for j : int in range(sz.x):
			var k : int = i * sz.x + j
			var idx : int = k * 4
			init_data[idx + 0] = (i / float(sz.x))
			init_data[idx + 1] = (j / float(sz.y))
			init_data[idx + 3] = 1.0
	return ShaderBuffer.new(rd.texture_create(format, RDTextureView.new(), [ init_data.to_byte_array() ]), RenderingDevice.UNIFORM_TYPE_IMAGE, sz.x * sz.y * 16)

func create_uniform_buffer(rd : RenderingDevice, parameters_count : int, bytes : PackedByteArray = PackedByteArray()) -> ShaderBuffer:
	# we count each parameter as a 32bit float. However, we will assume that we align on vec4.
	var rounded_size : int = ceil(parameters_count / 4.0) * 16 # 4 * 32bit components
	return ShaderBuffer.new(rd.uniform_buffer_create(rounded_size, bytes), RenderingDevice.UNIFORM_TYPE_UNIFORM_BUFFER, rounded_size, bytes)

func create_storage_buffer(rd : RenderingDevice, struct_size : int, element_count : int, bytes : PackedByteArray = PackedByteArray()) -> ShaderBuffer:
	var rounded_size : int = ceil(struct_size / 4.0) * 16
	var total_size : int = rounded_size * element_count
	return ShaderBuffer.new(rd.storage_buffer_create(total_size, bytes), RenderingDevice.UNIFORM_TYPE_STORAGE_BUFFER, total_size)

Comments