////
- Copyright (c) 2019-2024, Arm Limited and Contributors
-
- SPDX-License-Identifier: Apache-2.0
-
- Licensed under the Apache License, Version 2.0 the "License";
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
-     http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
-
////
= Appropriate use of surface rotation

ifdef::site-gen-antora[]
TIP: The source for this sample can be found in the https://github.com/KhronosGroup/Vulkan-Samples/tree/main/samples/performance/surface_rotation[Khronos Vulkan samples github repository].
endif::[]


== Overview

Mobile devices can be rotated, therefore the logical orientation of the application window and the physical orientation of the display may not match.
Applications then need to be able to operate in two modes: portrait and landscape.
The difference between these two modes can be simplified to just a change in resolution.
However, some display subsystems always work on the "native" (or "physical") orientation of the display panel.
Since the device has been rotated, to achieve the desired effect the application output must also rotate.

In OpenGL ES the GPU driver can transparently handle the logical rotation of window surface framebuffers, but the Vulkan specification has made this explicit in the API.
Therefore in Vulkan the application is responsible for supporting rotation.

In this sample we focus on the rotation step, and analyse the performance implications of implementing it correctly with Vulkan.

image::./images/rotation_cases.jpg[Pre-rotation]

== Pre-rotation

The rotation step can be carried out in different ways:

* It can efficiently and transparently be handled in hardware by the Display Processing Unit (DPU), but this is only possible in those devices that support it.
* It can be handled by Android, by introducing a compositor pass that rotates the output using the GPU, or some other dedicated block.
This is transparent to the application, but will have additional system-level costs such as extra memory bandwidth, or even GPU processing if the compositor uses the GPU as the rotation engine.
* It can be handled by the Vulkan application, by rendering into a window surface which is oriented to match the physical orientation of the display panel.
We call this pre-rotation.

An application has no means to tell whether the current device can support a free rotation during composition, so the only guaranteed method to avoid any additional processing cost is to render into a window surface which is oriented to match the physical orientation of the display panel, thus removing the Android compositor step.

== Demo application

The sample application you can find here shows how you can handle rotations in your Vulkan application in a way that avoids using the Android compositor for rotation.
It allows you to enable and disable pre-rotation at run time, so you can compare these two modes using the hardware counters shown on the display.
In this section we will go through the code required to carry out pre-rotation.
In the analysis section below we will explain the differences in more detail.
Note that not all devices will show obvious differences, as more and more include a DPU capable of performing the rotation in hardware.

In a nutshell, below are the steps required to handle pre-rotation:

|===
| No pre-rotation | Pre-rotation

| Destroy the Vulkan framebuffers and the swapchain
| Destroy the Vulkan framebuffers and the swapchain

| Re-create the swapchain using the new surface dimensions i.e.
the swapchain dimensions match the surface's.
Ignore the `preTransform` field in https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VkSwapchainCreateInfoKHR.html[`VK_STRUCTURE_TYPE_SWAPCHAIN_CREATE_INFO_KHR`].
This will not match the value returned by https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/vkGetPhysicalDeviceSurfaceCapabilitiesKHR.html[`vkGetPhysicalDeviceSurfaceCapabilitiesKHR`] and therefore the Android Compositor will rotate the scene before presenting it to the display
| Re-create the swapchain using the old swapchain dimensions, i.e.
the swapchain dimensions do not change.
Update the `preTransform` field in https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VkSwapchainCreateInfoKHR.html[`VK_STRUCTURE_TYPE_SWAPCHAIN_CREATE_INFO_KHR`] so that it matches the `currentTransform` field of the https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VkSurfaceCapabilitiesKHR.html[`VkSurfaceCapabilitiesKHR`] returned by the new surface.
This communicates to Android that it does not need to rotate the scene.

| Re-create the framebuffers
| Re-create the framebuffers

| n/a
| Adjust the MVP matrix so that the world is rotated

| Render the scene
| Render the scene
|===

== Rotation in Android

Android added pre-rotation to their https://developer.android.com/ndk/guides/graphics/design-notes[Vulkan Design Guidelines].
However, https://developer.android.com/guide/topics/resources/runtime-changes#HandlingTheChange[by default], Android calls `onDestroy` when the screen is rotated.
To disable this behavior and handle rotations in Vulkan, you must add the `orientation` (for API level 13 and lower) and `screenSize` attributes to the activity's `configChanges` in the Android manifest:

----
<activity android:name=".BPNativeActivity"
          android:configChanges="orientation|screenSize">
----

To track orientation changes, use Android's `APP_CMD_CONTENT_RECT_CHANGED` event:

----
void on_app_cmd(android_app *app, int32_t cmd)
{
	auto platform = reinterpret_cast<AndroidPlatform *>(app->userData);
	assert(platform && "Platform is not valid");

	switch (cmd)
	{
		case APP_CMD_INIT_WINDOW:
		{
			platform->get_window().resize(ANativeWindow_getWidth(app->window), ANativeWindow_getHeight(app->window));
			app->destroyRequested = !platform->prepare();
			break;
		}
		case APP_CMD_CONTENT_RECT_CHANGED:
		{
			// Get the new size
			auto width  = app->contentRect.right - app->contentRect.left;
			auto height = app->contentRect.bottom - app->contentRect.top;
			platform->resize(width, height);
			break;
		}
	}
}
----

Note that android NDK 21 and below does not support `APP_CMD_CONTENT_RECT_CHANGED`.
This is fixed in version 22+ which at the date of this addition has not been released.
This can be patched in the short term by registering a callback that mimics the expected behavior of `OnContentRectChanged`.
The callback we use in this project is:

----
void on_content_rect_changed(ANativeActivity *activity, const ARect *rect)
{
	... log content rect dimensions
	
	struct android_app *app = reinterpret_cast<struct android_app *>(activity->instance);
	auto                cmd = APP_CMD_CONTENT_RECT_CHANGED;

	app->contentRect = *rect;

	if (write(app->msgwrite, &cmd, sizeof(cmd)) != sizeof(cmd))
	{
		... failed to write message to the app glue command stream
	}
}
----

The above triggers the `APP_CMD_CONTENT_RECT_CHANGED` event and the new content rect dimensions should be accessible by the `+app->contentRect+` attribute.

There are other alternatives to `CONTENT_RECT_CHANGED`, an example can be found here https://android-developers.googleblog.com/2020/02/handling-device-orientation-efficiently.html[Handling Device Orientation Efficiently in Vulkan With Pre-Rotation]

== Swapchain re-creation

We need to sample the current transform from the surface:

----
VkSurfaceCapabilitiesKHR surface_properties;
VK_CHECK(vkGetPhysicalDeviceSurfaceCapabilitiesKHR(get_device().get_physical_device(),
                                                   get_surface(),
                                                   &surface_properties));

pre_transform = surface_properties.currentTransform;
----

`currentTransform` is a https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/chap32.html#VkSurfaceTransformFlagBitsKHR[`VkSurfaceTransformFlagBitsKHR`] value.
When we re-create the swapchain, we must set the swapchain's https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VkSwapchainCreateInfoKHR.html[`preTransform`] to match this value.
This informs the compositor that the application has handled the required transform so it does not have to.

To re-create the swapchain, the sample uses the helper function `update_swapchain` provided by the framework:

----
get_device().wait_idle();

auto surface_extent = get_render_context().get_surface_extent();

get_render_context().update_swapchain(surface_extent, select_pre_transform());
----

This function then takes care to safely destroy the framebuffers and use the new `preTransform` value to re-create the swapchain:

----
device.get_resource_cache().clear_framebuffers();

auto width  = extent.width;
auto height = extent.height;
if (transform == VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR || transform == VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR)
{
	// Pre-rotation: always use native orientation i.e. if rotated, use width and height of identity transform
	std::swap(width, height);
}

swapchain = std::make_unique<Swapchain>(*swapchain, VkExtent2D{width, height}, transform);
----

Note that if pre-rotation is enabled and the application has been rotated by 90 degrees, then the surface dimensions must be swapped with respect to the previous orientation.
This is done to preserve the dimensions of the swapchain images, since we are planning to rotate our geometry accordingly.

The framework then takes care to re-create the framebuffers.

== Rotating the scene

When rotating our geometry, normally all we need to do is adjust the Model View Projection (MVP) matrix that we provide to the vertex shader every frame.
In this case we want to rotate the scene just before applying the projection transformation.
Therefore we update the matrix that the camera will use to compute the projection matrix:

----
glm::mat4   pre_rotate_mat = glm::mat4(1.0f);
glm::vec3   rotation_axis  = glm::vec3(0.0f, 0.0f, 1.0f);
const auto &swapchain      = get_render_context().get_swapchain();

if (swapchain.get_transform() & VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR)
{
	pre_rotate_mat = glm::rotate(pre_rotate_mat, glm::radians(90.0f), rotation_axis);
}
else if (swapchain.get_transform() & VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR)
{
	pre_rotate_mat = glm::rotate(pre_rotate_mat, glm::radians(270.0f), rotation_axis);
}
else if (swapchain.get_transform() & VK_SURFACE_TRANSFORM_ROTATE_180_BIT_KHR)
{
	pre_rotate_mat = glm::rotate(pre_rotate_mat, glm::radians(180.0f), rotation_axis);
}

camera->set_pre_rotation(pre_rotate_mat)
----

The camera stores this pre-rotation matrix.
This way the framework will use the updated matrix before pushing the MVP to the shader:

----
void GeometrySubpass::update_uniform(CommandBuffer &command_buffer, sg::Node &node, size_t thread_index)
{
	GlobalUniform global_uniform;

	global_uniform.camera_view_proj = camera.get_pre_rotation() * vkb::vulkan_style_projection(camera.get_projection()) * camera.get_view();
----

For completion, here are the relevant sections of the vertex shader:

----
layout(location = 0) in vec3 position;

layout(set = 0, binding = 1) uniform GlobalUniform {
    mat4 model;
    mat4 view_proj;
    vec3 camera_position;
} global_uniform;

layout (location = 0) out vec4 o_pos;

void main(void)
{
    o_pos = global_uniform.model * vec4(position, 1.0);
    gl_Position = global_uniform.view_proj * o_pos;;
}
----

== Performance impact

The `surface_rotation` Vulkan sample allows you to toggle between pre-rotation mode and compositor mode.
Below is a screenshot of the sample running on a device that does not support native (DPU) rotation, but instead includes a separate 2D block which rotates the GPU output before presenting it to the display.

image::./images/android_compositor.jpg[Android compositor handling the rotation]

Compare this to the same scene rendered using pre-rotation:

image::./images/android_prerotation.jpg[Pre-rotating the scene]

As you can see there is a significant increase in the stall rate on the external memory bus if pre-rotation is not enabled, because the framebuffer is being read and written to the 2D rotation block.
For this device the additional system memory bandwidth generated by the 2D block increases the use of external memory, which is visible as an increase in memory back-pressure seen by the GPU.

This is more obvious if we trace both modes using Streamline.
If you enable all Mali counters and use the relevant template (Mali-G72 in this case) to visualize the data, we can see that we go from an average 12% read stall / 7% write stall to 22% read stall / 17% write stall.
In the image below pre-rotation is enabled and disabled every second (using the auto-toggle option).
The absolute traffic per cycle drops, but this is because of the drop in performance associated to the increased memory pressure.

image::./images/prerotate_streamline.png[Streamline capture.
Pre-rotate is enabled/disabled every second]

In this case the 2D rotation block is using a significant portion of the bandwidth, causing a drop in performance.
Note however that this scene is rendered in a memory-heavy fashion (no culling, no compressed textures) to make the effect of pre-rotation more visible.
Even if your scene is not memory-heavy, the extra load on the system resulting from performing the rotation during composition will have a negative impact on the battery life of the device.

In order to save battery life in those devices without a rotation-capable DPU, always ensure that your Vulkan renderer performs pre-rotation.

== Best-practice summary

*Do*

* To avoid presentation engine transformation passes ensure that swapchain https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VkSwapchainCreateInfoKHR.html[`preTransform`] matches the `currentTransform` value returned by https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/vkGetPhysicalDeviceSurfaceCapabilitiesKHR.html[`vkGetPhysicalDeviceSurfaceCapabilitiesKHR`].
* If a swapchain image acquisition returns https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VkResult.html[`VK_SUBOPTIMAL_KHR`] or https://www.khronos.org/registry/vulkan/specs/1.1-extensions/man/html/VkResult.html[`VK_ERROR_OUT_OF_DATE_KHR`] then recreate the swapchain taking into account any updated surface properties including potential orientation updates reported via `currentTransform`.

*Don't*

* Assume that supported presentation engine's transforms other than `currentTransform` are free;
many presentation engines can handle rotation and/or mirroring but at additional processing cost.
Note that Android will always return all transforms as supported, because the GPU is always available as a general purpose fallback.

*Impact*

* Non-native orientation may require additional transformation passes in the presentation engine.
This may require use of the GPU or a dedicated 2D block on some systems which cannot handle the transformation directly in the display controller.

*Debugging*

* You may use a system profiler such as Streamline to spot extra memory loads in the GPU counters, either as a direct effect (GPU composition) or as a side-effect (memory pressure).
s a direct effect (GPU composition) or as a side-effect (memory pressure).
