Cross layer traffic optimization for split xr

Abstract

An improved wireless split rendering system for displaying Extended Reality (XR) content is discussed. A rendering server and client head-mounted device (HMD) may communicate over a wireless medium, where communication control is given to a server application layer logic. This allows the server to use request pose information from the HMD only when needed for rendering, while preserving bandwidth on the wireless medium for transmitting frames of the rendered content. This reduces contention and improves channel efficiency.

What is claimed is: 
     
         1 . A method of rendering frames for display to a user at a client device, the method comprising:
 initiating a wireless connection with a server, wherein the wireless connection is a WiFi Reverse Direction Grant mode connection and the server is an Access Point (AP) on a WiFi network;   determining a pose information of the user;   responsive to receiving a trigger packet from the server, transmitting the pose information to the server over the wireless connection, wherein the trigger packet indicates permission from the server to transmit during the server's transmit opportunity (TXOP), wherein the server provides permission to transmit based on at least one of: an amount of time needed to render the frame, a Vsync information from a previous pose information, and an arrival time of the previous pose information;   receiving the rendered frame from the server over the wireless connection, the rendered frame rendered for display based on the transmitted pose information; and   displaying the received rendered frame to the user.   
     
     
         2 . The method of  claim 1 , wherein the transmitted pose information is a 6 degrees of freedom (6 DoF) pose. 
     
     
         3 . The method of  claim 2 , further comprising:
 executing asynchronous time warping (ATW) on the received rendered frame based on a latest pose information before displaying to the user.   
     
     
         4 . The method of  claim 1 , further comprising:
 transmitting the pose information after a predetermined period of time if no trigger packet is received.   
     
     
         5 . The method of  claim 1 , wherein the rendered frame is a first view of a three-dimensional Virtual Reality (VR) or Augmented Reality (AR) content for display to the user. 
     
     
         6 . The method of  claim 1 , wherein the trigger packet is transmitted responsive to determining there is no rendered frame to transmit from the server. 
     
     
         7 . The method of  claim 1 , wherein the trigger packet is transmitted responsive to needing the pose information to render the frame at the server. 
     
     
         8 . The method of  claim 7 , wherein the trigger packet is initiated by an application layer module executing on the server. 
     
     
         9 . An apparatus for rendering frames for display to a user, the apparatus comprising:
 a connection processor for initiating a wireless connection with a server and receive a trigger packet, wherein the wireless connection is a WiFi Reverse Direction Grant mode connection and the server is an Access Point (AP) on a WiFi network;   a sensor for determining a pose information of the user;   a processor configured to,   responsive to receiving the trigger packet from the server, transmit the pose information to the server over the wireless connection, wherein the trigger packet indicates permission from the server to transmit during the server's transmit opportunity (TXOP), wherein the server provides permission to transmit based on at least one of: an amount of time needed to render the frame, a Vsync information from a previous pose information, and an arrival time of the previous pose information,   receive the rendered frame from the server over the wireless connection, the rendered frame rendered for display based on the transmitted pose information; and   a display for displaying the received rendered frame to the user.   
     
     
         10 . The apparatus of  claim 9 , wherein the transmitted pose information is a 6 degrees of freedom (6 DoF) pose. 
     
     
         11 . The apparatus of  claim 10 , wherein the processor further configured to execute asynchronous time warping (ATW) on the received rendered frame based on a latest pose information before displaying to the user. 
     
     
         12 . The apparatus of  claim 9 , the processor further configured to transmit the pose information after a predetermined period of time if no trigger packet is received. 
     
     
         13 . The apparatus of  claim 9 , wherein the rendered frame is a first view of a three-dimensional Virtual Reality (VR) or Augmented Reality (AR) content for display to the user. 
     
     
         14 . The apparatus of  claim 9 , wherein the trigger packet is transmitted responsive to determining there is no rendered frame to transmit. 
     
     
         15 . The apparatus of  claim 9 , wherein the trigger packet is transmitted responsive to needing the pose information to render the frame. 
     
     
         16 . The apparatus of  claim 15 , wherein the trigger packet is initiated by an application layer module executing on the server. 
     
     
         17 . The apparatus of  claim 9 , wherein the sensor includes at least one of: a gyroscope and an accelerometer. 
     
     
         18 . The apparatus of  claim 9 , wherein the apparatus is at least one of: head-worn display or a head-mounted display (HMD). 
     
     
         19 . A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors to:
 initiating a wireless connection with a server, wherein the wireless connection is a WiFi Reverse Direction Grant mode connection and the server is an Access Point (AP) on a WiFi network;   determining a pose information of the user;   responsive to receiving a trigger packet from the server, transmitting the pose information to the server over the wireless connection, wherein the trigger packet indicates permission from the server to transmit during the server's transmit opportunity (TXOP), wherein the server provides permission to transmit based on at least one of: an amount of time needed to render the frame, a Vsync information from a previous pose information, and an arrival time of the previous pose information;   receiving the rendered frame from the server over the wireless connection, the rendered frame rendered for display based on the transmitted pose information; and   displaying the received rendered frame to the user.   
     
     
         20 . The medium of  claim 19 , wherein the transmitted pose information is a 6 degrees of freedom (6 DoF) pose. 
     
     
         21 . The medium of  claim 19 , wherein the rendered frame is a first view of a three-dimensional Virtual Reality (VR) or Augmented Reality (AR) content for display to the user. 
     
     
         22 . The medium of  claim 19 , wherein the trigger packet is transmitted responsive to determining there is no rendered frame to transmit. 
     
     
         23 . The medium of  claim 19 , wherein the trigger packet is transmitted responsive to needing the pose information to render the frame. 
     
     
         24 . The medium of  claim 19 , wherein the trigger packet is initiated by an application layer module executing on the server. 
     
     
         25 . An apparatus for rendering frames for display to a user, the apparatus comprising:
 a means for a wireless connection with a server and receive a trigger packet, wherein the wireless connection is a WiFi Reverse Direction Grant mode connection and the server is an Access Point (AP) on a WiFi network;   a means for sensing a pose information of the user;   a means for processing, the processing means configured to,   responsive to receiving the trigger packet from the server, transmit the pose information to the server over the wireless connection, wherein the trigger packet indicates permission from the server to transmit during the server's transmit opportunity (TXOP), wherein the server provides permission to transmit based on at least one of: an amount of time needed to render the frame, a Vsync information from a previous pose information, and an arrival time of the previous pose information,   receive the rendered frame from the server over the wireless connection, the rendered frame rendered for display based on the transmitted pose information; and   a means for display, the display means displaying the received rendered frame to the user.   
     
     
         26 . The apparatus of  claim 25 , wherein the transmitted pose information is a 6 degrees of freedom (6 DoF) pose. 
     
     
         27 . The apparatus of  claim 26 , wherein the processing means is further configured to execute asynchronous time warping (ATW) on the received rendered frame based on a latest pose information before displaying to the user. 
     
     
         28 . The apparatus of  claim 25 , the processing means further configured to transmit the pose information after a predetermined period of time if no trigger packet is received. 
     
     
         29 . The apparatus of  claim 25 , wherein the rendered frame is a first view of a three-dimensional Virtual Reality (VR) or Augmented Reality (AR) content for display to the user. 
     
     
         30 . The apparatus of  claim 29 , wherein the trigger packet is transmitted responsive to determining there is no rendered frame to transmit or needing the pose information to render the frame.