Systems and methods using image recognition processes and determined device orientation for improved operation of a laundry appliance

Abstract

A method of operating a washing machine appliance may include obtaining one or more images of the washing machine appliance from a camera assembly of a remote device spaced apart from the cabinet. The method may also include receiving a plurality of angle readings from the remote device and determining a position of the remote device relative to the washing machine appliance based on the plurality of angle readings. The method may further include determining a set camera angle for the camera assembly is met based on the determined position of the remote device. The method may still further include analyzing at least one obtained image using a machine learning image recognition process, estimating a load attribute of a load of clothes within the washing machine appliance based on the analysis, and directing a wash cycle within the washing machine appliance based on the estimated load attribute.

What is claimed is: 
     
       1. A method of operating a washing machine appliance, the washing machine appliance comprising a cabinet, a wash tub, and a wash basket, the wash tub being mounted within the cabinet, and the wash basket being rotatably mounted within a wash tub and defining a wash chamber configured for receiving a load of clothes, the method comprising:
 obtaining one or more images of the washing machine appliance from a camera assembly of a remote device spaced apart from the cabinet; 
 receiving a plurality of angle readings from the remote device independent of the one or more images; 
 determining a position of the remote device relative to the washing machine appliance based on the plurality of angle readings; 
 determining a set camera angle for the camera assembly is met based on the determined position of the remote device; 
 analyzing at least one obtained image using a machine learning image recognition process; 
 estimating a load attribute of a load of clothes within the washing machine appliance based on the analysis; and 
 directing a wash cycle within the washing machine appliance based on the estimated load attribute, 
 wherein directing comprises selecting an operating cycle parameter or adjusting a water or detergent fill amount and dispensing water to the wash chamber according to the selected operating cycle parameter or adjusted water or fill amount, 
 wherein the plurality of angle readings are detected at a measuring device fixed to the remote device, 
 wherein the measuring device comprises an accelerometer, and 
 wherein the plurality of angle readings comprise a plurality of acceleration signals received from the accelerometer. 
 
     
     
       2. The method of  claim 1 , wherein the load attribute comprises at least one of a fabric type, a load color, or a load size. 
     
     
       3. The method of  claim 1 , further comprising:
 determining a door of the washing machine appliance is open prior to determining the set camera angle is met. 
 
     
     
       4. The method of  claim 1 , further comprising:
 determining a door of the washing machine appliance is closed within a predetermined time period following obtaining one or more images of the washing machine appliance, 
 wherein directing the wash cycle is in response to determining the door is closed within the predetermined time period. 
 
     
     
       5. The method of  claim 1 , further comprising:
 generating a feedback signal prompting a feedback action at the remote device in response to determining the set camera angle is met. 
 
     
     
       6. The method of  claim 1 , wherein obtaining one or more images comprises receiving a video signal from the camera assembly, and wherein the method further comprises
 presenting a real-time feed of the camera assembly at the remote device according to the received video signal; and 
 overlaying a two-dimensional reference shape over the real-time feed. 
 
     
     
       7. The method of  claim 1 , wherein obtaining one or more images comprises receiving a video signal from the camera assembly, and wherein the method further comprises
 presenting a real-time feed of the camera assembly at the remote device according to the received video signal; and 
 displaying movement guidance with the real-time feed to guide the remote device to the set camera angle. 
 
     
     
       8. The method of  claim 1 , wherein the machine learning image recognition process comprises at least one of a convolution neural network (“CNN”), a region-based convolution neural network (“R-CNN”), a deep belief network (“DBN”), a deep neural network (“DNN”), or a vision transformer (“ViT”) image recognition process. 
     
     
       9. A method of operating a washing machine appliance, the washing machine appliance comprising a cabinet, a wash tub, and a wash basket, the wash tub being mounted within the cabinet, and the wash basket being rotatably mounted within a wash tub and defining a wash chamber configured for receiving a load of clothes, the method comprising:
 obtaining one or more images of the washing machine appliance from a camera assembly of a remote device spaced apart from the cabinet, obtaining one or more images comprising receiving a video signal from the camera assembly; 
 presenting a real-time feed of the camera assembly at the remote device according to the received video signal; 
 receiving a plurality of angle readings from the remote device, the angle readings being relative to gravity and independent of the one or more images; 
 determining a position of the remote device relative to the washing machine appliance based on the plurality of angle readings; 
 overlaying a two-dimensional reference shape over the real-time feed; 
 determining a set camera angle for the camera assembly is met based on the determined position of the remote device; 
 analyzing at least one obtained image using a machine learning image recognition process; 
 estimating a load attribute of a load of clothes within the washing machine appliance based on the analysis, the load attribute comprising at least one of a fabric type, a load color, or a load size; and 
 directing a wash cycle within the washing machine appliance based on the estimated load attribute, 
 wherein directing comprises selecting an operating cycle parameter or adjusting a water or detergent fill amount and dispensing water to the wash chamber according to the selected operating cycle parameter or adjusted water or fill amount, 
 wherein the plurality of angle readings are detected at a measuring device fixed to the remote device, 
 wherein the measuring device comprises an accelerometer, and 
 wherein the plurality of angle readings comprise a plurality of acceleration signals received from the accelerometer. 
 
     
     
       10. The method of  claim 9 , further comprising:
 determining a door of the washing machine appliance is open prior to determining the set camera angle is met. 
 
     
     
       11. The method of  claim 9 , further comprising:
 determining a door of the washing machine appliance is closed within a predetermined time period following obtaining one or more images of the washing machine appliance, 
 wherein directing the wash cycle is in response to determining the door is closed within the predetermined time period. 
 
     
     
       12. The method of  claim 9 , further comprising:
 generating a feedback signal prompting a feedback action at the remote device in response to determining the set camera angle is met. 
 
     
     
       13. The method of  claim 9 , wherein the method further comprises
 displaying movement guidance with the real-time feed to guide the remote device to the set camera angle. 
 
     
     
       14. The method of  claim 9 , wherein the machine learning image recognition process comprises at least one of a convolution neural network (“CNN”), a region-based convolution neural network (“R-CNN”), a deep belief network (“DBN”), a deep neural network (“DNN”), or a vision transformer (“ViT”) image recognition process. 
     
     
       15. A method of operating a washing machine appliance, the washing machine appliance comprising a cabinet, a wash tub, a wash basket, and a spout, the wash tub being mounted within the cabinet, the wash basket being rotatably mounted within a wash tub and defining a wash chamber configured for receiving a load of clothes, and the spout directed at the wash chamber to permit water thereto, the method comprising:
 obtaining one or more images of the washing machine appliance from a camera assembly of a remote device spaced apart from the cabinet, obtaining one or more images comprising receiving a video signal from the camera assembly; 
 presenting a real-time feed of the camera assembly at the remote device according to the received video signal; 
 receiving a plurality of angle readings from the remote device, the angle readings being relative to gravity and independent of the one or more images; 
 determining a position of the remote device relative to the washing machine appliance based on the plurality of angle readings; 
 overlaying a two-dimensional reference shape over the real-time feed; 
 determining a door of the washing machine appliance is open; 
 determining a set camera angle for the camera assembly is met based on the determined position of the remote device and the door being determined to be open; 
 analyzing at least one obtained image using a machine learning image recognition process; 
 estimating a load attribute of a load of clothes within the washing machine appliance based on the analysis, the load attribute comprising at least one of a fabric type, a load color, or a load size; and 
 directing a wash cycle within the washing machine appliance based on the estimated load attribute, 
 wherein directing comprises selecting an operating cycle parameter or adjusting a water or detergent fill amount and dispensing water to the wash chamber from the spout according to the selected operating cycle parameter or adjusted water or fill amount, 
 wherein the measuring device comprises an accelerometer, and 
 wherein the plurality of angle readings comprise a plurality of acceleration signals received from the accelerometer. 
 
     
     
       16. The method of  claim 15 , wherein the determined position is a second determined position, wherein the method further comprises
 determining, prior to the determined second position, a first position of the remote relative to the washing machine appliance based on the plurality of angle readings; 
 determining the set camera angle for the camera assembly is met, the set camera angle is not met based on the determined first position; and 
 displaying movement guidance with the real-time feed to guide the remote device to the set camera angle in response to determining the set camera angle is not met.