usb: ehci: Clear USBMODE_BE on LE MMIO

If the USB EHCI is configured for little endian MMIO, make sure to
clear the USBMODE_BE flag from the USBMODE register.

Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Cc: Hans de Goede <hdegoede@redhat.com>
diff --git a/drivers/usb/host/ehci-hcd.c b/drivers/usb/host/ehci-hcd.c
index b3eb08d..8f259be 100644
--- a/drivers/usb/host/ehci-hcd.c
+++ b/drivers/usb/host/ehci-hcd.c
@@ -134,6 +134,8 @@
 	tmp |= USBMODE_CM_HC;
 #if defined(CONFIG_EHCI_MMIO_BIG_ENDIAN)
 	tmp |= USBMODE_BE;
+#else
+	tmp &= ~USBMODE_BE;
 #endif
 	ehci_writel(reg_ptr, tmp);
 }