Now when it is a Hardwallet like this, a Ledger, a Trezor... your private keys remain on the physical offline device, the only thing that the device sends to the phone is the signed transaction and not the wallet key.