This at all doesn't seem like a bad thing, especially if you're doing this on your own personal systems. Doing this in an organization seems like a crazy step to implement.
Also what happens if you're phone dies, how do you update the authentication device?
There is a hidden file in the root account with the one time use codes and the key for the app. Local console access doesn't require 2FA, only SSH.
If you set it up in PAM correctly that is, I didn't read through this guide. I did one a while ago on here with steps for everything, and doing it that way only requires 2FA codes with SSH, not local console access.