Make sure the software on your phone is updated to the latest version to avoid any bugs in the older version.

Since you've done a factory reset, I guess it isn't so much of a setting issue. But, you can still try to play around the following settings to check if it actually helps disable the speaker while the headphones are plugged in (assuming you're using wired headphones here).

i. Settings > Sound & vibration > Sound assistant and disable Allow speaker sound

ii. Settings > Additional settings > Accessibility > Hearing and adjust Audio balance

For further support, you might want to consider visiting the nearest Xiaomi Service center for hardware inspection.