authentication problem on nonstandard port

[Dan]

I am running two database instances of mysql (one on a nonstandard
port). After granting users permissions I find that I can access
mysql from other machines only if I use the standard port. When I use
the non-standard port I can only authenticate if I am on the box where
the database is running. Also some previously set up users can
authenticate through the non-standard port (sadly no one knows how
those users were set up). What do I need to do to allow for
authentication on the nonstandard port ex:

mysql -h<HOST> -u<USERNAME> -p
(defaulting to standard port works)
mysql -h<HOST> -u<USERNAME> -P<NON-STANDARD PORT> -p
(fails)