Why am I keep getting "permission denied" when trying to connect to the instance on AWS?

I am following the instructions here: Tool-Solutions/mlops-cloud at master · ARM-software/Tool-Solutions (github.com)

But I am not able to connect to the instance. I tried both powershell and MobaXTerm and in both cases it fails.

Here is what I tried with powershell: 

ssh -i ArmVirtualHardwareKeyPair_pem.pem ubuntu@44.203.220.84 -v

And this is the output:

OpenSSH_for_Windows_8.1p1, LibreSSL 3.0.2
debug1: Connecting to 44.203.220.84 [44.203.220.84] port 22.
debug1: Connection established.
debug1: identity file ArmVirtualHardwareKeyPair_pem.pem type -1
debug1: identity file ArmVirtualHardwareKeyPair_pem.pem-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_for_Windows_8.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.4
debug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 44.203.220.84:22 as 'ubuntu'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:/QQuOSjmopkxTqei5Z4BRDufc79dyyCU837KSkayLHQ
debug1: Host '44.203.220.84' is known and matches the ECDSA host key.
debug1: Found key in C:\\Users\\Eyal/.ssh/known_hosts:1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory
debug1: Will attempt key: ArmVirtualHardwareKeyPair_pem.pem explicit
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: ArmVirtualHardwareKeyPair_pem.pem
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
ubuntu@44.203.220.84: Permission denied (publickey).

I also tried it this way: 

ssh -i "ArmVirtualHardwareKeyPair_pem.pem" ubuntu@ec2-44-203-220-84.compute-1.amazonaws.com -v

And this is the output:

OpenSSH_for_Windows_8.1p1, LibreSSL 3.0.2
debug1: Connecting to ec2-44-203-220-84.compute-1.amazonaws.com [44.203.220.84] port 22.
debug1: Connection established.
debug1: identity file ArmVirtualHardwareKeyPair_pem.pem type -1
debug1: identity file ArmVirtualHardwareKeyPair_pem.pem-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_for_Windows_8.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.4
debug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to ec2-44-203-220-84.compute-1.amazonaws.com:22 as 'ubuntu'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:/QQuOSjmopkxTqei5Z4BRDufc79dyyCU837KSkayLHQ
debug1: Host 'ec2-44-203-220-84.compute-1.amazonaws.com' is known and matches the ECDSA host key.
debug1: Found key in C:\\Users\\Eyal/.ssh/known_hosts:2
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory
debug1: Will attempt key: ArmVirtualHardwareKeyPair_pem.pem explicit
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: ArmVirtualHardwareKeyPair_pem.pem
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
ubuntu@ec2-44-203-220-84.compute-1.amazonaws.com: Permission denied (publickey).

In MobaXTerm I get: 

No supported authentication methods available (server sent: publickey)