🔐 How to Use Multiple RSA SSH Keys on the Same Machine

By hari, 23 March, 2025

Working with different servers, Git repositories, or organizations often requires managing multiple SSH identities. Whether you're juggling work and personal GitHub accounts or managing access to various cloud servers, keeping things clean and secure on your local machine is essential.

This guide will walk you through how to generate, configure, and use multiple RSA keys effectively on the same machine using SSH.

📂 Why Use Multiple SSH Keys?

Each SSH key represents a unique identity. You might need separate keys for:

  • Personal and work GitHub/GitLab accounts

  • Different server access (e.g., client environments)

  • Projects that require restricted or scoped credentials

  • Security compliance or key rotation policies

🛠 Step 1: Generate Multiple RSA Keys

To create a new RSA key, use the ssh-keygen command:

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

You’ll be asked where to save the key. Choose a meaningful name to differentiate:

Enter file in which to save the key (/home/you/.ssh/id_rsa): /home/you/.ssh/id_rsa_work

Repeat the process for each key:

  • ~/.ssh/id_rsa_work

  • ~/.ssh/id_rsa_personal

🚀 Step 2: Add Keys to the SSH Agent

Start the SSH agent:

eval "$(ssh-agent -s)"

Add your new keys:

ssh-add ~/.ssh/id_rsa_work
ssh-add ~/.ssh/id_rsa_personal


💡 Use ssh-add -l to view currently loaded keys.

🧠 Step 3: Create an SSH Config File

SSH doesn’t automatically know which key to use for each connection. Create or update the ~/.ssh/config file to map hosts to specific keys:

nano ~/.ssh/config

Add entries like this:

# Work GitHub
Host github.com-work
 HostName github.com
 User git
 IdentityFile ~/.ssh/id_rsa_work
 IdentitiesOnly yes
# Personal GitHub
Host github.com-personal
 HostName github.com
 User git
 IdentityFile ~/.ssh/id_rsa_personal
 IdentitiesOnly yes

 

This setup tells your system exactly which key to use for which host.

✅ Step 4: Use the Correct Host Alias

When cloning repositories or SSH-ing into servers, use the alias:

# Cloning using the alias
git clone git@github.com-work:yourcompany/repo.git
git clone git@github.com-personal:yourusername/myproject.git


🔐 Bonus Tips

  • Permissions: SSH private keys should have secure permissions.

chmod 600 ~/.ssh/id_rsa_*

  • Backups: Store your keys safely and securely.

  • Cleanup: Use ssh-add -D to remove all keys from the agent when switching contexts.

  • Password Protection: Consider setting a passphrase on your private keys for added security.

 

🛠 Troubleshooting: agent refused operation Error

If you get the following error when using your alias:


sign_and_send_pubkey: signing failed for RSA ".../.ssh/id_rsa_personal" from agent: agent refused operation
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Here are common fixes:

🔄 1. Re-add the key to the agent

ssh-add -d ~/.ssh/id_rsa_personal
ssh-add ~/.ssh/id_rsa_personal

Ensure the SSH agent is running:

eval "$(ssh-agent -s)"

Then add again.

🔐 2. If the key has a passphrase

Some environments don’t prompt for it properly. Use:

SSH_ASKPASS=/usr/bin/ssh-askpass ssh-add ~/.ssh/id_rsa_personal

If ssh-askpass isn't installed:

sudo apt install ssh-askpass

🔁 3. Generate a PEM-format RSA key (for compatibility)

ssh-keygen -t rsa -b 4096 -m PEM -f ~/.ssh/id_rsa_personal_new

Update your SSH config to use the new key:

IdentityFile ~/.ssh/id_rsa_personal_new

🔍 4. Test the alias directly

ssh -T git@github.com-personal

You should get:

Hi username! You've successfully authenticated...

🔒 5. Fix file permissions

chmod 600 ~/.ssh/id_rsa_personal

🧩 Conclusion

Managing multiple RSA keys on a single machine might seem tricky at first, but with proper naming, SSH agent use, and a clear ~/.ssh/config file, it becomes effortless and secure.

Whether you're freelancing, working across teams, or simply separating concerns, this is a must-know setup for every developer or sysadmin.

Tags