Ubuntu 22.04 SSH Guide

Understanding SSH

SSH is a protocol that uses encryption to secure data transmitted between a client and a server.
It enables users to execute commands, transfer files, and manage remote systems through an encrypted channel.
SSH is widely used by system administrators for managing servers, network devices, and other remote systems.

Installing SSH

To start using SSH, you'll need to install and configure both the server and client components.

• OpenSSH-Server
  • Is required to allow ssh connections
• OpenSSH-Client
  • Is used to login/connect to OpenSSH-Server

If you are using Ubuntu Desktop, the openssh client will be preinstalled, allowing you to connect to a server which is running openssh-server

If you are using Ubuntu Server, both the ssh client and openssh server are preinstalled by default.

Installing OpenSSH Server

On Ubuntu distributions, you can install the OpenSSH server by running:

sudo apt install openssh-server

Check the SSH server status with:

systemctl status ssh

Enable and Disable OpenSSH Server

By default once openssh-server installed it is enabled to start at reboot.

• Disable

sudo systemctl disable ssh

• Enable

sudo systemctl enable ssh

Installing OpenSSH Client

The OpenSSH client is usually pre-installed on most Linux and macOS systems.
For Windows, you can install the OpenSSH client by following the instructions on the official website:
https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse

Basic SSH Commands and Usage

Connecting to a remote server

Connecting to a remote server using SSH is a fundamental task when managing remote systems. Here's how to connect to a remote server using the command-line interface.

Install an SSH client: Ensure you have an SSH client installed on your local machine. Most Unix-based systems, including Linux and macOS, have an SSH client pre-installed. For Windows, you can use the built-in OpenSSH client (available in Windows 10 and later) or a third-party client like PuTTY.

Gather connection information

To connect to a remote server, you'll need the following information:

• The remote server's IP address or hostname
• The SSH port number (default is 22)
• Your username on the remote server
• The password for the user on remote server.

Connect using SSH

Open a terminal or command prompt on your local machine and use the following command to connect to the remote server:

ssh username@hostname_or_IP

Replace username with your username on the remote server, hostname_or_IP with the server's hostname or IP address.

If OpenSSH-Server is running/listening on a port other than the default port 22 include the port with the -p flag.

For example (If port 2222):

ssh john@example.com -p 2222

Or

ssh -p 2222 john@example.com

Connecting to a Remote Server

In this example, we connect to a remote Ubuntu VPS with the following credentials:

    Username: root
    IP address: 12.34.56.78
    Password: password2simple

Use the following command to connect to the remote server:

ssh root@12.34.56.78

You will be prompted to enter the password. Type password2simple and press Enter. This demonstrates how simple it can be to log into a remote computer with root access.

• If your Server is hosting SSHD on a port other than default 'port 22' include port number with the -p flag

Example with port 2222:
ssh -p 2222 root@12.34.56.78

Authenticate

When connecting for the first time, you'll see a prompt asking you to confirm the remote server's fingerprint. Verify the fingerprint and type "yes" to proceed. Next, you'll be prompted for your password. Enter your password to complete the authentication process.

Once authenticated, you'll have access to the remote server's command line. You can now execute commands and manage the remote server as if you were working on it directly.

Remember that you can use key-based authentication (with a private-public key pair) instead of a password for a more secure and convenient connection method.

Using SSH config file

An SSH config file allows you to define and manage multiple SSH connections, simplifying the process of connecting to remote servers. By creating an SSH config file, you can define custom options, such as port numbers, usernames, and key files, for each connection. The SSH config file is typically located in the ~/.ssh directory and named config.

Here's how to create and use an SSH config file:

• Create the SSH config file: If it doesn't exist, create the config file in the ~/.ssh directory using a text editor:

$EDITOR ~/.ssh/config

• Define a connection: To define a connection, you'll need to specify a Host entry followed by any options you want to apply to that connection. Here's an example:

Host server1
  HostName example.com
  User your_username
  Port 2222
  IdentityFile ~/.ssh/id_rsa_server1

In this example, we've defined a connection called server1 with the following options:

• HostName: The hostname or IP address of the remote server (example.com in this case).
• User: The username to use when connecting to the remote server (replace your_username with your actual username).
• Port: The port number to use for the SSH connection (2222 in this example).
• IdentityFile: The path to the private key file to use for authentication (replace ~/.ssh/id_rsa_server1 with the path to your private key file).

You can define multiple connections in the same config file by creating separate Host entries:

Host server2
  HostName 192.168.1.100
  User another_username
  Port 22
  IdentityFile ~/.ssh/id_rsa_server2

• Save and exit the file: Save your changes and exit the text editor.

• Connect using the SSH config file: To connect to a remote server using the defined connection, simply use the ssh command followed by the Host entry:

ssh server1

In this example, SSH will automatically use the options defined in the config file for server1, such as the hostname, username, port number, and identity file.

By using an SSH config file, you can simplify the process of managing multiple SSH connections and customize the options for each connection.

Key-based Authentication

Why use key-based authentication?

• Server1: 12.34.56.78
• Server2: 12.34.56.87

You are trying to login to Server1 (Your Server), but by mistake you enter your user and password to Server2 (Foo's Server), Can Server2 record the user and password you used? YES, Yes it can

Generating SSH key pairs

SSH key pairs consist of a private key and a public key. They provide a secure, passwordless authentication method for connecting to remote servers. The private key remains on your local machine, while the public key is added to the remote server's authorized keys. Here's how to generate an SSH key pair:

Open a terminal: On Unix-based systems (Linux and macOS), open a terminal. On Windows, open PowerShell or the Command Prompt.

Generate the key pair: Use the ssh-keygen command to create a new SSH key pair. The following command generates a 4096-bit RSA key pair:

ssh-keygen -t rsa -b 4096

You can also generate other types of keys, such as Ed25519, by changing the -t option:

ssh-keygen -t ed25519

Specify the key's location: When prompted, you can either accept the default location (~/.ssh/id_rsa for RSA keys, ~/.ssh/id_ed25519 for Ed25519 keys) or enter a custom path. It is recommended to use the default location unless you have a specific reason to change it.

Set a passphrase (optional): You can choose to protect your private key with a passphrase. If you do, you'll need to enter the passphrase every time you use the key. This adds an extra layer of security, but can be less convenient for automation or scripting. To set a passphrase, enter it when prompted; otherwise, leave the field blank

Selecting file name and path for keys

ssh-keygen -t rsa -b 4096 -f .ssh/nuc

The -f option in the ssh-keygen command is used to specify the output file for the generated key pair. In your example, ssh-keygen -t rsa -b 4096 -f .ssh/nuc, the command is generating an RSA key pair with a key length of 4096 bits, and the output files will be saved in the .ssh directory with the base name nuc.

Here's a breakdown of the options used in this command:

• -t rsa: Specifies the key type, in this case, RSA.
• -b 4096: Specifies the key length, which is 4096 bits in this case. This length offers good security and is generally recommended.
• -f .ssh/nuc: Specifies the file where the key pair will be saved. The private key will be saved as .ssh/nuc, and the public key will be saved as .ssh/nuc.pub.

After running this command, you'll have a new key pair with the private key in .ssh/nuc and the public key in .ssh/nuc.pub

Create keys with no passphase

ssh-keygen -t rsa -b 4096 -N "" -C "MYSERVER" -f ~/.ssh/serverkey

• -t rsa: Specifies the key type, in this case, RSA.
• -b 4096: Specifies the key length, which is 4096 bits in this case. This length offers good security and is generally recommended.
• -N "": Specifies an empty passphrase for the key pair. This means that the private key will not be encrypted, and no passphrase will be required when using it. This can be less secure, but more convenient for automated processes.
• -C "MYSERVER": Adds a comment to the generated key pair. In this case, the comment is "MYSERVER". Comments are useful for identifying keys when you have multiple keys in your ~/.ssh directory or on a remote server.
• -f ~/.ssh/serverkey: Specifies the file where the key pair will be saved. The private key will be saved as ~/.ssh/serverkey, and the public key will be saved as ~/.ssh/serverkey.pub.

After running this command, you'll have a new key pair with the private key in ~/.ssh/serverkey and the public key in ~/.ssh/serverkey.pub. The private key will have an empty passphrase and a comment "MYSERVER" for easier identification.

Remove the passphrase from an existing SSH private key

To remove the passphrase from an existing SSH private key, you can use the ssh-keygen command with the -p option, which is used for changing the passphrase. Follow these steps:

• Make a backup of your private key file, just in case something goes wrong during the process. You can do this by running the following command, replacing <your_private_key> with the filename of your private key:

cp <your_private_key> <your_private_key>.backup

• Run the ssh-keygen command with the -p option, specifying the private key file using the -f option:

• • -p: Indicates that you want to change the passphrase of an existing private key.
  • -f <your_private_key>: Specifies the private key file whose passphrase you want to change.

ssh-keygen -p -f <your_private_key>

• You will be prompted to enter the old passphrase for the private key. Type it in and press Enter.

• Next, you'll be prompted to enter a new passphrase. Since you want to remove the passphrase, leave this field empty and press Enter.

• You'll be asked to confirm the empty passphrase. Press Enter again to confirm.

Your private key now has its passphrase removed. Keep in mind that this makes the private key less secure, as anyone with access to the file can use it without needing to know the passphrase.

Add/Change a passphrase to an existing SSH Key

To add a passphrase to an existing SSH private key that doesn't have one, you can use the ssh-keygen command with the -p option, just like when you change or remove a passphrase. Here are the steps:

• Make a backup of your private key file, just in case something goes wrong during the process. You can do this by running the following command, replacing <your_private_key> with the filename of your private key:

cp <your_private_key> <your_private_key>.backup

• Run the ssh-keygen command with the -p option, specifying the private key file using the -f option:

ssh-keygen -p -f <your_private_key>

• You will be prompted to enter the old passphrase for the private key. Since your private key doesn't currently have a passphrase, just press Enter to proceed.

• Next, you'll be prompted to enter a new passphrase. Type in the passphrase you want to set for the private key and press Enter.

• You'll be asked to confirm the new passphrase. Type it again and press Enter to confirm.

Your private key now has a passphrase added to it. This provides an extra layer of security, as anyone using the key will need to know the passphrase to access it. Keep in mind that you should use a strong passphrase to ensure better security.

Copying public keys to the remote server

After generating an SSH key pair, you'll need to copy the public key to the remote server to enable key-based authentication. Here's how to do it:

Using ssh-copy-id

Use the ssh-copy-id command (Linux and macOS): On Unix-based systems, you can use the ssh-copy-id command to copy your public key to the remote server:

ssh-copy-id -i ~/.ssh/id_rsa.pub username@hostname_or_IP

Replace ~/.ssh/id_rsa.pub with the path to your public key file (e.g., ~/.ssh/id_ed25519.pub for Ed25519 keys), username with your username on the remote server, and hostname_or_IP with the server's hostname or IP address.

Manually copy the public key

Manually copy the public key (Windows and alternative method): If you don't have access to the ssh-copy-id command or prefer to do it manually, you can:

• Open your public key file (e.g., id_rsa.pub or id_ed25519.pub) with a text editor and copy its content.>
• Log in to the remote server via SSH.
  
• Create the ~/.ssh directory if it doesn't exist:

mkdir -p ~/.ssh

Edit or create the ~/.ssh/authorized_keys file using a text editor (e.g., nano, vim, or emacs), and paste the content of your public key at the end of the file. Save and close the file.

Set the correct file permissions: To ensure the security of your SSH setup, it's essential to set the proper file permissions on your local machine and the remote server:

• On your local machine:
  • Private key (id_rsa or id_ed25519): -rw------- (600)
  • Public key (id_rsa.pub or id_ed25519.pub): -rw-r--r-- (644)

• On the remote server:
  • ~/.ssh directory: drwx------ (700)
  • ~/.ssh/authorized_keys file: -rw------- (600)

To set the permissions on your local machine, use the chmod command:

chmod 600 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_rsa.pub

On the remote server, use the following commands:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Once you've copied your public key to the remote server and set the correct file permissions, you should be able to connect using key-based authentication without the need for a password.

Disabling password authentication (optional)

Disabling password authentication enhances the security of your SSH server by requiring key-based authentication for all connections. You can disable password authentication for specific users or for all users. To do so, follow these steps:

• Connect to the remote server: Log in to the remote server via SSH using your username and the server's hostname or IP address.

• Edit the SSH configuration file: Open the SSH server configuration file (usually located at /etc/ssh/sshd_config) with a text editor such as nano, vim, or emacs:

sudo nano /etc/ssh/sshd_config

• Disabling password authentication for all users: Find the line containing "PasswordAuthentication" and set its value to "no":

PasswordAuthentication no

If the line is commented out (i.e., it starts with a '#'), remove the '#' symbol.

• Disabling password authentication for a specific user: To disable password authentication only for a particular user, you can use a "Match User" block at the end of the sshd_config file:

Match User username
  PasswordAuthentication no

Replace username with the actual username for which you want to disable password authentication.

• Save and exit the file: Save your changes and exit the text editor.

• Restart the SSH server: Apply the changes by restarting the SSH server:

sudo systemctl restart ssh

Now, password authentication will be disabled for the specified user(s), and only key-based authentication will be allowed. Remember that if you disable password authentication, you must have a working SSH key pair set up to access the server, or you may be locked out.

Configuring the SSH server

Edit the SSH server configuration file located at /etc/ssh/sshd_config to set your desired settings. You can modify options like the listening port, allowing root login, and more.

Common sshd_config Options

The sshd_config file is located at /etc/ssh/sshd_config on most Linux systems. This file contains various options and settings that determine the behavior of the OpenSSH server. Each option is followed by its value, and lines starting with a # are considered comments.

Here's an overview of some common options in the sshd_config file:

Port

Specifies the port number that the SSH server listens on.

Port 22

AddressFamily

Determines the IP address family (IPv4, IPv6, or both) used by the SSH server.

• To specify that the SSH server should only listen for incoming IPv4 connections:

AddressFamily inet

• Or, if you want the SSH server to only listen for incoming IPv6 connections, set the 'AddressFamily' directive to 'inet6':

AddressFamily inet6

• If you want to allow both IPv4 and IPv6 connections, set the 'AddressFamily' directive to 'any':

AddressFamily any

ListenAddress

Specifies the IP address(es) the SSH server listens on. By default, it listens on all available addresses.

ListenAddress 192.168.1.10

Protocol

Defines the SSH protocol version. It's recommended to use only protocol 2.

Protocol 2

PermitRootLogin

Controls whether root login is allowed. It's generally advised to disable root login or set it to "without-password" to allow only key-based authentication for root.

PermitRootLogin no

PasswordAuthentication

Enables or disables password-based authentication.

PasswordAuthentication yes

PubkeyAuthentication

Enables or disables public key authentication.

PubkeyAuthentication yes

AuthorizedKeysFile

Specifies the location of the authorized keys file for public key authentication.

AuthorizedKeysFile .ssh/authorized_keys

LogLevel

Sets the logging level for the SSH server.

The LogLevel option in sshd_config controls the amount of information that SSH daemon (sshd) logs.

There are different log levels that can be set with this option, each providing a different level of detail:

• QUIET: Disables all logging.

• FATAL: Only logs fatal errors.

• ERROR: Logs error messages.

• INFO: Logs informational messages such as login attempts.

• VERBOSE: Logs more detailed information than INFO, including shell commands executed.

• DEBUG: Logs detailed debugging information, including raw protocol details.

The default log level is INFO, which is usually sufficient for most purposes. However, if you need to troubleshoot SSH connections or monitor user activity, setting a higher log level may be helpful.

To change the LogLevel in sshd_config, you can edit the file /etc/ssh/sshd_config (or the appropriate configuration file for your system), and add or modify the line:

LogLevel <log_level>

Where <log_level> is one of the log levels listed above.

LogLevel INFO

LoginGraceTime

Defines the time allowed for a user to successfully log in.

LoginGraceTime 2m

MaxAuthTries

Limits the number of authentication attempts allowed per connection.

MaxAuthTries 6

MaxSessions

Specifies the maximum number of simultaneous sessions allowed per network connection.

MaxSessions 10

AllowUsers, DenyUsers, AllowGroups, DenyGroups

These options control which users and groups are allowed or denied access to the SSH server. They provide a way to manage access control based on usernames and group membership.

• AllowUsers: Specifies a list of users allowed to access the SSH server. Other users will be denied access.

AllowUsers user1 user2 user3

• DenyUsers: Specifies a list of users denied access to the SSH server. Other users will be allowed access.

DenyUsers user4 user5

• AllowGroups: Specifies a list of groups whose members are allowed to access the SSH server. Users not belonging to these groups will be denied access.

AllowGroups group1 group2

• DenyGroups: Specifies a list of groups whose members are denied access to the SSH server. Users not belonging to these groups will be allowed access.

DenyGroups group3 group4

Note that the order in which these options are applied is DenyUsers, AllowUsers, DenyGroups, and finally AllowGroups.

Banner

The Banner option allows you to display a message or warning to users before they log in to the SSH server. This is often used to display legal notices, security warnings, or other important information.

To enable the banner, set the Banner option to the path of a text file containing the message you want to display:

Banner /etc/ssh/banner.txt

Create the /etc/ssh/banner.txt file and add your desired message. The content of this file will be displayed to users before they log in.

Advanced sshd_config Options

PermitTunnel

The PermitTunnel option enables or disables the use of SSH tunneling. Tunnels can be used to forward ports or create VPN-like connections between the client and the server.

• There are four possible values for this option:

"yes": Allows all types of tunnels.

"point-to-point": Allows only point-to-point (Layer 3) tunnels.

"ethernet": Allows only Ethernet (Layer 2) tunnels.

"no": Disables tunneling (default).

To enable tunneling, set the PermitTunnel option in the sshd_config file:

PermitTunnel yes

Keep in mind that enabling tunnels may expose your server to additional security risks. Only enable this option if you understand the implications and have a specific use case that requires it.

ChrootDirectory

The ChrootDirectory option allows you to restrict a user or a group to a specific directory (known as a chroot jail) when they log in via SSH. This can enhance security by isolating users and limiting their access to only the necessary parts of the filesystem.

To set up a chroot jail, follow these steps:

Create a directory that will serve as the chroot jail. For example, let's create a directory for user1:

sudo mkdir /home/user1/chroot

Change the ownership of the directory to the user and their primary group:

sudo chown user1:user1 /home/user1/chroot

In the sshd_config file, add a Match block at the end of the file to specify the ChrootDirectory for user1:

Match User user1
  ChrootDirectory /home/user1/chroot

Restart the SSH server to apply the changes:

sudo systemctl restart ssh

Now, when user1 logs in via SSH, they will be restricted to the /home/user1/chroot directory and won't be able to access other parts of the filesystem.

Note that the chroot jail should be owned by root and not writable by the user. If you need to provide write access to specific directories, create subdirectories inside the chroot jail and set appropriate permissions for those. Also, some features like SFTP may require additional configuration within the chroot jail.

ForceCommand

The ForceCommand option allows you to specify a command that will be executed when a user logs in via SSH, regardless of the command requested by the user. This can be useful for limiting the actions a user can perform or for automatically running specific tasks upon login.

To use the ForceCommand option, follow these steps:

In the sshd_config file, add a Match block at the end of the file to specify the ForceCommand for a specific user or group. For example, to force user1 to execute the command /usr/bin/my-command upon login:

Match User user1
  ForceCommand /usr/bin/my-command

Restart the SSH server to apply the changes:

sudo systemctl restart ssh

Now, when user1 logs in via SSH, the /usr/bin/my-command will be executed automatically, and they will not be able to run any other command.

Keep in mind that using ForceCommand may limit the user's ability to interact with the server or transfer files via SFTP. Make sure to test and verify the functionality for your specific use case.

Match Blocks

Match blocks in the sshd_config file allow you to apply specific configuration options based on certain criteria, such as the user, group, address, or host. This enables you to create custom rules and settings for different users, groups, or connections.

Match block syntax:

Match criteria
  Option value

Here are some examples of Match blocks and their usage:

Apply settings only for a specific user:

Match User user1
  PasswordAuthentication no
  AllowTcpForwarding yes

This configuration disables password authentication and enables TCP forwarding only for user1.

Apply settings for multiple users:

Match User user1,user2
  ChrootDirectory /home/%u/chroot

This configuration sets the chroot directory for both user1 and user2.

Apply settings for a specific group:

Match Group group1
  PasswordAuthentication yes

This configuration enables password authentication only for members of group1.

Apply settings based on the client's IP address:

Match Address 192.168.1.0/24
  PasswordAuthentication no

This configuration disables password authentication for clients connecting from the 192.168.1.0/24 subnet.

Combine multiple criteria:

Match User user1 Address 192.168.1.0/24
  PasswordAuthentication yes

This configuration enables password authentication only for user1 when they connect from the 192.168.1.0/24 subnet.

Remember to restart the SSH server after making changes to the sshd_config file:

sudo systemctl restart ssh

Match blocks offer flexibility in customizing your SSH server's configuration based on various criteria. Use them wisely to enhance security and optimize your server's settings.

Best Practices and Tips sshd_config

When configuring your sshd_config file, it's essential to follow best practices to ensure the security and stability of your SSH server. Here are some recommendations and tips:

• Keep the server up-to-date: Always update your SSH server software and the underlying operating system to ensure you have the latest security patches and features.

• Use strong authentication: Enable key-based authentication (PubkeyAuthentication) and consider disabling password authentication (PasswordAuthentication) to reduce the risk of brute-force attacks.

• Limit root access: Set "PermitRootLogin" to "no" or "without-password" to prevent direct root login or require key-based authentication for root.

• Use non-standard ports: Change the default SSH port (22) to a non-standard port to reduce the exposure to automated scans and attacks. Keep in mind this is security through obscurity and should be combined with other security measures.

Restrict user access: Use "AllowUsers," "DenyUsers," "AllowGroups," and "DenyGroups" options to control which users and groups can access the SSH server.

Monitor logs: Regularly check your SSH server logs for any suspicious activity or failed login attempts. Adjust the "LogLevel" setting in sshd_config as needed.

• Default Log Path Ubuntu 22.04: /var/log/auth.log

Use chroot jails: Isolate users by creating chroot jails using the "ChrootDirectory" option, especially when providing SFTP access or when users don't require full access to the server.

Configure connection settings: Set appropriate values for "LoginGraceTime" and "MaxAuthTries" to limit the time allowed for successful login and the number of authentication attempts per connection.

Use a strong firewall: Configure your server's firewall to only allow SSH connections from trusted IP addresses or networks.

Regularly review and audit: Periodically review your sshd_config settings and make adjustments as necessary. Keep up-to-date with SSH security best practices and recommendations.

By following these best practices and tips, you can enhance the security and performance of your SSH server, protecting it from unauthorized access and potential attacks.

Troubleshooting sshd_config Issues

When encountering problems with your SSH server configuration, it's important to know how to diagnose and resolve issues. Here are some common problems and troubleshooting steps:

Check syntax and configuration errors: If the SSH server is not starting or not functioning as expected, check the sshd_config file for any syntax or configuration errors. Use the following command to test the configuration file for errors:

sudo sshd -t

If there are any issues, the command will provide error messages with information on what needs to be fixed.

Review log files: Inspect the SSH server log files for any error messages or relevant information. The location of the log files may vary depending on your system, but common locations are /var/log/auth.log or /var/log/secure. Tail the log file while attempting to connect to get real-time information:

sudo tail -f /var/log/auth.log

Restart the SSH server

Check firewall settings: Ensure that the server's firewall is allowing SSH connections on the correct port. If you changed the default SSH port, update your firewall rules accordingly.

Verify user permissions: If a specific user is unable to connect, check the user's permissions, home directory, and the settings in the sshd_config file, such as "AllowUsers," "DenyUsers," "AllowGroups," or "DenyGroups."

SSH server from a client, use the verbose mode to get more detailed information about the connection process. This can help identify any issues with authentication or configuration. Run the following command to enable verbose mode:

ssh -v user@example.com

Replace "user@example.com" with the appropriate username and server address. You can increase the verbosity level by adding more "v" characters (e.g., -vv or -vvv) if needed.

Check file permissions: Ensure that the file permissions for the user's home directory, the .ssh directory, and the authorized_keys file are set correctly. The user's home directory should not be writable by other users, the .ssh directory should have permissions set to 700 (drwx------), and the authorized_keys file should have permissions set to 600 (-rw-------).

Test network connectivity: If you're unable to connect to the SSH server, verify that you can reach the server on the network. Use tools like ping, traceroute, or telnet to check the connection to the server and the specific SSH port.

By following these troubleshooting steps, you should be able to diagnose and resolve most issues related to the sshd_config file and the SSH server configuration. Remember to carefully review the settings in your sshd_config file and consult the server logs for additional information when encountering problems.

After making changes, restart the SSH server:

sudo systemctl restart ssh

Running commands on a remote server

Once you've connected to a remote server using SSH, you can execute commands on the remote machine just as you would on your local system. However, you can also run commands on a remote server without establishing an interactive SSH session.

This can be useful for automation, scripting, or quick tasks. Here's how to do it:

Use the SSH command: To run a command on a remote server without entering an interactive session, use the following syntax:

ssh username@hostname_or_IP -p port 'command'

Replace username with your username on the remote server, hostname_or_IP with the server's hostname or IP address, port with the SSH port number (if different from the default 22), and command with the command you want to execute.

For example, to list the contents of the remote server's home directory, you can use:

ssh john@example.com -p 22 'ls -la'

Handling multiple commands

If you need to execute multiple commands, you can chain them together using a semicolon or &&.

The semicolon allows you to run multiple commands sequentially, while the && operator runs the next command only if the previous command was successful.

For example, to update the package list and then upgrade the packages on a remote Ubuntu server:

ssh john@example.com -p 2222 'sudo apt-get update; sudo apt-get upgrade -y'

• Command output:

The output of the command will be displayed in your local terminal, just as if you were running the command on your local machine. Using key-based authentication

Transferring files with SCP

The Secure Copy Protocol (SCP) is a useful tool for transferring files between your local machine and a remote server using SSH. SCP ensures that the data is encrypted during transit, providing a secure and efficient way to transfer files.

Install an SCP client

Most Unix-based systems, including Linux and macOS, have an SCP client pre-installed. For Windows, you can use the built-in SCP client included with the OpenSSH package (available in Windows 10 and later) or a third-party client like WinSCP.

Transfer a file from your local machine to a remote server

To copy a file from your local machine to a remote server, use the following command:

• Note the use of the upper case -P for ports with scp

scp -P port local_file_path username@hostname_or_IP:remote_file_path

Replace port with the SSH port number (if different from the default 22), local_file_path with the path to the file on your local machine, username with your username on the remote server, hostname_or_IP with the server's hostname or IP address, and remote_file_path with the desired location on the remote server.

For example:

scp -P 22 /home/john/documents/report.pdf john@example.com:/home/john/reports/

This command will copy the "report.pdf" file from the local machine to the "reports" directory on the remote server.

Transfer a file from a remote server to your local machine

To copy a file from a remote server to your local machine, use the following command:

scp -P port username@hostname_or_IP:remote_file_path local_file_path

Replace port with the SSH port number (if different from the default 22), username with your username on the remote server, hostname_or_IP with the server's hostname or IP address, remote_file_path with the path to the file on the remote server, and local_file_path with the desired location on your local machine.

For example:

scp -P 2222 john@example.com:/home/john/reports/report.pdf /home/john/documents/

Or

scp john@example.com:/home/john/reports/report.pdf /home/john/documents/-

This command will copy the "report.pdf" file from the remote server's "reports" directory to the "documents" directory on your local machine.

Transferring directories

To transfer an entire directory, use the -r flag:

scp -r -P port local_directory_path username@hostname_or_IP:remote_directory_path

Or, to copy a directory from the remote server to your local machine:

scp -r -P port username@hostname_or_IP:remote_directory_path local_directory_path

Using SCP is a convenient and secure way to transfer files between your local machine and a remote server. It leverages the security of the SSH protocol to ensure that your data remains encrypted during transit.

Transferring from Remote Computer to Remote Computer

Copy the file stuff.txt from remote host 12.34.56.67 to host 11.22.33.44

scp name@12.34.56.67:/home/user/Documents/stuff.txt name@11.22.33.44:/home/user/Documents/

With the -3 flag copies between two remote hosts "12.34.56.67" and "11.22.33.44" are transferred through the local host running the command.

scp -3 name@12.34.56.67:/home/user/Documents/stuff.txt \ name@11.22.33.44:/home/user/Documents/

Transferring multiple files

Send files foo.txt and bar.txt to remote.

scp foo.txt bar.txt user@12.34.56.78:~/Documents/

Copy multiple files from remote "Documents" directory to local "Documents" directory.

scp user@11.22.33.44:/home/user/Documents/\{todo_list.txt,links.txt,stuff.txt\} /home/$USER/Documents/

Copy multiple files from the remote to local current directory.

scp name@12.34.56.78:~/\{README.md,.bashrc\} .

Transferring files with SFTP

The SSH File Transfer Protocol (SFTP) is another method for transferring files securely between your local machine and a remote server. Unlike SCP, SFTP provides an interactive interface that allows you to navigate, upload, and download files more easily.

Install an SFTP client: Most Unix-based systems, including Linux and macOS, have an SFTP client pre-installed. For Windows, you can use the built-in SFTP client included with the OpenSSH package (available in Windows 10 and later) or a third-party client like WinSCP or FileZilla.

Connect to a remote server: To start an SFTP session with a remote server, open a terminal or command prompt on your local machine and use the following command:

sftp -P port username@hostname_or_IP

Replace port with the SSH port number (if different from the default 22), username with your username on the remote server, and hostname_or_IP with the server's hostname or IP address.

For example:

sftp -P 22 john@example.com

Navigate the remote filesystem: Once connected, you can use commands similar to those available in a Unix shell to navigate the remote server's filesystem. Some common SFTP commands include:

ls: List files and directories

cd: Change the current directory

mkdir: Create a new directory

rmdir: Remove an empty directory

get: Download a file from the remote server

put: Upload a file to the remote server

rm: Remove a file

rename: Rename a file or directory

exit: Exit the SFTP session

Transfer files: To transfer files, use the put command to upload a file from your local machine to the remote server, and the get command to download a file from the remote server to your local machine. For example:

Upload a file:

put local_file_path remote_file_path

Download a file:

get remote_file_path local_file_path

Replace local_file_path and remote_file_path with the appropriate paths for the files you want to transfer.

Transferring directories: To transfer entire directories, use the -r flag with the put and get commands:

Upload a directory:

put -r local_directory_path remote_directory_path

Download a directory:

get -r remote_directory_path local_directory_path

Disconnect from the remote server: When you've finished transferring files, type exit to close the SFTP session.

SFTP offers a more user-friendly, interactive experience for transferring files compared to SCP. By utilizing the secure and encrypted SSH protocol, SFTP ensures that your data remains safe during transfer.

Advanced SSH Techniques

Port forwarding and tunneling

SSH port forwarding and tunneling allow you to securely forward network traffic between your local machine and a remote server. This can be useful for accessing remote services, bypassing firewalls, or securely transmitting sensitive data.

Local Port Forwarding: Local port forwarding creates a secure tunnel between your local machine and a remote server, allowing you to access remote services as if they were running on your local machine. To set up local port forwarding, use the -L flag with the SSH command:

ssh -L local_port:remote_host:remote_port username@hostname_or_IP

Replace local_port with an available port on your local machine, remote_host with the hostname or IP address of the remote server hosting the service, remote_port with the port number of the remote service, username with your username on the remote server, and hostname_or_IP with the server's hostname or IP address.

Remote Port Forwarding: Remote port forwarding enables you to expose a local service running on your machine to a remote network. To set up remote port forwarding, use the -R flag with the SSH command:

ssh -R remote_port:local_host:local_port username@hostname_or_IP

Replace remote_port with an available port on the remote server, local_host with the hostname or IP address of the local machine hosting the service, local_port with the port number of the local service, username with your username on the remote server, and hostname_or_IP with the server's hostname or IP address.

• • Forwarding X, Sound, and Video on Ubuntu 22.04 with Ubuntu 22.04 LXC**: To forward X, sound, and video from a remote Ubuntu 22.04 server to your local Ubuntu 22.04 machine, you'll need to enable X11 forwarding and install the necessary packages.

• Install required packages: On both your local machine and the remote server, install the x11-apps and pulseaudio packages:

sudo apt update
sudo apt install x11-apps pulseaudio

• Enable X11 forwarding: To enable X11 forwarding, you'll need to edit the SSH server configuration file (/etc/ssh/sshd_config) on the remote server:

sudo nano /etc/ssh/sshd_config

Find the line containing "X11Forwarding" and set its value to "yes":

X11Forwarding yes

If the line is commented out (i.e., it starts with a '#'), remove the '#' symbol. Save your changes and exit the text editor.

• Restart the SSH server: Apply the changes by restarting the SSH server:

sudo systemctl restart ssh

• Connect with X11 forwarding: From your local machine, use the -X flag to enable X11 forwarding when connecting to the remote server:

ssh -X username@hostname_or_IP

• Export PULSE_SERVER environment variable: On the remote server, export the PULSE_SERVER environment variable to forward sound:

export PULSE_SERVER=tcp:localhost

You can add this line to the remote user's ~/.bashrc or ~/.profile file to make the change permanent.

• Run applications: Now, you can run graphical applications on the remote server, and they will be displayed on your local machine with sound and video forwarded.

Please note that forwarding X, sound, and video might cause increased latency and reduced performance compared to running the applications locally.

SSH agent forwarding

SSH agent forwarding is a powerful feature that allows you to use your local SSH keys to authenticate with remote servers without having to copy your private keys to those servers. This is particularly useful when you need to access one remote server (Server B) through another remote server (Server A).

Start the SSH agent on your local machine

Before you enable SSH agent forwarding, you need to start the SSH agent on your local machine. Open a terminal and run the following command:

• For Linux and macOS:

eval "$(ssh-agent -s)"

For Windows (Git Bash or Cygwin):

eval $(ssh-agent)

This command starts the SSH agent and sets the required environment variables.

Add your SSH key to the agent

Next, add your private key to the SSH agent with the following command:

ssh-add ~/.ssh/your_private_key

Replace your_private_key with the filename of your private key. This might be id_rsa, id_ed25519, or another key file depending on your setup.

Configure SSH agent forwarding on your local machine

Edit your SSH config file to enable agent forwarding. The config file is usually located at ~/.ssh/config. If the file doesn't exist, create it.

Add the following lines to the config file:

Host server_a_alias
  HostName server_a_ip_or_hostname
  User your_username_on_server_a
  ForwardAgent yes

Host server_b_alias
  HostName server_b_ip_or_hostname
  User your_username_on_server_b
  ForwardAgent yes

Replace

• server_a_alias
• server_a_ip_or_hostname
• your_username_on_server_a
• server_b_alias
• server_b_ip_or_hostname
• your_username_on_server_b

with the appropriate values.

Make sure your public key is added to the remote servers

Before you can use SSH agent forwarding, you need to add your public key to the ~/.ssh/authorized_keys file on both Server A and Server B. If you haven't done this already, you can use the following command:

ssh-copy-id -i ~/.ssh/your_public_key user@server_ip_or_hostname

Replace your_public_key, user, and server_ip_or_hostname with the appropriate values.

Test SSH agent forwarding

First, SSH into Server A:

ssh server_a_alias

Then, from Server A, SSH into Server B:

ssh server_b_alias

If everything is set up correctly, you should be able to access Server B without being prompted for a password.

Verify SSH agent forwarding

To make sure that SSH agent forwarding is working, you can check the value of the SSH_AUTH_SOCK environment variable on Server B.

From Server B, run the following command:

echo $SSH_AUTH_SOCK

If SSH agent forwarding is working, this command should return a non-empty value.

That's it! You've successfully set up and tested SSH agent forwarding. Now you can use your local SSH keys to authenticate with remote servers without having to copy your private keys to those servers.

Command Restriction

The authorized_keys file can be used to restrict the commands that a specific SSH key can execute. This is especially useful for security purposes, to limit the potential damage that could be done if a key is compromised.

By including a command= directive in the authorized_keys file, you can specify the exact command that will be run when a client connects using the associated key. Any command provided by the client will be ignored, and the command specified in the authorized_keys file will be used instead.

Example:

command="/usr/bin/scp -t /home/rscp/media/" ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC...

is set up to always execute the scp command (used for secure copy of files over SSH) to the specified directory, no matter what command was originally issued by the client. This is a good way to create a "write-only" drop box, for instance.

However, the keyholder could potentially still execute arbitrary commands by carefully crafting the file names they upload, so additional precautions should be taken, such as using command= along with other directives like no-port-forwarding, no-X11-forwarding, and no-pty to further limit what can be done with the key.

command="/usr/bin/scp -t /home/rscp/media/",no-port-forwarding,no-X11-forwarding,no-pty ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC...

This entry does the following:

• The command= directive runs the specified command when a client connects using this key. In this case, the command is scp, which securely copies files to the /home/rscp/media/ directory.
• The no-port-forwarding directive prevents the client from using SSH's port forwarding features, which could potentially be used to create a secure tunnel for other network traffic.
• The no-X11-forwarding directive prevents the client from forwarding X11 graphical sessions, which could be used to run graphical applications over the SSH connection.
• The no-pty directive prevents the allocation of a pseudo-terminal, which means the client can't interact with a shell or run interactive commands.

The ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC... part is the public key of the client. Replace this with the actual key.

This configuration significantly limits the operations that can be performed with this key, providing an additional layer of security.

SCP Only

Use Case Example: Have a Server hosting XML Dumps, and want to automate sending a file or directory from Server1 to Server2 using a script and ssh-key so i don't need to enter password.

Create Account on Server

Create user account you are going to use:
adduser rscp

Make sure user has a .ssh directory to send public key to:
mkdir /home/rscp/.ssh

Make a Directory to transfer files to:
mkdir /home/rscp/media

Note: If you see error scp: /home/rscp/media/test.txt: Permission denied If you created directory media when logged in as root then check directory permissions and if need assign ownership to user account.
Example:chown rscp:rscp /home/rscp/media

Send your public key to server

After public_key/authorized_key is on server, edit authorized_keys and at the start before ssh-rsa <KEY>

command="/usr/bin/scp -t /home/rscp/media/"  ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC...

If from remote server you are sending a Directory include the -r flag in command:

After public_key/authorized_key is on server, edit authorized_keys and at the start before ssh-rsa <KEY>

command="/usr/bin/scp -t -r /home/rscp/media/"  ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC...

This entry in the authorized_keys file uses the command option to restrict the SSH command that can be run with the associated SSH key. The command option specifies that the scp command should be used to transfer files to the /home/rscp/media/ directory on the server.

Here's a breakdown of the entry:

• command="/usr/bin/scp -t /home/rscp/": This specifies that the scp command should be used as the SSH command for this key, with the -t option to specify that the remote end is a file (in this case, a directory), and the destination directory on the server is /home/rscp/. This means that the user can only use the SSH key to transfer files to the /home/rscp/ directory on the server.

• ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC...: This is the public key associated with the private key that is used for authentication.

By using the command option in this way, you can restrict the actions that the user can perform with the SSH key, which can help to improve security. In this case, the user can only transfer files to the specified directory on the server using the scp command.

Tip - transfer file to a path your USER does not have permissions for

You can write a shell script to check the /home/rscp/media directory every minute using a while loop and the sleep command. If any files are found in the directory, the script can move them to the /var/www/media directory using the mv command. Here's an example script:

#!/bin/bash

while true
do
  if [ "$(ls -A /home/rscp/media/)" ]; then
    mv /home/rscp/media/* /var/www/media/
  fi
  sleep 60
done

In this script, the while loop runs indefinitely (while true) and sleeps for 60 seconds at the end of each iteration (sleep 60).

The if statement checks if the /home/rscp/media directory is not empty ([ "$(ls -A /home/rscp/media/)" ]). If it is not empty, the mv command is used to move all files and directories from the /home/rscp/media/ directory to the /var/www/media/ directory.

Save this script to a file (e.g. move-files.sh) and make it executable using the chmod +x move-files.sh command. You can then run the script using ./move-files.sh & to start it in the background and allow it to run indefinitely. The & symbol is used to run the script in the background so that you can continue using the terminal.

Note that running this script indefinitely can consume system resources, so you may want to consider setting up a scheduled task (e.g. using cron) to run the script at a specific interval instead of running it indefinitely.

Tilde ~ the escape character

The tilde (~) character has a special meaning in the context of SSH. When using SSH, you can use the tilde character followed by a control sequence to perform certain actions. These are called "tilde escape sequences" or "tilde commands." They are useful for managing your SSH connections.

Here's how to use tilde escape sequences when connected to a remote server via SSH:

• Make sure you are at the beginning of a new line in your terminal. Press Enter if you are not.

• Type the tilde (~) character, followed by the appropriate control sequence. Note that you should not press Enter after typing the tilde character, but rather type the control sequence directly after it.

Here are some common tilde escape sequences:

~. : Close the SSH connection. This can be helpful if the connection is frozen or unresponsive.

~^Z : Suspend the SSH connection and return to your local shell. You can later resume the connection using the fg command.

~# : List all forwarded connections (both local and remote) that are active in the current SSH session.

~& : Run the SSH session in the background. This is useful if you want to perform other tasks on your local machine without closing the SSH connection.

~~ : Send a literal tilde character to the remote system. This is useful if you need to type a tilde character in the remote system without triggering an escape sequence.

Remember that these escape sequences only work if they are entered at the beginning of a new line in your terminal. If you're typing them in the middle of a command or text, they won't be recognized as special control sequences.

Troubleshooting and Best Practices

In this section, we'll cover some common issues and best practices related to SSH connections, including managing a large number of SSH keys.

Too many authentication attempts

When connecting to an SSH server, you might encounter the "Too many authentication attempts" error. This is often caused by having too many private keys in your ~/.ssh directory. By default, SSH tries each key until it finds the correct one, but many servers limit the number of authentication attempts.

Solution: To resolve this issue, you can create a separate directory for your keys and configure the SSH config file to use the appropriate key for each connection.

• Create a new directory for your keys:

mkdir ~/.ssh/keys

• Move your private key files to the new directory:

mv ~/.ssh/id_rsa_* ~/.ssh/keys/

Update your SSH config file to specify the correct key for each connection:

    Host server1
      ...
      IdentityFile ~/.ssh/keys/id_rsa_server1

    Host server2
      ...
      IdentityFile ~/.ssh/keys/id_rsa_server2

Permission issues

SSH is very strict about file and directory permissions. Ensure that your ~/.ssh directory and its contents have the correct permissions:

• The ~/.ssh directory should have permissions set to 700:

chmod 700 ~/.ssh

Private key files should have permissions set to 600:

chmod 600 ~/.ssh/id_rsa

The ~/.ssh/config file should have permissions set to 600:

chmod 600 ~/.ssh/config

• Best practices: Follow these best practices to maintain secure and efficient SSH connections:

• Use SSH key pairs instead of passwords for authentication, as they provide better security.
• Regularly update your SSH keys to maintain their security.
• Use strong, unique passphrases to protect your private keys.
• Disable password authentication and root login on your SSH server to reduce the risk of brute-force attacks.
• Regularly update your SSH server software to ensure you're running the latest security patches.
• Use non-standard port numbers for your SSH server to make it less likely to be targeted by automated attacks.
• Implement multi-factor authentication (MFA) for your SSH connections, if possible.
• Regularly review and remove any unnecessary authorized keys from the ~/.ssh/authorized_keys file on your servers.
• Use the Match directive in the sshd_config file to apply custom rules and settings for different users, groups, or connections.