Convenient Remote Access with SSH Config

If you are working with a lot of remote ssh hosts it becomes hard to remember all that host specific information: username, ip address, identity file, non-standard port or local/remote port forwarding. ssh_config to rescue. Here is a sample to give you an idea (file ~/.ssh/config):

Compression yes
IdentityFile ~/.ssh/id_rsa
LogLevel ERROR
Port 22

Host h1
    HostName 192.168.91.57
    User master
    IdentityFile ~/.ssh/h1.pem

Host db1
    HostName usca45d1.example.com
    User pg
    LocalForward 5432 127.0.0.1:5432

The above configuration let me access those hosts simply by name, e.g.:

ssh h1
scp schema.sql db1:~/

How to chroot SFTP (Secure File Transfer)

SFTP (SSH File Transfer Protocol) is a network protocol that provides file transfer functionality over reliable data stream. It has nothing related with old ftp protocol however it treated as a secure replacement. Here we are going to achieve the following:

1. Service root directory: /srv/sftp
2. Each user must have isolated sftp location, e.g. /srv/sftp/user1
3. User top level directory include directories: files, archive
4. User session is chrooted
5. User is limited to sftp only, no shell, no ssh access

If you have ssh installed, you have sftp. Just in case:

apt-get -y install ssh

SSH Configuration for SFTP

You need to ensure the sftp subsystem is enabled in ssh. We are going to use internal-sftp implementation, that is in-process ssh subsystem (file /etc/ssh/sshd_config):

#Subsystem sftp /usr/lib/openssh/sftp-server
Subsystem sftp internal-sftp

Let use sftp group to identify users for sftp. Here is a matching rule for ssh, add it at the end of /etc/ssh/sshd_config file:

Match group sftp
    ChrootDirectory /srv/sftp/%u
    X11Forwarding no
    AllowTcpForwarding no
    MaxAuthTries 2
    ForceCommand internal-sftp

Restart ssh so the changes take place:

/etc/init.d/ssh restart

Users

Let create a security group for our sftp users:

groupadd sftp

Here is a script that does the rest (file sftp-add.sh).

#!/bin/bash

sftproot=/srv/sftp

genpasswd() {
    local l=$1
    [ "$l" == "" ] && l=20
    tr -dc A-Za-z0-9_ < /dev/urandom \
        | head -c ${l} | xargs
}

if [ -z $1 ]; then 
    echo "Usage $0 username"
    exit 1
fi

# 1. User is created with home directory set to /, 
# this is the directory sftp change once chroot.
# 2. User added to group sftp.
# 3. Do not create home directory.
# 4. User has no shell, ssh login impossible.
useradd -d / -G sftp -M -s /bin/false $1

echo "Auto generated password:"
genpasswd
passwd $1

mkdir -p $sftproot/$1/{files,archive}
# Chroot directory must be owned by root
chown root:$1 $sftproot/$1 
# User has read-only access
chmod -R 750 $sftproot/$1
# User owns everything below chroot directory
chown $1:$1 $sftproot/$1/*

Just invoke it this way:

./sftp-add.sh user1

Now you should be able use sftp.

Debian KVM

Kernel-based Virtual Machine (KVM) is a virtual machine implementation using the operating system's kernel (read more here). Here are few steps to install kvm in debian:

Server

1. Setup SSH. Read more here.
2. Setup bridge-utils package...

   apt-get install bridge-utils
   

   ... and configure network interface (restart computer so network changes take place):

   auto eth0
   iface eth0 inet manual
   
   auto br0
   iface br0 inet static
        address 192.168.10.11
        netmask 255.255.255.0
        network 192.168.10.0
        broadcast 192.168.10.255
        gateway 192.168.10.1
        bridge_ports eth0
        bridge_stp off
        # 1.
        bridge_fd 0
        bridge_maxwait 0
        # 2.
        #bridge_fd 9
        #bridge_hello 2
        #bridge_maxage 12
   

3. Install qemu-kvm and libvirt-bin packages:

   apt-get -y install qemu-kvm libvirt-bin
   

4. Add a user that will be managing kvm to group libvirt (e.g. user1):

   adduser user1 libvirt
   

Client

1. Setup Password-less ssh login to kvm server. Read more here.
2. Install virt-manager package:

   apt-get -y install virt-manager
   

3. If your client is not going to host kvm virtual machines you can disable the following daemons:

   update-rc.d ebtables disable
   update-rc.d libvirt-bin disable
   update-rc.d libvirt-guests disable
   update-rc.d lvm2 disable
   

4. Open Virtual Machine Manager from Applications > System Tools.
5. In File menu select Add Connection. In dialog that appears ensure method ssh and user that you added on server to group libvirt).

Performance Tuning

1. The KVM host can take benefit of KSM by finding and sharing memory blocks between vitual machines (add the following to /etc/rc.local).

   echo 100 > /sys/kernel/mm/ksm/sleep_millisecs
   echo 1 > /sys/kernel/mm/ksm/run
   

   You can take a look at pages sharing / shared:

   cat /sys/kernel/mm/ksm/pages_sharing
   cat /sys/kernel/mm/ksm/pages_shared
   

   Another useful thing is to use vhost-net kernel module to boost virtual machine network performance (ensure guest vm uses virtio network device).

   echo vhost-net >> /etc/modules
   

2. The KVM linux guest IO performance can be improved by:
   • using virtio as disk bus
   • setting virtual disk performance options to: cache mode - none, IO mode - native
   • using noop IO scheduler for each guest (file /etc/default/grub):

   GRUB_CMDLINE_LINUX_DEFAULT="quiet elevator=noop"
   

   Update grub by issuing update-grub command.

How to properly "halt" virtual machine in LXC

While running a number of virtual machines in LXC you might need gracefully shutdown each virtual machine while host reboot. Here is a script (file /usr/local/sbin/lxc-shutdown):

#!/bin/sh

name=$1
timeout=15

if lxc-info -n $name | grep -qs "STOPPED"
then
    echo $name not running...
    exit 0
fi                                           
                                                                               
ssh $name halt &                                                               
#if [ -e /usr/bin/lxc-halt ]; then                                             
#    /usr/bin/lxc-halt -n $name                                                
#else                                                                          
#    ssh $name halt &                                                          
#fi

while [ $timeout -gt 0 ]
do
    timeout=$(($timeout-1));sleep 1
    if lxc-info -n $name | grep -qs "STOPPED"
    then
        exit 0
    fi
done

lxc-stop -n $name
lxc-wait -n $name -s 'STOPPED'

This approach requires root to have password-less ssh login (see more here). So now that you have a script that let you halt gracefully virtual machine, let make few changes to /etc/init.d/lxc (somewhere around line 56):

# ...
    stop)
    log_daemon_msg "Stopping $DESC"
    #action_all "lxc-stop -n"
    # Uncomment below if you need to halt containers 
    # in reverse order
    CONTAINERS=`echo $CONTAINERS | tac -s ' '`
    action_all "lxc-halt"
    ;;
# ...

Use the following two commands to override lxc-shutdown for lxc v0.8+

                                           
update-alternatives --install /usr/bin/lxc-shutdown \                        
   lxc-shutdown /usr/local/sbin/lxc-shutdown 1                                
update-alternatives --set lxc-shutdown \                                     
   /usr/local/sbin/lxc-shutdown

Recovering from Ctrl+S in Putty

The problem is related to XON/XOFF command that is mapped to Ctrl+S sequence. The terminal doesn't echo the commands you issue, so you need to remember press Ctrl+Q in order to turn flow control ON. There is a way to ignore such behaviour. What you need to do is to change your terminal characteristics.

stty -ixon

Consider add this command to your /etc/profile.d/ixon.sh file.

Using TortoiseSVN SSH

TortoiseSVN is a windows shell extension for subversion. Here we are going access svn repository over ssh. You can read how to install and configure svn in this post, how to configure svnserve here and take a look at password-less ssh login here.

• TortoiseSVN > Settings > Network > SSH Client, browse for TortoisePlink.exe, typical path is "C:\Program Files\TortoiseSVN\bin\TortoisePlink.exe"
• In Checkout dialog enter path to the remote repository, e.g. svn+ssh://user1@deby/project1
• In popup window type password.
• If you setup password-less ssh login you need to add the private key to pageant (you can download it here). In this case authentication will go transparently.
• If you already have open ssh session via PuTTY, you can use tunneling feature. In PuTTY configuration, under Category Connection > SSH > Tunnels set Source port to 22, Destination to localhost:22. Click Add, Apply. In this case URL to repository will be svn+ssh://user1@localhost/project1

Read more about subversion here.

Combining port knocking and password-less ssh login to a single click

You need to follow previous posts related to port knocking and password-less ssh. Here is a script that combines both:

@echo off

set ip=XXX.XXX.XXX.XXX
cd nmap-5.00
cmd /c knockin.cmd %ip% AAA BBB CCC DDD

cd ..\putty
start putty.exe -file deby %ip%

Here are few comments to the script:

• Both nmap-5.00 and putty are sub directories of the script location.
• Replace XXX.XXX.XXX.XXX with your remote host ip address
• Replace AAA BBB CCC DDD with your knockin code
• Putty uses file session (settings) stored in file deby.

The only thing you have to do is create a shortcut to your quick launch toolbar and you are done.

Password-less ssh login

SSH is often used to login without requiring passwords. It requires you generate your own personal set of private/public pair.

RSA security key

Generate personal set of private/public pair (do not use a passphrase):

user1@deby:~$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user1/.ssh/id_rsa):
Created directory '/home/user1/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/user1/.ssh/id_rsa.
Your public key has been saved in /home/user1/.ssh/id_rsa.pub.
The key fingerprint is:
81:95:1a:bd:32:89:3b:c7:34:da:a2:a0:14:24:26:73 user1@deby
The key's randomart image is:
+--[ RSA 2048]----+
|       ...       |
|+oE   .oo        |
|=o   ..+..       |
| .  . B ..       |
|  .  * +S        |
|..  = +          |
|o. . +           |
|. .              |
|                 |
+-----------------+

Let ssh know your public key (here we are copy public ssh key from the client to remote server):

cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys

Secure .ssh directory so nobody except you can get access there:

user1@deby:~$ chmod -R go-rwx .ssh/
user1@deby:~$ ls -la .ssh
total 20
drwx------ 2 user1 user1 4096 2010-06-09 15:33 .
drwxr-xr-x 4 user1 user1 4096 2010-06-09 15:22 ..
-rw------- 1 user1 user1 393  2010-06-09 15:33 authorized_keys
-rw------- 1 user1 user1 1675 2010-06-09 15:22 id_rsa
-rw------- 1 user1 user1 393  2010-06-09 15:22 id_rsa.pub

Troubleshooting ssh localhost login

You might need this while using existing ssh tunneling feature, e.g. svn+ssh access.

user1@deby:~$ ssh deby
ssh_exchange_identification: Connection closed by remote host

You need to add localhost to /etc/hosts.allow, e.g.

sshd: localhost

Here is another issue that is related to pam_access module (if it configured to prohibit local logins):

user1@deby:~$ ssh deby
Connection closed by 127.0.0.1

Here is a rule that prohibit local logins except from localhost (file /etc/security/access.conf):

# Disallow console logins
- : ALL : LOCAL EXCEPT 127.0.0.1

Windows client

If you are using a windows machine to connect to your remote ssh server with PuTTY you need few extra steps to import private key.

• You need PuTTYgen. Download it from here.
• Import the key. Menu Conversions > Import key.
• Save private key (so PuTTY can understand it): Menu File > Save private key (do not set password).
• Load previously saved session in PuTTY
• In Category select Connection > Data, enter your remote username into Auto-login username
• In Category select Connection > SSH, choose SSH2 as your preferred protocol version
• In Category select Connection > SSH > Auth, browse the private key that you saved with PuTTYgen previously.
• Save your session

ssh-copy-id

Mac OS X doesn't come with ssh-copy-id, here is a single line command:

cat ~/.ssh/id_rsa.pub | ssh user@machine \
  "mkdir ~/.ssh; cat >> ~/.ssh/authorized_keys"

You can download script here.

Control remote access with pam_access

You need enabled pam_access module first. Uncomment the following line in /etc/pam.d/login and /etc/pam.d/sshd files:

account  required       pam_access.so

Secure Administrative Logins

Modify /etc/security/access.conf to disallow remote logins to administrative accounts, disallow local logins to non-administrative account. The order of entries is important:

#
# Disallow non-root logins on tty1
#
- : ALL EXCEPT root : tty1
#
# Allow root login on tty1
+ : root : tty1
#
# Disallow console logins
- : ALL : LOCAL
#
# ...
#
# User "root" should be denied to get access from all 
# other sources
- : root : ALL

Secure Network Logins

Setup a group to control users who can access the system remotely (via ssh).

groupadd -r sshusers

Modify /etc/security/access.conf in order to allow only sshusers group network access.

# Allow group 'sshusers' get access from everythere
+ : (sshusers) : ALL
#
# All other users should be denied to get access from 
# all sources.
- : ALL : ALL

Add users to group sshusers:

usermod -a -G sshusers user1

The changes take place immediately, you do not have to reboot.

Port knocking using iptables

The following let you in basic firewall through sequential port knocking to open SSH access for 5 seconds:

# ----------- BEGIN OF CUSTOM RULES -----------
#
# Note: Knock ports 100,200,300,400 to open SSH port for 5 seconds.
-N INTO-PHASE2
-A INTO-PHASE2 -m recent --name PHASE1 --remove
-A INTO-PHASE2 -m recent --name PHASE2 --set
-A INTO-PHASE2 -j LOG --log-prefix "INTO PHASE2: "
-A INTO-PHASE2 -j DROP
-N INTO-PHASE3
-A INTO-PHASE3 -m recent --name PHASE2 --remove
-A INTO-PHASE3 -m recent --name PHASE3 --set
-A INTO-PHASE3 -j LOG --log-prefix "INTO PHASE3: "
-A INTO-PHASE3 -j DROP
-N INTO-PHASE4
-A INTO-PHASE4 -m recent --name PHASE3 --remove
-A INTO-PHASE4 -m recent --name PHASE4 --set
-A INTO-PHASE4 -j LOG --log-prefix "INTO PHASE4: "
-A INTO-PHASE4 -j DROP

-A INPUT -m recent --name PHASE1 --update

-A INPUT -p tcp --dport 100 -i eth0 -m recent --set --name PHASE1
-A INPUT -p tcp --dport 200 -m recent --rcheck --name PHASE1 -j INTO-PHASE2
-A INPUT -p tcp --dport 300 -m recent --rcheck --name PHASE2 -j INTO-PHASE3
-A INPUT -p tcp --dport 400 -m recent --rcheck --name PHASE3 -j INTO-PHASE4

-A INPUT -p tcp --dport 22 -i eth0 -m recent --rcheck --seconds 5 --name PHASE4 -j ACCEPT

#
# ------------ END OF CUSTOM RULES ------------

If you are knocking from windows client you can use nmap tool. Download command-line zipfile nmap-5.21-win32.zip. Add to knockin.cmd:

@echo off
echo Knock in... %1
nmap -PN --host_timeout 1501 --max-retries 0 -p %2 %1 1>&0 2>&0
nmap -PN --host_timeout 1501 --max-retries 0 -p %3 %1 1>&0 2>&0
nmap -PN --host_timeout 1501 --max-retries 0 -p %4 %1 1>&0 2>&0
nmap -PN --host_timeout 1501 --max-retries 0 -p %5 %1 1>&0 2>&0

Run as the following (suppose you are knocking to 192.168.1.100):

C:\Program Files\nmap-5.00>knockin.cmd 192.168.1.100 100 200 300 400

Right after you issued above command the SSH port remains open for 5 seconds. Use your favorite SSH client to login. Just in case have a look here.

Make use of SSH

Secure Shell or SSH is a network protocol that allows data to be exchanged over a secure channel between two computers. SSH is typically used to log into a remote machine and execute commands, but it also supports tunneling, forwarding arbitrary TCP ports; file transfer can be accomplished using the associated SFTP or SCP protocols.

Install

Here is how to install it (Debian):

apt-get install ssh

Client

The ssh client configuration is in /etc/ssh/ssh_config. It recommended to change 'Protocol' line to (Only Protocol 2 will be used, since Protocol 1 is considered insecure):

Protocol 2

I would recommend you PuTTY Tray if you are connecting from Windows. You can also download sample registry sessions here.

Server

The SSH daemon configuration file can be found in /etc/ssh/sshd_config.
Disable SSH connections on ipv6:

#AddressFamily any # default
AddressFamily inet # IPv4 only
#AddressFamily inet6 # IPv6 only

To allow access only for some users add this line:

AllowUsers userA userB

However consider manage this at user group level:

AllowGroups sshusers

It is recommended prohibit root login:

PermitRootLogin no

Configure idle log out timeout interval (in seconds):

# Sets a timeout interval in seconds after which if no data has
# been received from the client, sshd will send a message through
# the encrypted channel to request a response from the client.  The
# default is 0, indicating that these messages will not be sent to
# the client.
ClientAliveInterval 300

# Sets the   number of client alive messages (see above) which may be sent
# without sshd receiving any messages back from the client.  If this
# threshold is reached while client alive messages are being sent, sshd
# will disconnect the client, terminating the session.
ClientAliveCountMax 0

Secure Server

To let other people ssh to your machine you need to adjust /etc/hosts.allow:

# let everyone connect to you
sshd: ALL
# OR you can restrict it to a certain ip
sshd: 192.168.0.1
# OR restrict for an IP range
sshd: 10.0.0.0/255.255.255.0
# OR restrict for an IP match
sshd: 192.168.1.

So with allowed rules we need prohibit everyone else /etc/hosts.deny:

ALL: ALL: DENY

Restart sshd deamon (Debian):

/etc/init.d/ssh restart

That's it. You can read more about ssh here. Best practices securing ssh are here.