Configuring ISCSI is simple, but the configuration options can be complex. This is a simple illustration of how you can configure an ISCSI target (storage space on a server) and connect to it using and initiator (client that will use the space). Everything will be as trim as possible, with basic IP restriction and simple CHAP security. If you want to get a bit fancier just look up the options for the various commands that are used most things that I’ve wanted to do have been simple modifications to this basic pattern.
For this example I will be using a virtual machine dev01 as the target that hosts the storage and a virtual machine dev02 as the initiator that connects to and uses the storage on dev01. Everything done in this example is done while logged in as root on the respective system.
Configuring the Target
Setting up the local firewall
If you are using netfilter (note the change to the firewalld service in fedora 16 from the iptables service in fedora 15) then you will need to allow incoming TCP connections to the port configured for ISCSI. The default port is 3260 and used throughout this post. You can change it, but if you do you will need to specify the port in all connection commands and change the port opened in netfilter. I used the following commands to log inbound connections, open the port and save it so that it will persist through reboots.
[root@dev01 ~]# iptables -I INPUT 3 -p tcp --dport 3260 -m state --state NEW,ESTABLISHED -j ACCEPT [root@dev01 ~]# iptables -I INPUT 3 -p tcp --dport 3260 -m limit --limit 5/m --limit-burst 7 -j LOG --log-prefix="iptables-iscsi: " [root@dev01 ~]# iptables-save > /etc/sysconfig/iptables
Using SELinux
If you are using SELinux then you may have to configure SELinux to allow tgtd access to the file. To do this the simplest way is to set SELinux to permissive complete the installation and testing then run audit2allow to create a new SELinux module based on the tgtd alerts in the audit file. So start by setting your SELinux status to permissive.
[root@dev01 ~]# setenforce 0
Once you have completed all of the configuration items in the following sections you should then run the audit2allow script for the tgtd service entries. If you are using files as storage rather than devices then you could just set the labels on the files to ‘system_u:object_r:tgtd_var_lib_t:s0′. However if you still have trouble then I suggest that you perform the following anyway.
[root@dev01 ~]# cat /var/log/audit/audit.log | audit2allow -M iscsi-disk1.pol [root@dev01 ~]# semodule -i iscsi-disk1.pol.pp
Creating the space
In ISCSI terms the target is the server on which the storage space actually resides. The storage space can be anything from a file on a disk system, a dedicated physical device, a RAM disk or even another network mounted storage device. In this case to prepare the device I will be creating a file for the storage space.
[root@dev01 ~]# mkdir /iscsi-disks [root@dev01 ~]# dd if=/dev/zero of=/iscsi-disks/iscsi-disk1 bs=1M count=2000
Installing the SCSI tools and services
First you need to install the tools and services required with their associated dependencies. Once the service has been installed it needs to be turned on and set to turn on at start up.
[root@dev01 ~]# yum -y install scsi-target-utils [root@dev01 ~]# service tgtd start [root@dev01 ~]# chkconfig tgtd on
Creating the ISCSI device
Next we get to actually configure the resource itself. This involves creating the ISCSI device and adding logical units (LUNs) to it. In this case we only add 1LUN but you could add more. The details for the ISCSI device are held in a database in ‘/var/lib/iscsi/nodes/’ and are manipulated and viewed using the ‘tgtadm’ tool. In the listing below the last command illustrates how to view the changes that you have just made.
We also create a name for the ISCSI device. The name could be anything but there is a standard to be applied that keeps us out of name collision difficulties. The ISCSI standard specifies ‘iqn.yyyy-mm.<reversed domain name>[:identifier]‘ as the standard naming convention. So in this case our device is named ‘iqn.2012-01.nz.net.chesterproductions:dev01′.
[root@dev01 ~]# tgtadm --lld iscsi --mode target --op new --tid=1 --targetname iqn.2012-01.nz.net.chesterproductions:dev01 [root@dev01 ~]# tgtadm --lld iscsi --mode logicalunit --op new --tid 1 --lun 1 -b /iscsi-disks/iscsi-disk1 [root@dev01 ~]# tgtadm --lld iscsi --mode target --op show
So that was the way to do it manually. If you want to have tgtd automatically load your device when it starts then you need to add an entry in to the ‘/etc/tgt/targets.conf’ file. The file installed by default is full of examples that are commented out. Given that there is so much fine information I’ll be brief and just provide an example that works for me. The one warning that I would give though is that this will be executed when tgtd starts, if you are like me and prefer to control when this appears then you may like to script the manual solution above somewhere more convenient for you.
<target iqn.2012-01.nz.net.chesterproductions:dev01>
backing-store /iscsi-disks/iscsi-disk1
incominguser auser secret
# outgoinguser anotheruser anothersecret
</target>
Configuring access to the device
Finally we configure the user details (CHAP target authentication in this case) and allow connections from any IP address opening it up to be accessed by the initiator (client that uses the storage space). Once this is done we are ready to configure and hook up our initiator. Note that this configuration will not persist between machine restarts but can be scripted to do so.
[root@dev01 ~]# tgtadm --lld iscsi --mode account --op new --user auser --password secret [root@dev01 ~]# tgtadm --lld iscsi --mode account --op bind --tid 1 --user auser [root@dev01 ~]# tgtadm --lld iscsi --mode target --op bind --tid 1 -I ALL [root@dev01 ~]# tgtadm --lld iscsi --mode account --op show
Configuring the Initiator (client)
Installing the ISCSI tools and services
First you need to install the tools and services required with their associated dependencies. Once the service has been installed it needs to be turned on and set to turn on at start up.
[root@dev02 ~]# yum -y install iscsi-initiator-utils [root@dev02 ~]# service iscsid start [root@dev02 ~]# chkconfig iscsid on
Creating the local ISCSI device
Once the services and tools have been installed you can create the ISCSI local device. This is all done through the ‘iscsiadm’ tool and like the ‘tgtadm’ tool on the target the ‘iscsiadm’ tool manipulates the database in ‘/var/lib/iscsi/nodes/’. You can use the command ‘iscsiadm -m node’ to view the discovered nodes in the database. There are a number of steps to the configuration, but basically you just need to discover the target – which puts its details in the database, configure the authentication details and then log in. At the end of this process ‘sfdisk’ should recognise a new ‘/dev/sdX’ device provided by ISCSI that it can format and use like any other.
[root@dev02 ~]# iscsiadm -m discovery -t st -p dev01 Starting iscsid: [ OK ] 192.168.122.11:3260,1 iqn.2012-01.nz.net.chesterproductions:dev01 [root@dev02 ~]# iscsiadm -m node 192.168.122.11:3260,1 iqn.2012-01.nz.net.chesterproductions:dev01 [root@dev02 ~]# iscsiadm -m node --targetname "iqn.2012-01.nz.net.chesterproductions:dev01" --portal dev01 --op update --name node.session.auth.authmethod --value CHAP [root@dev02 ~]# iscsiadm -m node --targetname "iqn.2012-01.nz.net.chesterproductions:dev01" --portal dev01 --op update --name node.session.auth.username --value auser [root@dev02 ~]# iscsiadm -m node --targetname "iqn.2012-01.nz.net.chesterproductions:dev01" --portal dev01 --op update --name node.session.auth.password --value password [root@dev02 ~]# iscsiadm -m node --targetname "iqn.2012-01.nz.net.chesterproductions:dev01" --portal dev01 --op update --name node.session.auth.password --value secret [root@dev02 ~]# iscsiadm -m node --targetname "iqn.2012-01.nz.net.chesterproductions:dev01" --portal dev01 --login
Format and mount the new disk
Next we make the device usable in just the same way as you would any other device. There’s no rocket science here.
[root@dev02 ~]# fdisk /dev/sda [root@dev02 ~]# mkfs.ext4 /dev/sda1 [root@dev02 ~]# mkdir /data [root@dev02 ~]# mount /dev/sda1 /data
Configure the disk to auto mount at boot
To make the device auto mount at boot all you need to do is add a standard entry to ‘/etc/fstab’ with the parameters of ‘defaults,auto,_netdev’. That’s it you can now use your ISCSI storage just like the on board disk physically in your machine.
If you have more than one disk then the ‘/dev/sdX/’ link is not guaranteed to be assigned to the same disk each time. Each disk takes the first available device name as it is connected. To mount specific disks to specific mount points you need to use the UUID. The UUID is a unique identifier assigned tot he disk whent he file system is created. To locate the UUID you can use the ‘blkid’ command as seen below.
Alternatively you can use the logical volume manager and it will take care of all of this for you.
[root@dev02 ~]# blkid | grep sda1 /dev/sda1: UUID="d894a751-93a4-4739-806b-b1f5fd30b4e1" TYPE="ext4" [root@dev02 ~]# vim /etc/fstab ... UUID=d894a751-93a4-4739-806b-b1f5fd30b4e1 /data ext4 defaults,auto,_netdev 0 0
Other Places to Look for Help
There are several other places that I used to look for help. These include the local man pages and the help pages that were installed with the software. The locally installed help pages have similar useful examples but go way beyond the scope of this simple introductory example. The following references were especially useful.
http://fedoraproject.org/wiki/Scsi-target-utils_Quickstart_Guide
http://www.howtoforge.com/using-iscsi-on-fedora-10-initiator-and-target
http://linux.die.net/man/8/tgtadm