iSCSI: Set up and use LIO targets
Hands-On Storage
Currently, three network technologies dominate the SAN: Fibre Channel (FC), iSCSI, and Fibre Channel over Ethernet (FCoE). Of the three, only iSCSI can be set up in a standard LAN without much effort. In contrast, FCoE necessitates changes to Ethernet, and Fibre Channel even needs a different network. iSCSI, however, just tunnels SCSI commands over standard TCP connections and can thus be routed over IP like many other protocols.
This means that iSCSI is not just a cheap alternative to the previously dominant Fibre Channel but is also very useful when you need to stage test systems. For example, you can simulate a storage rack with a virtual machine by installing a suitable iSCSI target on the virtual machine's operating system. I will look at exactly that use case in this article, with a special focus on hands-on implementation.
Until 2011, iSCSI Enterprise Target (IET) was the standard SCSI target in the Linux kernel [1]. It was then replaced by LIO (which is a brand name by Datera Inc.; see the Linux SCSI Target wiki [2] for more details). The wider support for various SCSI standards was a major improvement, in particular Asymmetric Logical Unit Assignment (ALUA) and SCSI-3 Persistent Reservations [3]. I will be returning to the latter later in the article.
LIO has been covered previously in Linux Magazine [4], but this article looks into the configuration options with the help of the lio-utils. These tools have been replaced with the newer targetcli, and I will look at how to install and manage this tool.
Installation
The operating system used for the iSCSI target here is openSUSE 13.2. For the test setups at least, you will want to choose the latest versions of the software, so I recommend building targetcli from the source code. The RPMs available on the web are older than version 3. If you want to use the approach described here to build your own package for the latest version, you will need some prior knowledge of building RPM packages.
If you install a new openSUSE 13.2 for your tests, it makes sense to install the C/C++ and Python developer environments while you are doing so. You will also need rpmbuild and Git, which Zypper can easily retrieve for you. Additionally, you need to install a number of Python packages retroactively, again with the help of Zypper (Listing 1).
Listing 1: Install Python Packages with Zypper
zypper in git zypper in rpmbuild zypper in python-parsing zypper in epydoc zypper in python-netifaces zypper in python-ipaddr zypper in python urwid zypper in python-prettytable
The sources for targetcli and the required rtslib or configshell are available on GitHub (Figure 1).
The next step is to run the make rpm command to build the RPM packages for configshell and rtslib. If you see an error message, this is probably because the specifications files include references to pyparsing instead of python-parsing. As a workaround, you can modify the specifications files and repeat the build.
Finally, you can install the RPM packages you just built. Next, build and install the RPM package for targetcli and launch the target service (Listing 2).
Listing 2: Launch the Target Service
# rpm -qa | egrep -e targetcli\|configshell\|rtslib python-rtslib-3.0.pre4.9~g6fd0bbf-1.noarch targetcli-3.0.pre4.3~g0fba804-1.noarch python-configshell-1.6.1~g020d540-1.noarch # chkconfig target on # service target start
With a little bit of practice, you should have no problem using the targetcli tool, and if you do get into trouble, the help command will prove useful. Additionally, you can find useful documentation on the LIO project website [5].
Many Roads …
Targetcli can use a variety of backstores that store the data. These can be, for example, disks or partitions, but also volumes. You can publish these backstores to your clients using iSCSI by first creating an iSCSI target. The targetcli command automatically generates the iSCSI Qualified Name (IQN), which acts as the address for the target in iSCSI.
Then, you just need to create a portal with an IP address on which your iSCSI target waits for TCP calls. There was no need to state the port (3260) because this is the default. As in production use, you want the same volume to be accessible via two IP addresses--high-availability paths (Figure 2). It makes sense for a test setup to disable iSCSI authentication and switch LIO to Demo mode (Figure 3).
The client operating system needs an iSCSI initiator, which it uses to log in to the iSCSI target. On Linux, you can use the open-iscsi package for this; it provides the iscsiadm command. The iSCSI initiator needs a configuration that in turn assigns at least one IQN. However, the operating system can generate this itself. For discovery and the client login, you could use something similar to Listing 3.
Listing 3: Discovery and Login
initiator # iscsiadm -m discovery -t st -p 10.20.10.1 10.20.10.1:3260,1 iqn.2003-01.org.linux-iscsi.target.x8664:sn.71604388deb0 10.30.10.1:3260,1 iqn.2003-01.org.linux-iscsi.target.x8664:sn.71604388deb0 [...] initiator # iscsiadm -m mode -p 10.20.10.1 -l Logging in to [iface: default, target: iqn.2003-01.org.linux-iscsi.target.\ x8664:sn.71604388deb0, portal: 10.20.10.1,3260] (multiple) Logging to [iface: default, target: iqn.2003-01.org.linux-iscsi.target.\ x8664:sn.71604388deb0, portal: 10.20.10.1,3260] successful. [...] initiator # fdisk -l [...] Disk /dev/sda: 107,4 GB, 107373133824 Bytes 25 heads, 63 sectors/track, 13054 cylinders, total 209713152 sectors Unist = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 4194304 bytes Disk identifier: 0x00000000 Disk /dev/sda doens't contain a valid partition table. Disk /dev/sdb: 107,4 GB, 107373133824 Bytes [...]
Attentive readers may have noticed that the example created two devices for the one volume client-side, because two paths exist. You would now need a multipathing tool on the client to detect that the two devices represent the same LUN.
Conclusions
With the principal introduced here, you could set up a Symantec cluster [6] with virtual machines, including multipathing and SCSI-3 fencing. One major element here is the support of SCSI-3 persistent reservations by LIO. Without this fencing, which other iSCSI targets do not offer, the Symantec cluster would not work. Other products, such as the SAP HANA in-memory database (of late) also need SCSI-3 persistent reservations.