ISCSI IET

Данный вариант не работает с кластером Hyper-V / Windows Server!
Этот же метод ISCSI использует Openfiler (openfileresa-2.99.1-x86_64-disc1.iso), поэтому его использовать не получтся, если нужен Web интерфейс нужно использовать Nas4Free или FreeNas.

Тут минимальная настройка, подходит для тестирования, не нужно это применять в реальных проектах.

Установка таргета

Устанавливаем в Ubuntu/ Debian

aptitude install iscsitarget iscsitarget-dkms

Правим файл для того чтобы можно было запустить:
/etc/default/iscsitarget

ISCSITARGET_ENABLE=true

Редактируем конфиг таргета в старых версиях конфиг тут:
/etc/ietd.conf

В новых тут:
/etc/iet/ietd.conf

Target iqn.2012-04.local.dim:ietd
        IncomingUser
        OutgoingUser
        Lun 0 Path=/dev/sdb,Type=fileio
        Alias LUN0

Разрешаем доступы клиентам, конфиг в старых версиях тут:
/etc/initiators.allow
В новых тут:
/etc/iet/initiators.allow
Указываем разрешения всем на все:

ALL ALL

Запуск :
/etc/init.d/iscsitarget start
Настройка закончена, можно приступать к тестам.

***********************************************************************************
***********************************************************************************

Тесты скорости винта с файловой системой XFS внутри раздающиго компьютера

dd if=/dev/zero of=/var/tst/test bs=512K count=5000
2621440000 bytes (2,6 GB) copied, 15,2617 s, 172 MB/s
dd if=/dev/zero of=/var/tst/test bs=512K count=10000
5242880000 bytes (5,2 GB) copied, 37,1689 s, 141 MB/s
dd if=/dev/zero of=/var/tst/test bs=512K count=20000
10485760000 bytes (10 GB) copied, 77,2739 s, 136 MB/s
write: (g=0): rw=write, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 1.59
Starting 1 process
write: Laying out IO file(s) (1 file(s) / 10240MB)
Jobs: 1 (f=1): [W] [100.0% done] [0K/128.9M /s] [0 /32.3K iops] [eta 00m:00s]
write: (groupid=0, jobs=1): err= 0: pid=1139
  write: io=10240MB, bw=134510KB/s, iops=33627 , runt= 77955msec
    clat (usec): min=1 , max=15204 , avg=29.04, stdev=519.11
     lat (usec): min=1 , max=15204 , avg=29.15, stdev=519.11
    bw (KB/s) : min=117120, max=1184439, per=100.02%, avg=134537.78, stdev=85233.52
  cpu          : usr=3.51%, sys=8.61%, ctx=6821, majf=1, minf=24
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued r/w/d: total=0/2621440/0, short=0/0/0
     lat (usec): 2=6.94%, 4=78.71%, 10=13.89%, 20=0.18%, 50=0.01%
     lat (usec): 100=0.01%, 250=0.01%
     lat (msec): 4=0.01%, 10=0.04%, 20=0.22%

Run status group 0 (all jobs):
  WRITE: io=10240MB, aggrb=134510KB/s, minb=137738KB/s, maxb=137738KB/s, mint=77955msec, maxt=77955msec

Disk stats (read/write):
  sda: ios=0/19511, merge=0/3, ticks=0/11096276, in_queue=11134484, util=99.67%

Скорость на принмающей стороне

write: (g=0): rw=write, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 1.59
Starting 1 process
write: Laying out IO file(s) (1 file(s) / 10240MB)
Jobs: 1 (f=1): [W] [99.2% done] [0K/63262K /s] [0 /15.5K iops] [eta 00m:01s]
write: (groupid=0, jobs=1): err= 0: pid=1661
  write: io=10240MB, bw=85278KB/s, iops=21319 , runt=122960msec
    clat (usec): min=1 , max=52817 , avg=46.32, stdev=474.63
     lat (usec): min=1 , max=52817 , avg=46.41, stdev=474.64
    bw (KB/s) : min= 7992, max=1904176, per=100.09%, avg=85357.91, stdev=140637.                 76
  cpu          : usr=1.70%, sys=4.78%, ctx=28516, majf=1, minf=24
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued r/w/d: total=0/2621440/0, short=0/0/0
     lat (usec): 2=23.24%, 4=66.01%, 10=8.50%, 20=1.11%, 50=0.08%
     lat (usec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
     lat (msec): 2=0.01%, 4=1.04%, 10=0.01%, 20=0.01%, 50=0.01%
     lat (msec): 100=0.01%

Run status group 0 (all jobs):
  WRITE: io=10240MB, aggrb=85277KB/s, minb=87324KB/s, maxb=87324KB/s, mint=12296                 0msec, maxt=122960msec

Disk stats (read/write):
  sdc: ios=0/18426, merge=0/3, ticks=0/17496032, in_queue=17557388, util=99.64%

 

dd if=/dev/zero of=/var/data/test bs=512K count=5000
2621440000 bytes (2,6 GB) copied, 16,3093 s, 161 MB/s

 

dd if=/dev/zero of=/var/data/test bs=512K count=10000
5242880000 bytes (5,2 GB) copied, 50,1288 s, 105 MB/s
dd if=/dev/zero of=/var/data/test bs=512K count=20000
10485760000 bytes (10 GB) copied, 118,308 s, 88,6 MB/s

Bonding

auto eth0
  iface eth0 inet manual
  bond-master bond0

auto eth1
  iface eth1 inet manual
  bond-master bond0

auto bond0
  iface bond0 inet static
  address 10.10.10.6
  netmask 255.255.255.0
  bond-mode 0
  bond-miimon 100
  bond-slaves none
  mtu 7000

Запись:

write: (g=0): rw=write, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 1.59
Starting 1 process
write: Laying out IO file(s) (1 file(s) / 10240MB)
Jobs: 1 (f=1): [W] [100.0% done] [0K/138.5M /s] [0 /34.7K iops] [eta 00m:00s]
write: (groupid=0, jobs=1): err= 0: pid=1954
  write: io=10240MB, bw=142058KB/s, iops=35514 , runt= 73813msec
    clat (usec): min=1 , max=709781 , avg=27.62, stdev=685.32
     lat (usec): min=1 , max=709781 , avg=27.70, stdev=685.32
    bw (KB/s) : min= 1154, max=1775224, per=100.66%, avg=142996.62, stdev=177575.10
  cpu          : usr=2.82%, sys=7.32%, ctx=15041, majf=1, minf=24
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued r/w/d: total=0/2621440/0, short=0/0/0
     lat (usec): 2=25.05%, 4=66.62%, 10=6.77%, 20=0.86%, 50=0.14%
     lat (usec): 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01%
     lat (msec): 2=0.01%, 4=0.51%, 10=0.01%, 20=0.02%, 50=0.01%
     lat (msec): 100=0.01%, 750=0.01%

Run status group 0 (all jobs):
  WRITE: io=10240MB, aggrb=142058KB/s, minb=145467KB/s, maxb=145467KB/s, mint=73813msec, maxt=73813msec

Disk stats (read/write):
  sdc: ios=0/18377, merge=0/2, ticks=0/10441548, in_queue=10484100, util=99.35%
dd if=/dev/zero of=/var/data/test bs=512K count=5000
2621440000 bytes (2,6 GB) copied, 11,4682 s, 229 MB/s

dd if=/dev/zero of=/var/data/test bs=512K count=10000
5242880000 bytes (5,2 GB) copied, 30,9697 s, 169 MB/s

dd if=/dev/zero of=/var/data/test bs=512K count=20000
10485760000 bytes (10 GB) copied, 74,9882 s, 140 MB/s

Чтение:

read: (g=0): rw=read, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 1.59
Starting 1 process
read: Laying out IO file(s) (1 file(s) / 5120MB)
Jobs: 1 (f=1): [R] [100.0% done] [135.8M/0K /s] [33.1K/0  iops] [eta 00m:00s]
read: (groupid=0, jobs=1): err= 0: pid=2012
  read : io=5120.0MB, bw=128119KB/s, iops=32029 , runt= 40922msec
    clat (usec): min=0 , max=38315 , avg=30.69, stdev=252.36
     lat (usec): min=0 , max=38315 , avg=30.76, stdev=252.36
    bw (KB/s) : min=112415, max=138024, per=100.13%, avg=128284.72, stdev=6172.92
  cpu          : usr=1.92%, sys=6.89%, ctx=21210, majf=1, minf=26
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued r/w/d: total=1310720/0/0, short=0/0/0
     lat (usec): 2=95.86%, 4=1.21%, 10=0.21%, 20=0.94%, 50=0.20%
     lat (usec): 100=0.02%, 500=0.01%, 750=0.01%, 1000=0.01%
     lat (msec): 2=1.14%, 4=0.41%, 10=0.01%, 20=0.01%, 50=0.01%

Run status group 0 (all jobs):
   READ: io=5120.0MB, aggrb=128118KB/s, minb=131193KB/s, maxb=131193KB/s, mint=40922msec, maxt=40922msec

Disk stats (read/write):
  sdc: ios=20418/3, merge=2/0, ticks=77504/1424, in_queue=78892, util=99.82%
dd of=/dev/zero if=/var/data/test bs=512K count=20000
10485760000 bytes (10 GB) copied, 65,1324 s, 161 MB/s

multipath-tools

defaults
{
      path_selector       "round-robin 0"
      path_grouping_policy multibus
      rr_min_io           200
      no_path_retry       5
      failback            immediate
      user_friendly_names yes
}
blacklist
{
            devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
            devnode "^hd[a-z][[0-9]*]"
            devnode "^sda$"
            devnode "^sdb$"

}
multipaths
{
   multipath
   {
      wwid 1IET_00010001
   }
}

Запись:

write: (g=0): rw=write, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 1.59
Starting 1 process
write: Laying out IO file(s) (1 file(s) / 10240MB)
Jobs: 1 (f=1): [W] [100.0% done] [0K/108.0M /s] [0 /27.0K iops] [eta 00m:00s]
write: (groupid=0, jobs=1): err= 0: pid=2128
  write: io=10240MB, bw=118331KB/s, iops=29582 , runt= 88614msec
    clat (usec): min=1 , max=18328 , avg=33.03, stdev=514.45
     lat (usec): min=1 , max=18328 , avg=33.15, stdev=514.45
    bw (KB/s) : min=87602, max=1901872, per=100.20%, avg=118568.61, stdev=140575.25
  cpu          : usr=2.74%, sys=8.67%, ctx=9742, majf=1, minf=24
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued r/w/d: total=0/2621440/0, short=0/0/0
     lat (usec): 2=9.64%, 4=72.10%, 10=15.69%, 20=2.03%, 50=0.15%
     lat (usec): 100=0.01%, 250=0.01%
     lat (msec): 4=0.01%, 10=0.31%, 20=0.06%

Run status group 0 (all jobs):
  WRITE: io=10240MB, aggrb=118330KB/s, minb=121170KB/s, maxb=121170KB/s, mint=88614msec, maxt=88614msec

Disk stats (read/write):
  dm-0: ios=0/17983, merge=0/4, ticks=0/12530912, in_queue=12591484, util=99.50%, aggrios=4/9012, aggrmerge=0/0, aggrticks=1980/3920492, aggrin_queue=3930372, aggrutil=92.22%
    sdc: ios=5/8983, merge=0/0, ticks=1952/4186472, in_queue=4201152, util=92.22%
    sdd: ios=4/9041, merge=0/0, ticks=2008/3654512, in_queue=3659592, util=86.93%
dd if=/dev/zero of=/var/data/test bs=512K count=20000
10485760000 bytes (10 GB) copied, 87,0831 s, 120 MB/s

Чтение:

read: (g=0): rw=read, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio 1.59
Starting 1 process
read: Laying out IO file(s) (1 file(s) / 5120MB)
Jobs: 1 (f=1): [R] [100.0% done] [103.1M/0K /s] [25.1K/0  iops] [eta 00m:00s]
read: (groupid=0, jobs=1): err= 0: pid=2110
  read : io=5120.0MB, bw=102822KB/s, iops=25705 , runt= 50990msec
    clat (usec): min=0 , max=29024 , avg=38.43, stdev=304.55
     lat (usec): min=0 , max=29024 , avg=38.50, stdev=304.55
    bw (KB/s) : min=90624, max=115760, per=100.17%, avg=102996.87, stdev=6601.59
  cpu          : usr=1.53%, sys=5.35%, ctx=21172, majf=1, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued r/w/d: total=1310720/0/0, short=0/0/0
     lat (usec): 2=96.66%, 4=1.16%, 10=0.02%, 20=0.59%, 50=0.01%
     lat (usec): 100=0.01%, 750=0.01%, 1000=0.01%
     lat (msec): 2=0.31%, 4=1.25%, 10=0.01%, 20=0.01%, 50=0.01%

Run status group 0 (all jobs):
   READ: io=5120.0MB, aggrb=102821KB/s, minb=105289KB/s, maxb=105289KB/s, mint=50990msec, maxt=50990msec

Disk stats (read/write):
  dm-0: ios=20478/3, merge=2/0, ticks=98048/5116, in_queue=103124, util=99.89%, aggrios=10244/1, aggrmerge=0/0, aggrticks=48990/8, aggrin_queue=48998, aggrutil=56.40%
    sdc: ios=10288/1, merge=0/0, ticks=55224/16, in_queue=55240, util=56.40%
    sdd: ios=10200/2, merge=0/0, ticks=42756/0, in_queue=42756, util=43.90%
dd of=/dev/zero if=/var/data/test bs=512K count=20000
10485760000 bytes (10 GB) copied, 73,0658 s, 144 MB/s
Запись опубликована в рубрике ISCSI. Добавьте в закладки постоянную ссылку.