Замер производительности дисковой подсистемы

от ·Нет комментариев
Существует множество способов и утилит произвести замер IOps и скорости чтения/записи дискового массива. Но к сожалению нет какого-то эталона или правила, как и с какими параметрами производить эти замеры. Поэтому в этой статье я опишу то, чем пользуюсь я и приведу примеры замеров того оборудования, с которым я сталкивался.
ИМХО: а вообще я считаю эти искусственные замеры не показывают, насколько хорошо годиться дисковая подсистема для конкретной задачи, слишком много условий. Поэтому замеры я делаю чисто понять, насколько новая железка более производительная чем старая.

В качестве инструмента я использую утилиту от компании Microsoft — DiskSpd.exe. Ссылка на скачивание exe. Она пришла на замену Powershell скрипту DiskPerformance.ps1 совместно с SQLIO.exe.
DiskSpd.exe запускается с определенным набором параметров из консоли CMD или Powershell.
Ниже пример команды, которой я произвожу тестирование:
PS C:\tmp\> .\diskspd.exe -c50G -d300 -r -w40 -t8 -o32 -b64K -Sh -L C:\tmp\testfile.dat
Описание основных параметров:
  • -c50G — тестирование проходит на файле размером в 50Гб. Чем больше файл, тем лучше, т.к. маленький файл может весь попасть в кэш контроллера.
  • -d300 — продолжительность измерения
  • -r — Random запись/чтение (если указать -s — будет последовательная запись/чтение)
  • -w40 — процент записи (в данном примере получается 60% на чтение и 40% на запись)
  • -t8 — кол-во потоков
  • -o32 — длина очереди
  • -b64K — размер блока
  • -Sh — не использовать кэширование
  • -L — измерять задержки (latency)
  • C:\tmp\testfile.dat — путь для тестового файла
Тестирование СХД Lenovo DS4200
Технические данные:
Сервер: ThinkSystem SN550 с ОС vmWare ESXi 6.7
Связь с СХД: Fibre Channel 16Gb/s
Тестовая VM: Windows 10 — 4Core, 4 Gb ram
 
Raid10 из 6-ти SSD 1.6Tb
Input parameters:
 
        timespan:   1
        ————-
        duration: 300s
        warm up time: 5s
        cool down time: 0s
        measuring latency
        random seed: 0
        path: ‘C:\temp\testfile.dat’
                think time: 0ms
                burst size: 0
                software cache disabled
                hardware write cache disabled, writethrough on
                performing mix test (read/write ratio: 60/40)
                block size: 65536
                using random I/O (alignment: 65536)
                number of outstanding I/O operations: 32
                thread stride size: 0
                threads per file: 8
                using I/O Completion Ports
                IO priority: normal
 
 
 
Results for timespan 1:
*******************************************************************************
 
actual test time:       300.00s
thread count:           8
proc count:             4
 
CPU |  Usage |  User  |  Kernel |  Idle
——————————————-
   0|  13.30%|   1.34%|   11.96%|  86.70%
   1|  13.41%|   1.44%|   11.97%|  86.59%
   2|  35.96%|   0.82%|   35.14%|  64.04%
   3|  13.22%|   1.47%|   11.76%|  86.78%
——————————————-
avg.|  18.97%|   1.27%|   17.71%|  81.03%
 
Total IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
——————————————————————————————————
     0 |     44280840192 |       675672 |     140.76 |    2252.24 |   14.212 |    28.072 | C:\temp\testfile.dat (50GiB)
     1 |     44284313600 |       675725 |     140.78 |    2252.42 |   14.214 |    21.080 | C:\temp\testfile.dat (50GiB)
     2 |     43600904192 |       665297 |     138.60 |    2217.66 |   14.436 |    20.308 | C:\temp\testfile.dat (50GiB)
     3 |     44350177280 |       676730 |     140.99 |    2255.77 |   14.188 |    19.784 | C:\temp\testfile.dat (50GiB)
     4 |     44225069056 |       674821 |     140.59 |    2249.40 |   14.232 |    23.223 | C:\temp\testfile.dat (50GiB)
     5 |     44279136256 |       675646 |     140.76 |    2252.15 |   14.219 |    23.678 | C:\temp\testfile.dat (50GiB)
     6 |     43532812288 |       664258 |     138.39 |    2214.19 |   14.471 |    26.346 | C:\temp\testfile.dat (50GiB)
     7 |     44311314432 |       676137 |     140.86 |    2253.79 |   14.203 |    20.604 | C:\temp\testfile.dat (50GiB)
——————————————————————————————————
total:      352864567296 |      5384286 |    1121.73 |   17947.61 |   14.271 |    23.059
 
Read IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
——————————————————————————————————
     0 |     26538147840 |       404940 |      84.36 |    1349.80 |   12.790 |     9.769 | C:\temp\testfile.dat (50GiB)
     1 |     26569801728 |       405423 |      84.46 |    1351.41 |   12.789 |     9.769 | C:\temp\testfile.dat (50GiB)
     2 |     26155745280 |       399105 |      83.15 |    1330.35 |   13.022 |     9.901 | C:\temp\testfile.dat (50GiB)
     3 |     26604208128 |       405948 |      84.57 |    1353.16 |   12.804 |     9.813 | C:\temp\testfile.dat (50GiB)
     4 |     26480017408 |       404053 |      84.18 |    1346.84 |   12.793 |     9.774 | C:\temp\testfile.dat (50GiB)
     5 |     26542604288 |       405008 |      84.38 |    1350.03 |   12.782 |     9.746 | C:\temp\testfile.dat (50GiB)
     6 |     26122584064 |       398599 |      83.04 |    1328.66 |   12.988 |     9.745 | C:\temp\testfile.dat (50GiB)
     7 |     26560561152 |       405282 |      84.43 |    1350.94 |   12.780 |     9.756 | C:\temp\testfile.dat (50GiB)
——————————————————————————————————
total:      211573669888 |      3228358 |     672.57 |   10761.19 |   12.843 |     9.785
 
Write IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
——————————————————————————————————
     0 |     17742692352 |       270732 |      56.40 |     902.44 |   16.340 |    42.620 | C:\temp\testfile.dat (50GiB)
     1 |     17714511872 |       270302 |      56.31 |     901.01 |   16.352 |    30.985 | C:\temp\testfile.dat (50GiB)
     2 |     17445158912 |       266192 |      55.46 |     887.31 |   16.557 |    29.602 | C:\temp\testfile.dat (50GiB)
     3 |     17745969152 |       270782 |      56.41 |     902.61 |   16.264 |    28.752 | C:\temp\testfile.dat (50GiB)
     4 |     17745051648 |       270768 |      56.41 |     902.56 |   16.378 |    34.552 | C:\temp\testfile.dat (50GiB)
     5 |     17736531968 |       270638 |      56.38 |     902.13 |   16.368 |    35.352 | C:\temp\testfile.dat (50GiB)
     6 |     17410228224 |       265659 |      55.35 |     885.53 |   16.695 |    39.809 | C:\temp\testfile.dat (50GiB)
     7 |     17750753280 |       270855 |      56.43 |     902.85 |   16.333 |    30.162 | C:\temp\testfile.dat (50GiB)
——————————————————————————————————
total:      141290897408 |      2155928 |     449.15 |    7186.42 |   16.410 |    34.306
 
 
 
total:
  %-ile |  Read (ms) | Write (ms) | Total (ms)
———————————————-
    min |      4.273 |      4.676 |      4.273
   25th |      9.180 |     10.148 |      9.414
   50th |     10.746 |     12.973 |     11.508
   75th |     13.249 |     15.445 |     14.376
   90th |     16.570 |     21.192 |     17.932
   95th |     19.865 |     25.831 |     23.767
   99th |     67.118 |    108.693 |     78.532
3-nines |    109.000 |    315.040 |    229.451
4-nines |    141.611 |   1014.329 |    687.713
5-nines |    235.820 |   3179.126 |   2023.188
6-nines |    246.224 |   8088.128 |   6240.128
7-nines |    256.618 |  15750.365 |  15750.365
8-nines |    256.618 |  15750.365 |  15750.365
9-nines |    256.618 |  15750.365 |  15750.365
    max |    256.618 |  15750.365 |  15750.365
Raid6 из 8-ми SSD 1.6Tb
Input parameters:
 
        timespan:   1
        ————-
        duration: 300s
        warm up time: 5s
        cool down time: 0s
        measuring latency
        random seed: 0
        path: ‘C:\temp\testfile.dat’
                think time: 0ms
                burst size: 0
                software cache disabled
                hardware write cache disabled, writethrough on
                performing mix test (read/write ratio: 60/40)
                block size: 65536
                using random I/O (alignment: 65536)
                number of outstanding I/O operations: 32
                thread stride size: 0
                threads per file: 8
                using I/O Completion Ports
                IO priority: normal
 
 
 
Results for timespan 1:
*******************************************************************************
 
actual test time:       300.00s
thread count:           8
proc count:             4
 
CPU |  Usage |  User  |  Kernel |  Idle
——————————————-
   0|  13.30%|   1.34%|   11.96%|  86.70%
   1|  13.41%|   1.44%|   11.97%|  86.59%
   2|  35.96%|   0.82%|   35.14%|  64.04%
   3|  13.22%|   1.47%|   11.76%|  86.78%
——————————————-
avg.|  18.97%|   1.27%|   17.71%|  81.03%
 
Total IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
——————————————————————————————————
     0 |     44280840192 |       675672 |     140.76 |    2252.24 |   14.212 |    28.072 | C:\temp\testfile.dat (50GiB)
     1 |     44284313600 |       675725 |     140.78 |    2252.42 |   14.214 |    21.080 | C:\temp\testfile.dat (50GiB)
     2 |     43600904192 |       665297 |     138.60 |    2217.66 |   14.436 |    20.308 | C:\temp\testfile.dat (50GiB)
     3 |     44350177280 |       676730 |     140.99 |    2255.77 |   14.188 |    19.784 | C:\temp\testfile.dat (50GiB)
     4 |     44225069056 |       674821 |     140.59 |    2249.40 |   14.232 |    23.223 | C:\temp\testfile.dat (50GiB)
     5 |     44279136256 |       675646 |     140.76 |    2252.15 |   14.219 |    23.678 | C:\temp\testfile.dat (50GiB)
     6 |     43532812288 |       664258 |     138.39 |    2214.19 |   14.471 |    26.346 | C:\temp\testfile.dat (50GiB)
     7 |     44311314432 |       676137 |     140.86 |    2253.79 |   14.203 |    20.604 | C:\temp\testfile.dat (50GiB)
——————————————————————————————————
total:      352864567296 |      5384286 |    1121.73 |   17947.61 |   14.271 |    23.059
 
Read IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
——————————————————————————————————
     0 |     26538147840 |       404940 |      84.36 |    1349.80 |   12.790 |     9.769 | C:\temp\testfile.dat (50GiB)
     1 |     26569801728 |       405423 |      84.46 |    1351.41 |   12.789 |     9.769 | C:\temp\testfile.dat (50GiB)
     2 |     26155745280 |       399105 |      83.15 |    1330.35 |   13.022 |     9.901 | C:\temp\testfile.dat (50GiB)
     3 |     26604208128 |       405948 |      84.57 |    1353.16 |   12.804 |     9.813 | C:\temp\testfile.dat (50GiB)
     4 |     26480017408 |       404053 |      84.18 |    1346.84 |   12.793 |     9.774 | C:\temp\testfile.dat (50GiB)
     5 |     26542604288 |       405008 |      84.38 |    1350.03 |   12.782 |     9.746 | C:\temp\testfile.dat (50GiB)
     6 |     26122584064 |       398599 |      83.04 |    1328.66 |   12.988 |     9.745 | C:\temp\testfile.dat (50GiB)
     7 |     26560561152 |       405282 |      84.43 |    1350.94 |   12.780 |     9.756 | C:\temp\testfile.dat (50GiB)
——————————————————————————————————
total:      211573669888 |      3228358 |     672.57 |   10761.19 |   12.843 |     9.785
 
Write IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
——————————————————————————————————
     0 |     17742692352 |       270732 |      56.40 |     902.44 |   16.340 |    42.620 | C:\temp\testfile.dat (50GiB)
     1 |     17714511872 |       270302 |      56.31 |     901.01 |   16.352 |    30.985 | C:\temp\testfile.dat (50GiB)
     2 |     17445158912 |       266192 |      55.46 |     887.31 |   16.557 |    29.602 | C:\temp\testfile.dat (50GiB)
     3 |     17745969152 |       270782 |      56.41 |     902.61 |   16.264 |    28.752 | C:\temp\testfile.dat (50GiB)
     4 |     17745051648 |       270768 |      56.41 |     902.56 |   16.378 |    34.552 | C:\temp\testfile.dat (50GiB)
     5 |     17736531968 |       270638 |      56.38 |     902.13 |   16.368 |    35.352 | C:\temp\testfile.dat (50GiB)
     6 |     17410228224 |       265659 |      55.35 |     885.53 |   16.695 |    39.809 | C:\temp\testfile.dat (50GiB)
     7 |     17750753280 |       270855 |      56.43 |     902.85 |   16.333 |    30.162 | C:\temp\testfile.dat (50GiB)
——————————————————————————————————
total:      141290897408 |      2155928 |     449.15 |    7186.42 |   16.410 |    34.306
 
 
 
total:
  %-ile |  Read (ms) | Write (ms) | Total (ms)
———————————————-
    min |      4.273 |      4.676 |      4.273
   25th |      9.180 |     10.148 |      9.414
   50th |     10.746 |     12.973 |     11.508
   75th |     13.249 |     15.445 |     14.376
   90th |     16.570 |     21.192 |     17.932
   95th |     19.865 |     25.831 |     23.767
   99th |     67.118 |    108.693 |     78.532
3-nines |    109.000 |    315.040 |    229.451
4-nines |    141.611 |   1014.329 |    687.713
5-nines |    235.820 |   3179.126 |   2023.188
6-nines |    246.224 |   8088.128 |   6240.128
7-nines |    256.618 |  15750.365 |  15750.365
8-nines |    256.618 |  15750.365 |  15750.365
9-nines |    256.618 |  15750.365 |  15750.365
    max |    256.618 |  15750.365 |  15750.365
Тестирование СХД Lenovo V3700v2
Технические данные:
Сервер: ThinkSystem SR650 с ОС vmWare ESXi 6.7
Связь с СХД: SAS 12Gb/s
Тестовая VM: Windows 10 — 4Core, 4 Gb ram
 
Raid10 из 8-ми HDD 1.2Tb
Input parameters:
 
        timespan:   1
        ————-
        duration: 300s
        warm up time: 5s
        cool down time: 0s
        measuring latency
        random seed: 0
        path: ‘C:\tmp\testfile.dat’
                think time: 0ms
                burst size: 0
                software cache disabled
                hardware write cache disabled, writethrough on
                performing mix test (read/write ratio: 60/40)
                block size: 65536
                using random I/O (alignment: 65536)
                number of outstanding I/O operations: 32
                thread stride size: 0
                threads per file: 8
                using I/O Completion Ports
                IO priority: normal
 
 
 
Results for timespan 1:
*******************************************************************************
 
actual test time:       300.00s
thread count:           8
proc count:             2
 
CPU |  Usage |  User  |  Kernel |  Idle
——————————————-
   0|   5.36%|   1.10%|    4.26%|  94.64%
   1|  11.41%|   1.17%|   10.24%|  88.59%
——————————————-
avg.|   8.39%|   1.14%|    7.25%|  91.61%
 
Total IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
——————————————————————————————————
     0 |      3516465152 |        53657 |      11.18 |     178.86 |  178.896 |    89.420 | C:\tmp\testfile.dat (50GiB)
     1 |      3501850624 |        53434 |      11.13 |     178.11 |  179.654 |    87.727 | C:\tmp\testfile.dat (50GiB)
     2 |      3506307072 |        53502 |      11.15 |     178.34 |  179.440 |    92.605 | C:\tmp\testfile.dat (50GiB)
     3 |      3503489024 |        53459 |      11.14 |     178.20 |  179.582 |    90.849 | C:\tmp\testfile.dat (50GiB)
     4 |      3500670976 |        53416 |      11.13 |     178.05 |  179.725 |    91.092 | C:\tmp\testfile.dat (50GiB)
     5 |      3506044928 |        53498 |      11.15 |     178.33 |  179.454 |    88.880 | C:\tmp\testfile.dat (50GiB)
     6 |      3505717248 |        53493 |      11.14 |     178.31 |  179.456 |    90.562 | C:\tmp\testfile.dat (50GiB)
     7 |      3504930816 |        53481 |      11.14 |     178.27 |  179.503 |    89.441 | C:\tmp\testfile.dat (50GiB)
——————————————————————————————————
total:       28045475840 |       427940 |      89.15 |    1426.46 |  179.463 |    90.083
 
Read IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
——————————————————————————————————
     0 |      2113011712 |        32242 |       6.72 |     107.47 |  187.230 |    63.483 | C:\tmp\testfile.dat (50GiB)
     1 |      2106589184 |        32144 |       6.70 |     107.15 |  187.787 |    63.673 | C:\tmp\testfile.dat (50GiB)
     2 |      2097348608 |        32003 |       6.67 |     106.68 |  187.883 |    63.435 | C:\tmp\testfile.dat (50GiB)
     3 |      2095841280 |        31980 |       6.66 |     106.60 |  187.880 |    62.855 | C:\tmp\testfile.dat (50GiB)
     4 |      2102394880 |        32080 |       6.68 |     106.93 |  187.806 |    62.939 | C:\tmp\testfile.dat (50GiB)
     5 |      2103181312 |        32092 |       6.69 |     106.97 |  188.073 |    63.715 | C:\tmp\testfile.dat (50GiB)
     6 |      2098987008 |        32028 |       6.67 |     106.76 |  187.614 |    63.369 | C:\tmp\testfile.dat (50GiB)
     7 |      2097348608 |        32003 |       6.67 |     106.68 |  188.060 |    63.543 | C:\tmp\testfile.dat (50GiB)
——————————————————————————————————
total:       16814702592 |       256572 |      53.45 |     855.24 |  187.791 |    63.378
 
Write IO
thread |       bytes     |     I/Os     |    MiB/s   |  I/O per s |  AvgLat  | LatStdDev |  file
——————————————————————————————————
     0 |      1403453440 |        21415 |       4.46 |      71.38 |  166.348 |   117.068 | C:\tmp\testfile.dat (50GiB)
     1 |      1395261440 |        21290 |       4.44 |      70.97 |  167.375 |   113.770 | C:\tmp\testfile.dat (50GiB)
     2 |      1408958464 |        21499 |       4.48 |      71.66 |  166.872 |   122.829 | C:\tmp\testfile.dat (50GiB)
     3 |      1407647744 |        21479 |       4.47 |      71.60 |  167.228 |   120.021 | C:\tmp\testfile.dat (50GiB)
     4 |      1398276096 |        21336 |       4.44 |      71.12 |  167.573 |   120.714 | C:\tmp\testfile.dat (50GiB)
     5 |      1402863616 |        21406 |       4.46 |      71.35 |  166.532 |   115.665 | C:\tmp\testfile.dat (50GiB)
     6 |      1406730240 |        21465 |       4.47 |      71.55 |  167.282 |   119.163 | C:\tmp\testfile.dat (50GiB)
     7 |      1407582208 |        21478 |       4.47 |      71.59 |  166.753 |   116.755 | C:\tmp\testfile.dat (50GiB)
——————————————————————————————————
total:       11230773248 |       171368 |      35.70 |     571.23 |  166.995 |   118.286
 
 
 
total:
  %-ile |  Read (ms) | Write (ms) | Total (ms)
———————————————-
    min |     47.425 |     46.367 |     46.367
   25th |    149.868 |    129.191 |    139.985
   50th |    178.619 |    152.818 |    166.230
   75th |    213.378 |    174.553 |    197.617
   90th |    259.509 |    199.680 |    244.662
   95th |    293.236 |    239.224 |    287.586
   99th |    395.146 |    650.713 |    534.227
3-nines |    693.805 |   1597.480 |   1197.029
4-nines |    813.842 |   2602.867 |   2247.943
5-nines |    876.860 |   3913.429 |   3344.462
6-nines |    960.990 |   6291.888 |   6291.888
7-nines |    960.990 |   6291.888 |   6291.888
8-nines |    960.990 |   6291.888 |   6291.888
9-nines |    960.990 |   6291.888 |   6291.888
    max |    960.990 |   6291.888 |   6291.888
Тестирование СХД Dell SC 4020
Технические данные:
Сервер: ThinkSystem SR650 с ОС vmWare ESXi 6.7
Связь с СХД: ISCSI 10Gb/s
Тестовая VM: Windows 10 — 2Core, 4 Gb ram
 
Tier1 Raid10 из 6-ти SSD 900Gb

Input parameters:

timespan: 1
————-
duration: 300s
warm up time: 5s
cool down time: 0s
measuring latency
random seed: 0
path: ‘C:\temp\testfile.dat’
think time: 0ms
burst size: 0
software cache disabled
hardware write cache disabled, writethrough on
performing mix test (read/write ratio: 60/40)
block size: 65536
using random I/O (alignment: 65536)
number of outstanding I/O operations: 32
thread stride size: 0
threads per file: 8
using I/O Completion Ports
IO priority: normal

System information:

computer name: DESKTOP-03410T6
start time: 2019/10/04 10:28:37 UTC

Results for timespan 1:
*******************************************************************************

actual test time: 300.02s
thread count: 8
proc count: 2

CPU | Usage | User | Kernel | Idle
——————————————-
0| 30.84%| 2.54%| 28.31%| 69.16%
1| 31.12%| 2.63%| 28.49%| 68.88%
——————————————-
avg.| 30.98%| 2.58%| 28.40%| 69.02%

Total IO
thread | bytes | I/Os | MiB/s | I/O per s | AvgLat | LatStdDev | file
——————————————————————————————————
0 | 45014319104 | 686864 | 143.09 | 2289.43 | 13.977 | 6.015 | C:\temp\testfile.dat (50GiB)
1 | 45095321600 | 688100 | 143.35 | 2293.55 | 13.952 | 5.956 | C:\temp\testfile.dat (50GiB)
2 | 45007634432 | 686762 | 143.07 | 2289.09 | 13.979 | 6.426 | C:\temp\testfile.dat (50GiB)
3 | 45050494976 | 687416 | 143.20 | 2291.27 | 13.966 | 6.373 | C:\temp\testfile.dat (50GiB)
4 | 45007962112 | 686767 | 143.07 | 2289.10 | 13.979 | 6.095 | C:\temp\testfile.dat (50GiB)
5 | 45066354688 | 687658 | 143.25 | 2292.07 | 13.961 | 6.026 | C:\temp\testfile.dat (50GiB)
6 | 44991053824 | 686509 | 143.02 | 2288.24 | 13.984 | 6.147 | C:\temp\testfile.dat (50GiB)
7 | 45058424832 | 687537 | 143.23 | 2291.67 | 13.964 | 6.722 | C:\temp\testfile.dat (50GiB)
——————————————————————————————————
total: 360291565568 | 5497613 | 1145.28 | 18324.43 | 13.970 | 6.225

Read IO
thread | bytes | I/Os | MiB/s | I/O per s | AvgLat | LatStdDev | file
——————————————————————————————————
0 | 26977763328 | 411648 | 85.76 | 1372.09 | 14.263 | 4.891 | C:\temp\testfile.dat (50GiB)
1 | 27056340992 | 412847 | 86.01 | 1376.09 | 14.244 | 4.892 | C:\temp\testfile.dat (50GiB)
2 | 26995654656 | 411921 | 85.81 | 1373.00 | 14.256 | 4.887 | C:\temp\testfile.dat (50GiB)
3 | 27019444224 | 412284 | 85.89 | 1374.21 | 14.247 | 4.886 | C:\temp\testfile.dat (50GiB)
4 | 26950828032 | 411237 | 85.67 | 1370.72 | 14.260 | 4.877 | C:\temp\testfile.dat (50GiB)
5 | 27014070272 | 412202 | 85.87 | 1373.94 | 14.246 | 4.889 | C:\temp\testfile.dat (50GiB)
6 | 26993623040 | 411890 | 85.81 | 1372.90 | 14.262 | 4.887 | C:\temp\testfile.dat (50GiB)
7 | 27007320064 | 412099 | 85.85 | 1373.59 | 14.243 | 4.881 | C:\temp\testfile.dat (50GiB)
——————————————————————————————————
total: 216015044608 | 3296128 | 686.66 | 10986.52 | 14.253 | 4.886

Write IO
thread | bytes | I/Os | MiB/s | I/O per s | AvgLat | LatStdDev | file
——————————————————————————————————
0 | 18036555776 | 275216 | 57.33 | 917.34 | 13.549 | 7.363 | C:\temp\testfile.dat (50GiB)
1 | 18038980608 | 275253 | 57.34 | 917.46 | 13.514 | 7.244 | C:\temp\testfile.dat (50GiB)
2 | 18011979776 | 274841 | 57.26 | 916.09 | 13.565 | 8.193 | C:\temp\testfile.dat (50GiB)
3 | 18031050752 | 275132 | 57.32 | 917.06 | 13.545 | 8.087 | C:\temp\testfile.dat (50GiB)
4 | 18057134080 | 275530 | 57.40 | 918.39 | 13.560 | 7.536 | C:\temp\testfile.dat (50GiB)
5 | 18052284416 | 275456 | 57.38 | 918.14 | 13.534 | 7.388 | C:\temp\testfile.dat (50GiB)
6 | 17997430784 | 274619 | 57.21 | 915.35 | 13.565 | 7.638 | C:\temp\testfile.dat (50GiB)
7 | 18051104768 | 275438 | 57.38 | 918.08 | 13.546 | 8.767 | C:\temp\testfile.dat (50GiB)
——————————————————————————————————
total: 144276520960 | 2201485 | 458.62 | 7337.90 | 13.547 | 7.793

total:
%-ile | Read (ms) | Write (ms) | Total (ms)
———————————————-
min | 1.584 | 1.644 | 1.584
25th | 11.285 | 11.273 | 11.280
50th | 12.464 | 12.537 | 12.495
75th | 15.208 | 13.985 | 14.480
90th | 21.041 | 16.522 | 19.296
95th | 25.052 | 19.546 | 23.253
99th | 32.767 | 28.304 | 32.083
3-nines | 39.992 | 94.817 | 81.874
4-nines | 46.188 | 247.305 | 160.804
5-nines | 55.537 | 652.923 | 535.564
6-nines | 62.648 | 1091.422 | 949.683
7-nines | 65.221 | 2258.252 | 2258.252
8-nines | 65.221 | 2258.252 | 2258.252
9-nines | 65.221 | 2258.252 | 2258.252
max | 65.221 | 2258.252 | 2258.252

 

Тестирование СХД Lenovo DE4000h
Технические данные:
Сервер: ThinkSystem SR650 с ОС vmWare ESXi 6.7
Связь с СХД: SAN16Gb/s
Тестовая VM: Windows 10 — 4Core, 8Gb ram
 
Raid6 из 12-ти SSD 1600Gb

Input parameters:

timespan: 1
————-
duration: 300s
warm up time: 5s
cool down time: 0s
measuring latency
random seed: 0
path: ‘C:\tmp\testfile.dat’
think time: 0ms
burst size: 0
software cache disabled
hardware write cache disabled, writethrough on
performing mix test (read/write ratio: 60/40)
block size: 65536
using random I/O (alignment: 65536)
number of outstanding I/O operations: 32
thread stride size: 0
threads per file: 8
using I/O Completion Ports
IO priority: normal

System information:

computer name: DESKTOP-ILCC1T8
start time: 2020/06/24 18:38:41 UTC

Results for timespan 1:
*******************************************************************************

actual test time: 300.00s
thread count: 8
proc count: 4

CPU | Usage | User | Kernel | Idle
——————————————-
0| 12.11%| 1.54%| 10.57%| 87.89%
1| 38.89%| 1.15%| 37.73%| 61.11%
2| 11.30%| 1.41%| 9.89%| 88.70%
3| 11.64%| 1.41%| 10.23%| 88.36%
——————————————-
avg.| 18.48%| 1.38%| 17.11%| 81.52%

Total IO
thread | bytes | I/Os | MiB/s | I/O per s | AvgLat | LatStdDev | file
——————————————————————————————————
0 | 47784591360 | 729135 | 151.90 | 2430.45 | 13.167 | 8.482 | C:\tmp\testfile.dat (50GiB)
1 | 46319730688 | 706783 | 147.25 | 2355.94 | 13.583 | 8.590 | C:\tmp\testfile.dat (50GiB)
2 | 47801171968 | 729388 | 151.96 | 2431.29 | 13.163 | 8.474 | C:\tmp\testfile.dat (50GiB)
3 | 47799730176 | 729366 | 151.95 | 2431.22 | 13.162 | 8.471 | C:\tmp\testfile.dat (50GiB)
4 | 47771877376 | 728941 | 151.86 | 2429.80 | 13.172 | 8.495 | C:\tmp\testfile.dat (50GiB)
5 | 46314553344 | 706704 | 147.23 | 2355.68 | 13.586 | 8.602 | C:\tmp\testfile.dat (50GiB)
6 | 47797370880 | 729330 | 151.94 | 2431.10 | 13.164 | 8.475 | C:\tmp\testfile.dat (50GiB)
7 | 47802286080 | 729405 | 151.96 | 2431.35 | 13.162 | 8.474 | C:\tmp\testfile.dat (50GiB)
——————————————————————————————————
total: 379391311872 | 5789052 | 1206.05 | 19296.84 | 13.268 | 8.509

Read IO
thread | bytes | I/Os | MiB/s | I/O per s | AvgLat | LatStdDev | file
——————————————————————————————————
0 | 28635365376 | 436941 | 91.03 | 1456.47 | 12.479 | 8.370 | C:\tmp\testfile.dat (50GiB)
1 | 27787919360 | 424010 | 88.34 | 1413.37 | 12.850 | 8.493 | C:\tmp\testfile.dat (50GiB)
2 | 28664987648 | 437393 | 91.12 | 1457.98 | 12.495 | 8.531 | C:\tmp\testfile.dat (50GiB)
3 | 28679340032 | 437612 | 91.17 | 1458.71 | 12.482 | 8.418 | C:\tmp\testfile.dat (50GiB)
4 | 28613541888 | 436608 | 90.96 | 1455.36 | 12.500 | 8.518 | C:\tmp\testfile.dat (50GiB)
5 | 27764260864 | 423649 | 88.26 | 1412.16 | 12.872 | 8.638 | C:\tmp\testfile.dat (50GiB)
6 | 28665249792 | 437397 | 91.12 | 1457.99 | 12.486 | 8.462 | C:\tmp\testfile.dat (50GiB)
7 | 28652339200 | 437200 | 91.08 | 1457.33 | 12.495 | 8.508 | C:\tmp\testfile.dat (50GiB)
——————————————————————————————————
total: 227463004160 | 3470810 | 723.09 | 11569.37 | 12.580 | 8.493

Write IO
thread | bytes | I/Os | MiB/s | I/O per s | AvgLat | LatStdDev | file
——————————————————————————————————
0 | 19149225984 | 292194 | 60.87 | 973.98 | 14.197 | 8.545 | C:\tmp\testfile.dat (50GiB)
1 | 18531811328 | 282773 | 58.91 | 942.58 | 14.682 | 8.618 | C:\tmp\testfile.dat (50GiB)
2 | 19136184320 | 291995 | 60.83 | 973.32 | 14.163 | 8.288 | C:\tmp\testfile.dat (50GiB)
3 | 19120390144 | 291754 | 60.78 | 972.51 | 14.183 | 8.447 | C:\tmp\testfile.dat (50GiB)
4 | 19158335488 | 292333 | 60.90 | 974.44 | 14.176 | 8.359 | C:\tmp\testfile.dat (50GiB)
5 | 18550292480 | 283055 | 58.97 | 943.52 | 14.656 | 8.437 | C:\tmp\testfile.dat (50GiB)
6 | 19132121088 | 291933 | 60.82 | 973.11 | 14.180 | 8.392 | C:\tmp\testfile.dat (50GiB)
7 | 19149946880 | 292205 | 60.88 | 974.02 | 14.160 | 8.323 | C:\tmp\testfile.dat (50GiB)
——————————————————————————————————
total: 151928307712 | 2318242 | 482.97 | 7727.47 | 14.297 | 8.429

total:
%-ile | Read (ms) | Write (ms) | Total (ms)
———————————————-
min | 0.463 | 0.373 | 0.373
25th | 10.940 | 12.456 | 11.414
50th | 11.907 | 13.555 | 12.539
75th | 13.031 | 14.979 | 13.931
90th | 14.355 | 16.427 | 15.533
95th | 15.398 | 17.417 | 16.615
99th | 19.154 | 21.259 | 20.261
3-nines | 160.887 | 159.757 | 160.363
4-nines | 202.945 | 202.124 | 202.772
5-nines | 215.037 | 214.996 | 215.037
6-nines | 217.878 | 217.198 | 217.843
7-nines | 218.450 | 218.234 | 218.450
8-nines | 218.450 | 218.234 | 218.450
9-nines | 218.450 | 218.234 | 218.450
max | 218.450 | 218.234 | 218.450

Сводная таблица:

Дисковая система Условие Ср. скорость чтения MiB/s Ср. скорость записи MiB/s Ср. read IOps Ср. write IOps
Lenovo DS4200 Raid10 из 6-ти SSD 1.6Tb 672.57 449.15 10761.19 7186.42
Lenovo DS4200 Raid6 из 8-ми SSD 1.6Tb 395.01 263.85 6320.17 4221.67
Lenovo V3700v2
Raid10 из 8-ми HDD 10k 1.2Tb
 53.45 35.70 855.24 571.23
Dell SC 4020
Tier1 Raid10 из 6-ти SSD 900Gb
 686.66 458.62 10986.52 7337.90
Dell VRTX
Raid10 из 12-ти HDD 10k 600Gb
 78.43 52.33 1254.87 837.21
Lenovo DE4000h
Raid6 из 12-ти SSD 10k 1.6Tb
 723.09 482.97 11569.37 7727.47
 

Добавить комментарий

Ваш адрес email не будет опубликован.