HP ProLiant DL380 G5 Tuning Guide

From PostgreSQL Wiki

Jump to: navigation, search

This hardware was generously donated by HP.

Contents

Upcoming tests

  • SLUB vs. SLAB (memory management)
  • shared_buffers
  • effective cache size
  • physical disk partitioning
  • number of connections
  • stats_targets (dbt3/tpc-h dependent)
  • auto vacuum
  • I/O Elevators
  • SCSI controller tuning (might not be possible)

Hardware Details

  • 2 x Quad Core Intel(R) Xeon(R) E5405 (2.00GHz, 1333MHz FSB, 80W) Processor
  • HP 32GB Fully Buffered DIMM PC2-5300 8x4GB DR LP Memory
  • HP Smart Array P800 Controller
  • 8 x HP 72GB Hot Plug 2.5 SAS 15,000 rpm Hard Drive
  • MSA70 with 25 x HP 72GB Hot Plug 2.5 SAS 15,000 rpm Hard Drive

Failed Hardware

Software Details

  • 2.6.25-gentoo-r6 (Gentoo)

Hardware Baseline

fio

We will use fio v1.20 within the autotest framework to get some simple system characteristics using the internal disks of the system on various file systems, software and hardware raid configurations, and volume management configurations. 5 simple tests will be run: a sequential read, sequential write, random read, random write, and read-write test.

sequential read 

[seq-read]
rw=read
; this will be total of all individual files per process, we want to total
; 2x physical ram
size=8g
directory=/test
fadvise_hint=0
blocksize=8k
direct=0
numjobs=8
; this is number of files total per process
nrfiles=1
runtime=1h
exec_prerun=echo 3 > /proc/sys/vm/drop_caches

sequential write 

[seq-write]
rw=write
; this will be total of all individual files per process, we want to total
; 2x physical ram
size=8g
directory=/test
fadvise_hint=0
blocksize=8k
direct=0
numjobs=8
; this is number of files total per process
nrfiles=1
runtime=1h
exec_prerun=echo 3 > /proc/sys/vm/drop_caches


random read 

[random-read]
rw=randread
; this will be total of all individual files per process, we want to total
; 2x physical ram
size=8g
directory=/test
fadvise_hint=0
blocksize=8k
direct=0
numjobs=8
; this is number of files total per process
nrfiles=1
runtime=1h
exec_prerun=echo 3 > /proc/sys/vm/drop_caches

random write 

[random-write]
rw=randwrite
; this will be total of all individual files per process, we want to total
; 2x physical ram
size=8g
directory=/test
fadvise_hint=0
blocksize=8k
direct=0
numjobs=8
; this is number of files total per process
nrfiles=1
runtime=1h
exec_prerun=echo 3 > /proc/sys/vm/drop_caches

read-write

[read-write]
rw=rw
rwmixread=50
; this will be total of all individual files per process, we want to total
; 2x physical ram
size=8g
directory=/test
fadvise_hint=0
blocksize=8k
direct=0
numjobs=8
; this is number of files total per process
nrfiles=1
runtime=1h
exec_prerun=echo 3 > /proc/sys/vm/drop_caches

Results

One Disk

Aggregate Bandwidth (MB/s) - No partition table
sequential read sequential write random read random write read-write
ext2 57.9 89.0 2.3 6.6 8.0/5.6
ext2-noatime 59.4 92.0 2.3 6.5 8.0/6.8
ext3 59.4 81.8 2.3 22.8 8.2/7.0
ext3 (noatime)
ext3 (data=journal) 61.4 72.4 2.3 11.0 7.5/7.5
ext3 (data=journal,noatime} 60.8 71.2 2.3 11.0 7.5/7.5
jfs 65.4 97.0 2.3 4.8 8.2/7.0
jfs (noatime) 65.5 95.0 2.3 4.8 6.8/8.0
xfs 71.9 96.2 * * *
xfs (noatime) 68.8 96.3 2.3 * *
  • - Test did not complete successfully.

Two Disks

Software Raid

Aggregate Bandwidth (MB/s) - RAID 0 (64KB chunk) - No partition table
sequential read sequential write random read random write read-write
ext2 87.0 177.0 4.2 11.9 13.6/13.6
ext2-noatime 80.2 180.0 4.3 12.0 13.4/13.4
ext3 86.2 158 4.2 38.1 17.8/9.0
ext3 (noatime) 88.2 161.0 4.3 39.6 13.4/13.4
ext3 (data=journal) 85.8 83.0 4.2 17.1 7.9/13.9
ext3 (data=journal,noatime) 87.6 83.4 4.2 17.1 10.3/14.4
jfs 88.0 181 4.1 7.9 14.1/18.8
jfs (noatime) 88.7 175 4.3 7.8 16.4/16.8
reiserfs 96.2 116 4.3 64.5 13.6/18.2
reiserfs (noatime) 92.6 114 4.3 64.5 16.0/11.4

Hardware Raid

Aggregate Bandwidth (MB/s) - RAID 0 (128KB stripe) - No partition table
sequential read sequential write random read random write read-write
ext2 78.8 166.0 4.3 12.0 17.8/13.4
ext2-noatime 78.9 170.0 4.3 12.0 13.4/15.6
ext3 73.9 82.5 4.3 41.0 13.0/13.0
ext3 (noatime) 74.5 77.5 4.3 48.0 15.2/11.0
ext3 (data=journal) 78.9 75.9 4.3 16.6 15.8/15.8
ext3 (data=journal,noatime) 78.0 75.9 4.3 16.4 13.8/9.9
jfs 78.8 128.0 4.4 8.0 16.2/15.9
jfs (noatime) 79.3 133.0 4.4 8.0 18.2/11.5
reiserfs 75 115 4.32 61.2 13.1/10.1
reiserfs (noatime) ND ND ND ND ND

LVM

Aggregate Bandwidth (MB/s) - 4MB Stripesize - No partition table
sequential read sequential write random read random write read-write
ext2 136.0 186.0 4.5 11.9 26.4/26.4
ext2 (noatime) 137.0 190.0 4.5 12.1 23.2/23.2
ext3 129.0 161.0 4.5 38.9 12.8/22.8
ext3 (noatime) 136.0 169.0 4.5 38.4 26.2/19.9
ext3 (data=journal) 135.0 90.4 4.4 16.6 15.2/21.2
ext3 (data=journal,noatime) 137.0 89.6 4.5 16.9 21.2/15.2
jfs 148.0 180.0 4.5 7.9 17.9/25.0
jfs (noatime) 147.0 180.0 4.5 8.1 24.9/21.4

Three Disks

Software RAID

Aggregate Bandwidth (MB/s) - RAID 5 (4MB chunk) - No partition table
sequential read sequential write random read random write read-write
ext2 200.0 43.1 5.8 4.2 8.2/9.9
ext2 (noatime) 200.0 43.3 5.8 4.2 11.5/9.9
ext3 199.0 45.6 5.7 9.6 24.0/24.0
ext3 (noatime) 201.0 46.3 5.7 9.5 4.7/12.6
ext3 (data=journal) 199.0 24.3 5.8 4.7 4.7/7.3
ext3 (data=journal,noatime) 200.0 24.4 5.8 4.6 6.1/8.5
jfs 218.0 43.2 5.8 2.7 12.0/12.0
jfs (noatime) 218.0 43.3 5.9 2.7 6.9/12.1
Aggregate Bandwidth (MB/s) - RAID 5 (4MB chunk) - 8192KB readahead buffer - No partition table
sequential read sequential write random read random write read-write
ext2 199.0 42.6 5.2 4.2 9.9/9.9

Hardware RAID

Aggregate Bandwidth (MB/s) - RAID 5 - No partition table
sequential read sequential write random read random write read-write
ext2 [1] 164.0 5.6 6.2 7.0/8.4
ext2 (noatime) 108.0 169.0 5.6 6.2 9.3/9.3
ext3 104.0 129.0 5.5 23.5 7.9/9.2
ext3 (noatime) 94.6 131.0 5.5 23.6 8.1/9.4
ext3 (data=journal) 107.0 79.2 5.5 9.7 8.0/5.4
ext3 (data=journal,noatime) 108.0 79.0 5.5 9.4 5.3/8.0
jfs 105.0 179.0 5.3 4.6 10.0/8.6
jfs (noatime) 104.0 179.0 5.6 4.5 8.6/8.6
reiserfs 108.0 76.5 5.6 58.8 8.3/9/7
reiserfs (noatime) 109.0 75.3 5.6 59.0 9.7/8.5
Aggregate Bandwidth (MB/s) - RAID 5 (256KB stripe) - No partition table
sequential read sequential write random read random write read-write
ext2 113.0 170.0 5.8 6.4 10.9/7.9
ext2 (noatime) 103.0 169.0 5.8 6.3 12.4/9.4
ext3 106.0 131.0 5.7 21.0 11.7/10.0
ext3 (noatime) 112.0 135.0 5.7 21.4 11.9/10.4
ext3 (data=journal) 112.0 74.8 5.7 8.0 11.7/10.3
ext3 (data=journal,noatime) 112.0 74.2 5.8 8.1 11.7/5.8
ext3 (data=writeback) 113.0 135.0 5.7 5.2 11.0/3.1
jfs 110.0 178.0 5.5 4.4 9.5/11.1
jfs (noatime) 110.0 172.0 5.8 4.5 9.8/9.5
reiserfs 114.0 92.6 5.8 58.3 10.8/12.3
reiserfs (noatime) 112.0 78.7 5.8 58.6 6.5/12.3
Aggregate Bandwidth (MB/s) - RAID 5 (256KB stripe) - 1 whole disk partition
sequential read sequential write random read random write read-write
ext2 113.0 171.0 5.8 6.3 7.8/10.9
ext2 (noatime) 108.0 169.0 5.8 6.3 7.8/10.9
ext3 109.0 132.0 5.7 21.0 10.5/9.0
ext3 (noatime) 112.0 135.0 5.7 20.9 10.5/9.0
ext3 (data=journal) 113.0 75.4 5.7 8.3 4.4/11.4
ext3 (data=journal,noatime) 113.0 75.4 5.8 8.3 8.9/11.7
jfs 110.0 172.0 5.8 4.5 8.0/11.1
jfs (noatime) 111.0 178.0 5.8 4.6 6.4/9.6
reiserfs 113.0 113.0 5.8 58.4 7.7/10.8
reiserfs (noatime) 114.0 98.0 5.8 59.0 10.8/6.3
Aggregate Bandwidth (MB/s) - RAID 5 (256KB stripe) - 256KB readahead
sequential read sequential write random read random write read-write
ext2 124.0 172.0 5.8 6.3 12.8/8.0
ext2 (noatime) 125.0 174.0 5.8 6.3 8.0/11.2
ext3 118.0 131.0 5.7 21.0 10.7/5.1
ext3 (noatime) 119.0 136.0 5.7 21.1 10.7/9.3
ext3 (data=journal) 116.0 73.8 5.7 8.4 10.5/4.6
ext3 (data=journal,noatime) 125.0 74.2 5.2 8.2 10.5/12.0
ext3 (data=writeback) 121.0 134.0 5.7 5.2 6.6/7.9
ext3 (data=writeback,noatime) 118.0 133.0 5.7 5.3 11.1/12.7
jfs 121.0 178.0 5.8 4.6 8.4/9.8
jfs (noatime) 120.0 178.0 5.8 4.5 9.9/9.8
reiserfs 125.0 93.3 5.8 58.4 9.5/12.7
reiserfs (noatime) 131.0 74.0 5.8 58.5 11.1/12.7
Aggregate Bandwidth (MB/s) - RAID 5 (256KB stripe) - 512KB readahead
sequential read sequential write random read random write read-write
ext2 156.0 167.0 5.8 6.3 6.6/8.3
ext2 (noatime) 141.0 171.0 5.8 6.4 11.6/9.9
ext3 154.0 133.0 5.7 21.2 6.3/11.0
ext3 (noatime) 155.0 132.0 5.2 20.7 9.5/12.6
ext3 (data=journal) 156.0 76.1 5.7 8.3 12.4/9.3
ext3 (data=journal,noatime) 155.0 74.8 5.8 8.4 12.4/9.4
ext3 (data=writeback) 163.0 132.0 5.7