Starting from the previous two papers and thanks to Jens Axboe advices I decided to start several test trying to tune the cfq scheduler.
The database has been recreated as well as the TPCC schema. For this reason I performed a couple of benchmarch with the deadline scheduler as a baseline for the tests on cfq. These have been made changing one (or two) parameter at one time and the comparing the results in the usual table.
Important note: The db block size has been shrinked from 8K to 4K since I wished to compare the performance of a transactional workload having my DB block size identical to the filesystem blocksize.
The cfq default values are (the yellow parameters are common to all the schedulers):
| slice_idle | 8 | How long a sync slice is allowed to idle |
| slice_async_rq | 2 | How many requests an async disk slice lasts |
| slice_async | 40 | How many msec an async disk slice lasts |
| slice_sync | 100 | How many msec a sync disk slice lasts |
| back_seek_penalty | 2 | penalty of a backwards seek |
| back_seek_max | 16384 | maximum backwards seek |
| fifo_expire_async | 248 | fifo timeout for async requests |
| fifo_expire_sync | 124 | fifo timeout for sync requests |
| queued | 8 | minimum request allocate limit per-queue |
| quantum | 4 | max number of requests moved in one round of service |
| max_sectors_kb | 64 | max transfer size allowed for this queue |
| max_hw_sectors_kb | 64 | Max transfer size that the driver and/or device can handle |
| read_ahead_kb | 512 | Size of read-ahead window |
| nr_requests | 128 | Maximum number of requests allowed in a queue |
The tested configuration:
0. The baseline test with the deadline cheduler (and its default parameters).
a. datafile with cfq and
b. The above configuration plus:
c. configuration b with slice_idle at default:
d. configuration a and b plus some more tuning on the queues:
echo 64 > /sys/block/sdb/queue/iosched/max_depth
echo 32 > /sys/block/sdb/queue/iosched/queued
echo 64 > /sys/block/sdb/queue/iosched/quantum
e. redolog and datafiles with cfq tuned and queues at default:
echo 64 > /sys/block/sd[b,c]/queue/iosched/max_depth
echo 8 > /sys/block/sd[b,c]/queue/iosched/queued
echo 4 > /sys/block/sd[b,c]/queue/iosched/quantum
(queued and quantum are the default values).
f. previous configuration with queues increased:
echo 64 > /sys/block/sd[b,c]/queue/iosched/max_depth
echo 32 > /sys/block/sd[b,c]/queue/iosched/queued
echo 64 > /sys/block/sd[b,c]/queue/iosched/quantum
Results:
For any scheduler you can see an AWR report following the below links:
| transaction per second | log file sync % | user calls | physical reads | physical writes | |
| test 0 | 53.34 | .7 | 248.87 | 310.22 | 109.15 |
| test a | 37.41 | .6 | 142.70 | 223.40 | 71.15 |
| test b | 52.60 | .8 | 246.77 | 312.64 | 106.40 |
| test c | 31.51 | .3 | 147.38 | 187.46 | 64.13 |
| test d | 52.26 | .3 | 242.08 | 313.31 | 108.89 |
| test e | 53.24 | .8 | 246.53 | 313.00 | 105.62 |
| test f | 52.05 | .8 | 242.55 | 313.96 | 106.04 |
The cfq scheduler performs a lot better in any test with the specified parameters modified from their default values.
However it seems the redologs gives better results if placed on a device with the deadline scheduler activated.
Note: when the redologs are on cfq the "log file sync" is lower than when placed on deadline. The overall performance, instead, shows better reults in the first cases (with deadline).
Looking at the tpm graph during the a and b tests shows that the transaction per minute in b is more constant (the values of the table are the mean values).
The same can be seen in c and d.
After the above test I decided to reset the whole system regenerating the TPC-C schema and rerunning all the tests I already performed to be sure the results are coherent plus several new trying to further tune cfq and deadline schedulers.
Contact information:
fabrizio.magni _at_ gmail.com
- Log in to post comments