NS20 NFS Performance, Part Deux

As a follow-on to the previous post, I wanted to see what happened with various block sizes as compared to the default 32768 v3xfersize parameter on Celerra NFSv3.

 

Test 1: 35840 byte block sizes (35KB as per customer)

Test 2: 8192 byte block sizes (8KB)

Test 3: 32768 byte block sizes (32KB)

Test 4: 65536 byte block sizes (64KB)

Test 5: 75368 byte block sizes (73.6KB chosen at random)

 

Before at 35830 block sizes, which we chose to match the customers typical file sizes, we had the following data set:

 

Command: ./perf.pl 35840 100000 1 1 10

Running Parameters:

basedir = /mnt/nfs3/testing

dd blocksize = 35840

dd blocksperfile = 100000

total files = 1

files per directory = 1

loop iterations = 10

dd command: dd if=/dev/zero of=mkfile.dat bs=32768 count=100000 2>/dev/null

 

Block Size

Block Count

Capacity (MB)

Iteration

Total Seconds

Transfer Rate (MB/s)

Transfer Rate (Mb/s)

35840

100000

3417.969

1

53

64.48997642

515.9198113

35840

100000

3417.969

2

53

64.48997642

515.9198113

35840

100000

3417.969

3

53

64.48997642

515.9198113

35840

100000

3417.969

4

54

63.29571759

506.3657407

35840

100000

3417.969

5

53

64.48997642

515.9198113

35840

100000

3417.969

6

53

64.48997642

515.9198113

35840

100000

3417.969

7

54

63.29571759

506.3657407

35840

100000

3417.969

8

53

64.48997642

515.9198113

35840

100000

3417.969

9

53

64.48997642

515.9198113

35840

100000

3417.969

10

55

62.14488636

497.1590909

 

 

 

Average

53.4

64.01661565

512.1329252

 

Changing the block size to fit within the v3xfersize block size, we see that we have slightly faster transfer times:

 

Command: ./perf.pl 8192 400000 1 1 10

Running Parameters:

basedir = /mnt/nfs3/testing

dd blocksize = 8192

dd blocksperfile = 400000

total files = 1

files per directory = 1

loop iterations = 10

 

Block Size

Block Count

Capacity (MB)

Iteration

Total Seconds

Transfer Rate (MB/s)

Transfer Rate (Mb/s)

8192

400000

3125.000

1

48

65.10416667

520.8333333

8192

400000

3125.000

2

47

66.4893617

531.9148936

8192

400000

3125.000

3

48

65.10416667

520.8333333

8192

400000

3125.000

4

48

65.10416667

520.8333333

8192

400000

3125.000

5

47

66.4893617

531.9148936

8192

400000

3125.000

6

48

65.10416667

520.8333333

8192

400000

3125.000

7

47

66.4893617

531.9148936

8192

400000

3125.000

8

48

65.10416667

520.8333333

8192

400000

3125.000

9

47

66.4893617

531.9148936

8192

400000

3125.000

10

48

65.10416667

520.8333333

 

 

 

Average

47.6

65.65824468

525.2659574

 

Command: ./perf.pl 32768 100000 1 1 10

Running Parameters:

basedir = /mnt/nfs3/testing

dd blocksize = 32768

dd blocksperfile = 100000

total files = 1

files per directory = 1

loop iterations = 10

 

Block Size

Block Count

Capacity (MB)

Iteration

Total Seconds

Transfer Rate (MB/s)

Transfer Rate (Mb/s)

32768

100000

3125.000

1

48

65.10416667

520.8333333

32768

100000

3125.000

2

48

65.10416667

520.8333333

32768

100000

3125.000

3

48

65.10416667

520.8333333

32768

100000

3125.000

4

50

62.5

500

32768

100000

3125.000

5

48

65.10416667

520.8333333

32768

100000

3125.000

6

48

65.10416667

520.8333333

32768

100000

3125.000

7

48

65.10416667

520.8333333

32768

100000

3125.000

8

48

65.10416667

520.8333333

32768

100000

3125.000

9

48

65.10416667

520.8333333

32768

100000

3125.000

10

49

63.7755102

510.2040816

 

 

 

Average

48.3

64.71088435

517.6870748

 

Command: ./perf.pl 65536 50000 1 1 10

Running Parameters:

basedir = /mnt/nfs3/testing

dd blocksize = 65536

dd blocksperfile = 50000

total files = 1

files per directory = 1

loop iterations = 10

 

Block Size

Block Count

Capacity (MB)

Iteration

Total Seconds

Transfer Rate (MB/s)

Transfer Rate (Mb/s)

65536

50000

3125.000

1

47

66.4893617

531.9148936

65536

50000

3125.000

2

48

65.10416667

520.8333333

65536

50000

3125.000

3

48

65.10416667

520.8333333

65536

50000

3125.000

4

49

63.7755102

510.2040816

65536

50000

3125.000

5

48

65.10416667

520.8333333

65536

50000

3125.000

6

48

65.10416667

520.8333333

65536

50000

3125.000

7

47

66.4893617

531.9148936

65536

50000

3125.000

8

48

65.10416667

520.8333333

65536

50000

3125.000

9

47

66.4893617

531.9148936

65536

50000

3125.000

10

48

65.10416667

520.8333333

 

 

 

Average

47.8

65.38685953

523.0948762

 

And going back to another random sized I/O block:

 

Command: ./perf.pl 75368 50000 1 1 10

Running Parameters:

basedir = /mnt/nfs3/testing

dd blocksize = 75368

dd blocksperfile = 50000

total files = 1

files per directory = 1

loop iterations = 10

 

Block Size

Block Count

Capacity (MB)

Iteration

Total Seconds

Transfer Rate (MB/s)

Transfer Rate (Mb/s)

75368

50000

3593.826

1

56

64.17546953

513.4037563

75368

50000

3593.826

2

56

64.17546953

513.4037563

75368

50000

3593.826

3

58

61.96252231

495.7001785

75368

50000

3593.826

4

56

64.17546953

513.4037563

75368

50000

3593.826

5

55

65.34229625

522.73837

75368

50000

3593.826

6

55

65.34229625

522.73837

75368

50000

3593.826

7

55

65.34229625

522.73837

75368

50000

3593.826

8

57

63.0495841

504.3966728

75368

50000

3593.826

9

55

65.34229625

522.73837

75368

50000

3593.826

10

55

65.34229625

522.73837

 

 

 

Average

55.8

64.42499963

515.399997

 

Note that while sniffing the NS20 port for NFS traffic, all packets came across as 32768 bytes, regardless of the data within. I expected to see better performance with block sizes aligning with the v3xfersize parameter. The slight anomaly was at 32768, where the transfer rate dropped slight compared to the 8192 and 65536 block sizes.

 

All in all though, I got fairly consistent transfer rates between the various test case scenarios.

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s