Recently we saw that the journal device location, unfortunately, didn't make much of a difference on ext4 metadata performance. But can the size of the journal will have an impact on metadata performance? The first in a series of articles examining the journal size and performance.
Appendix – Table of Results
The first combination tested was for small files (4 KiB) with a shallow directory structure. Table 1 below lists the results with an average value and just below it, in red, is the standard deviation.
Table 1 – Benchmark Times Small Files (4 KiB) – Shallow Directory Structure
| Journal Size |
Directory Create
(secs.) |
File Create
(secs.) |
File Remove
(secs.) |
Directory Remove
(secs.) |
| 16MB |
28.40
1.28 |
346.70
2.53 |
70.90
0.94 |
6.80
3.89 |
| 64MB |
13.70
1.10 |
338.40
5.22 |
65.80
2.40 |
4.10
1.76 |
| 256MB |
10.0
0.45 |
330.80
0.98 |
53.80
1.17 |
15.70
0.64 |
| 1GB |
10.20
1.33 |
330.70
1.27 |
53.40
0.92 |
2.70
2.83 |
Table 2 below lists the performance results with an average value and just below it, in red, is the standard deviation.
Table 2 – Performance Results of Small Files (4 KiB) – Shallow Directory Structure
| Journal Size |
Directory Create
(Dirs/sec) |
File Create
(Files/sec) |
File Create
(KiB/sec) |
File Remove
(Files/sec) |
Directory Remove
(Dirs/sec) |
| 16MB |
296.50
13.30 |
971.10
7.06 |
3,885.90
28.73 |
4,751.50
63.66 |
1,529.50
513.69 |
| 64MB |
617.90
47.78 |
995.70
15.32 |
3,982.10
61.59 |
4,665.50
1,399.38 |
2,554.00
1,200.46 |
| 256MB |
843.60
38.15 |
1,017.60
2.94 |
4,072.50
11.94 |
6,263.40
134.59 |
6,315.60
2,681.55 |
| 1GB |
837.90
99.13 |
1,017.90
3.81 |
4,073.70
15.44 |
6,309.10
108.52 |
5,698.00
2,928.20 |
The next test was for small files (4 KiB) with a deep directory structure. Table 3 below lists the benchmark times with an average value and just below it, in red, is the standard deviation.
Table 3 – Benchmark Times Small Files (4 KiB) – Deep Directory Structure
| Journal Size |
Directory Create
(secs.) |
File Create
(secs.) |
File Remove
(secs.) |
Directory Remove
(secs.) |
| 16MB |
324.50
3.17 |
633.30
7.09 |
330.60
2.15 |
214.40
1.36 |
| 64MB |
218.60
7.31 |
500.50
8.86 |
224.70
5.92 |
75.60
4.27 |
| 256MB |
148.10
3.45 |
426.00
15.49 |
169.80
8.94 |
54.20
3.19 |
| 1GB |
127.60
11.09 |
406.20
11.68 |
147.40
13.17 |
47.80
10.03 |
Table 4 below lists the performance results with an average value and just below it, in red, is the standard deviation.
Table 4 – Performance Results of Small Files (4 KiB) – Deep Directory Structure
| Journal Size |
Directory Create
(Dirs/sec) |
File Create
(Files/sec) |
File Create
(KiB/sec) |
File Remove
(Files/sec) |
Directory Remove
(Dirs/sec) |
| 16MB |
272.50
2.80 |
559.00
6.36 |
2,237.70
25.28 |
1,071.30
6.96 |
412.40
2.42 |
| 64MB |
405.00
13.54 |
707.50
12.70 |
2,831.80
50.11 |
1,577.20
41.77 |
1,175.10
67.83 |
| 256MB |
597.90
14.59 |
832.10
30.30 |
3,330.60
121.08 |
2,091.90
110.24 |
1,639.70
102.91 |
| 1GB |
698.90
57.52 |
872.40
23.33 |
3,491.10
93.44 |
2,420.10
191.32 |
1,924.00
345.24 |
The third combination tested was for medium files (4 MiB) with a shallow directory structure. Table 5 below lists the benchmark times with an average value and just below it, in red, is the standard deviation.
Table 5 – Benchmark Times Medium Files (4 MiB) – Shallow Directory Structure
| Journal Size |
Directory Create
(secs.) |
File Create
(secs.) |
File Remove
(secs.) |
Directory Remove
(secs.) |
| 16MB |
0.20
0.40 |
154.40
3.41 |
12.20
3.49 |
0.10
0.30 |
| 64MB |
0.30
0.46 |
154.10
3.27 |
11.40
2.69 |
0.00
0.00 |
| 256MB |
0.50
0.67 |
156.80
2.09 |
9.40
3.01 |
0.10
0.30 |
| 1GB |
0.30
0.46 |
154.20
1.54 |
11.40
3.56 |
0.20
0.40 |
Table 6 below lists the performance results with an average value and just below it, in red, is the standard deviation.
Table 6 – Performance Results of Medium Files (4 MiB) – Shallow Directory Structure
| Journal Size |
Directory Create
(Dirs/sec) |
File Create
(Files/sec) |
File Create
(KiB/sec) |
File Remove
(Files/sec) |
Directory Remove
(Dirs/sec) |
| 16MB |
61.40
122.80 |
19.40
0.66 |
79,570.40
1,682.43 |
271.60
72.43 |
30.70
92.10 |
| 64MB |
92.10
140.69 |
19.50
0.67 |
79,722.90
1,644.04 |
282.50
58.21 |
0.00
0.00 |
| 256MB |
107.30
138.10 |
19.10
0.30 |
78,329.70
1,050.68 |
386.60
219.08 |
.00
0.00 |
| 1GB |
92.10
140.69 |
19.30
0.46 |
79,644.20
784.75 |
305.00
123.77 |
61.40
122.80 |
The fourth and final combination tested was for medium files (4 MiB) with a deep directory structure. Table 7 below lists the benchmark times with an average value and just below it, in red, is the standard deviation.
Table 7 – Benchmark Times Medium Files (4 MiB) – Deep Directory Structure
| Journal Size |
Directory Create
(secs.) |
File Create
(secs.) |
File Remove
(secs.) |
Directory Remove
(secs.) |
| 16MB |
4.20
0.60 |
225.90
1.58 |
15.30
2.69 |
1.50
0.50 |
| 64MB |
3.00
0.00 |
220.30
1.55 |
14.10
3.51 |
1.80
1.25 |
| 256MB |
2.80
1.47 |
220.90
0.94 |
14.20
3.71 |
3.30
3.16 |
| 1GB |
2.90
0.83 |
220.40
1.28 |
14.30
3.00 |
2.50
1.20 |
Table 8 below lists the performance results with an average value and just below it, in red, is the standard deviation.
Table 8 – Results of Medium Files (4 MiB) – Deep Directory Structure
| Journal Size |
Directory Create
(Dirs/sec) |
File Create
(Files/sec) |
File Create
(KiB/sec) |
File Remove
(Files/sec) |
Directory Remove
(Dirs/sec) |
| 16MB |
497.50
76.57 |
17.80
0.40 |
72,495.30
507.54 |
225.90
1.58 |
1,535.00
512.00 |
| 64MB |
682.10
0.30 |
18.00
0.00 |
74,338.20
523.04 |
311.40
90.36 |
1,541.90
639.70 |
| 256MB |
847.60
239.91 |
18.20
0.40 |
74,133.90
317.48 |
309.60
85.20 |
1,210.80
697.18 |
| 1GB |
767.10
219.19 |
18.10
0.30 |
74,303.30
430.36 |
300.10
70.44 |
1,108.40
639.67 |
Jeff Layton is an Enterprise Technologist for HPC at Dell. He can be found lounging around at a nearby Frys enjoying the coffee and waiting for sales (but never during working hours).
Comments on "Size Can Matter: Improving Metadata Performance with Ext4 Journal Sizing – Part I"
I dont get the 256MB recommendation. Am I reading the results wrong? It seems that there is no negative to going 1GB, other than the 744MB extra storage. These days, with 1TB going for < $100, looks like its worth the space.
Well it\’s not really a recommendation but if someone held a gun to my head and said, \”pick a journal size,\” I would pick 256MB. The simple reason is that I don\’t want to waste the space. But as you point out it doesn\’t hurt at all to go to 1GB for the configurations tested.
I guess I\’m old enough where I compress data that isn\’t being actively used and I try to reduce the number of variables in my code to save space, etc. Really what I\’m saying that is I\’m old and I remember when we did everything we could to save space. Unfortunately, old habits die hard :)
Jeff