File 0002-journal-compress-improve-xz-compression-pe... of Package systemd

Overview Repositories Revisions Requests Users Attributes Meta

File 0002-journal-compress-improve-xz-compression-performance.patch of Package systemd (Revision 3fbdfd6f1d98f519f9689b679d6fe622)

Currently displaying revision 3fbdfd6f1d98f519f9689b679d6fe622 , Show latest

Based on 1930eed2a7855d2df06ccf51f9e394428bf547e2 Mon Sep 17 00:00:00 2001
From: Jon Severinsson <jon@severinsson.net>
Date: Tue, 8 Jul 2014 18:29:46 +0200
Subject: [PATCH] journal/compress: improve xz compression performance

The new lzma2 compression options at the top of compress_blob_xz are
equivalent to using preset "0", exept for using a 1 MiB dictionary
(the same as preset "1"). This makes the memory usage at most 7.5 MiB
in the compressor, and 1 MiB in the decompressor, instead of the
previous 92 MiB in the compressor and 8 MiB in the decompressor.

According to test-compress-benchmark this commit makes XZ compression
20 times faster, with no increase in compressed data size.
Using more realistic test data (an ELF binary rather than repeating
ASCII letters 'a' through 'z' in order) it only provides a factor 10
speedup, and at a cost if a 10% increase in compressed data size.
But that is still a worthwhile trade-off.

According to test-compress-benchmark XZ compression is still 25 times
slower than LZ4, but the compressed data is one eighth the size.
Using more realistic test data XZ compression is only 18 times slower
than LZ4, and the compressed data is only one quarter the size.

---
 src/journal/compress.c |   33 ++++++++++++++-------------------
 1 file changed, 14 insertions(+), 19 deletions(-)

--- src/journal/compress.c
+++ src/journal/compress.c	2014-07-09 12:09:45.814235274 +0000
@@ -28,8 +28,15 @@
 #include "compress.h"
 
 bool compress_blob(const void *src, uint64_t src_size, void *dst, uint64_t *dst_size) {
-        lzma_stream s = LZMA_STREAM_INIT;
+        static const lzma_options_lzma opt = {
+                1u << 20u, NULL, 0, LZMA_LC_DEFAULT, LZMA_LP_DEFAULT,
+                LZMA_PB_DEFAULT, LZMA_MODE_FAST, 128, LZMA_MF_HC3, 4};
+        static const lzma_filter filters[2] = {
+                {LZMA_FILTER_LZMA2, (lzma_options_lzma*) &opt},
+                {LZMA_VLI_UNKNOWN, NULL}
+        };
         lzma_ret ret;
+        size_t out_pos = 0;
         bool b = false;
 
         assert(src);
@@ -40,29 +47,17 @@ bool compress_blob(const void *src, uint
         /* Returns false if we couldn't compress the data or the
          * compressed result is longer than the original */
 
-        ret = lzma_easy_encoder(&s, LZMA_PRESET_DEFAULT, LZMA_CHECK_NONE);
-        if (ret != LZMA_OK)
+        if (src_size < 80)
                 return false;
 
-        s.next_in = src;
-        s.avail_in = src_size;
-        s.next_out = dst;
-        s.avail_out = src_size;
-
-        /* Does it fit? */
-        if (lzma_code(&s, LZMA_FINISH) != LZMA_STREAM_END)
-                goto fail;
-
-        /* Is it actually shorter? */
-        if (s.avail_out == 0)
-                goto fail;
+        ret = lzma_stream_buffer_encode((lzma_filter*) filters, LZMA_CHECK_NONE, NULL,
+                                        src, src_size, dst, &out_pos, src_size - 1);
+        if (ret != LZMA_OK)
+                return false;
 
-        *dst_size = src_size - s.avail_out;
+        *dst_size = out_pos;
         b = true;
 
-fail:
-        lzma_end(&s);
-
         return b;
 }

Places

File 0002-journal-compress-improve-xz-compression-performance.patch of Package systemd (Revision 3fbdfd6f1d98f519f9689b679d6fe622)

Places