C語言實現短字符串壓縮的三種方法詳解
前言
上一篇探索瞭LZ4的壓縮和解壓性能,以及對LZ4和ZSTD的壓縮、解壓性能進行瞭橫向對比。文末的最後也給瞭一個彩蛋:任意長度的字符串都可以被ZSTD、LZ4之類的壓縮算壓縮得很好嗎?
本篇我們就來一探究竟。
一、通用算法的短字符壓縮
開門見山,我們使用一段比較短的文本:Narrator: It is raining today. So, Peppa and George cannot play outside.Peppa: Daddy, it's stopped raining.
使用ZSTD與LZ4分別壓縮一下上面這段短文本。下面分別是它們的壓縮結果。
ZSTD:
LZ4:
對短文本的壓縮,zstd的壓縮率很低,lz4壓縮後的文本長度盡然超過瞭原有字符串的長度。這是為什麼?說實話在這之前我也沒想到。
引用兩位大佬的名言:
Are you ok?
What's your problem?
二、短字符串壓縮
從上面的結果可以得知,任何壓縮算法都有它的使用場景,並不是所有長度的字符串都適合被某種算法壓縮。一般原因是通用壓縮算法維護瞭被壓縮字符串的,用於字符串還原的相關數據結構,而這些數據結構的長度超過瞭被壓縮短字符串的自身長度。
那麼問題來瞭,“我真的有壓縮短字符串的需求,我想體驗壓縮的極致感,怎麼辦?”。
短字符壓縮算法它來瞭。這裡挑選瞭3種比較優異的短字符壓縮算法,分別是smaz,shoco,以及壓軸的unisox2。跟前兩章一樣,還是從壓縮率,壓縮和解壓縮性能的角度,一起看看他們在短字符壓縮場景的各自表現吧。
(1)Smaz
1、Smaz的壓縮和解壓縮
#include <stdio.h> #include <string.h> #include <iostream> #include "smaz.h" using namespace std; int main() { int buf_len; int com_size; int decom_size; char com_buf[4096] = {0}; char decom_buf[4096] = {0}; char str_buf[1024] = "Narrator: It is raining today. So, Peppa and George cannot play outside.Peppa: Daddy, it's stopped raining."; buf_len = strlen(str_buf); com_size = smaz_compress(str_buf, buf_len, com_buf, 4096); cout << "text size:" << buf_len << endl; cout << "compress text size:" << com_size << endl; cout << "compress ratio:" << (float)buf_len / (float)com_size << endl << endl; decom_size = smaz_decompress(com_buf, com_size, decom_buf, 4096); cout << "decompress text size:" << decom_size << endl; if(strncmp(str_buf, decom_buf, buf_len)) { cout << "decompress text is not equal to source text" << endl; } return 0; }
執行結果如下:
通過smaz壓縮後的短字符串長度為77,和源字符串相比,減少瞭30Byte。
2、Smaz的壓縮和解壓縮性能
#include <stdio.h> #include <string.h> #include <iostream> #include <sys/time.h> #include "smaz.h" using namespace std; int main() { int cnt = 0; int buf_len; int com_size; int decom_size; timeval st, et; char *com_ptr = NULL; char* decom_ptr = NULL; char str_buf[1024] = "Narrator: It is raining today. So, Peppa and George cannot play outside.Peppa: Daddy, it's stopped raining."; buf_len = strlen(str_buf); gettimeofday(&st, NULL); while(1) { com_ptr = (char *)malloc(buf_len); com_size = smaz_compress(str_buf, buf_len, com_ptr, buf_len); free(com_ptr); cnt++; gettimeofday(&et, NULL); if(et.tv_sec - st.tv_sec >= 10) { break; } } cout << endl <<"compress per second:" << cnt/10 << " times" << endl; cnt = 0; com_ptr = (char *)malloc(buf_len); com_size = smaz_compress(str_buf, buf_len, com_ptr, buf_len); gettimeofday(&st, NULL); while(1) { // decompress length not more than origin buf length decom_ptr = (char *)malloc(buf_len + 1); decom_size = smaz_decompress(com_ptr, com_size, decom_ptr, buf_len + 1); // check decompress length if(buf_len != decom_size) { cout << "decom error" << endl; } free(decom_ptr); cnt++; gettimeofday(&et, NULL); if(et.tv_sec - st.tv_sec >= 10) { break; } } cout << "decompress per second:" << cnt/10 << " times" << endl << endl; free(com_ptr); return 0; }
結果如何?
壓縮性能在40w條/S,解壓在百萬級,好像還不錯哈!
(2)Shoco
1、Shoco的壓縮和解壓縮
#include <stdio.h> #include <string.h> #include <iostream> #include "shoco.h" using namespace std; int main() { int buf_len; int com_size; int decom_size; char com_buf[4096] = {0}; char decom_buf[4096] = {0}; char str_buf[1024] = "Narrator: It is raining today. So, Peppa and George cannot play outside.Peppa: Daddy, it's stopped raining."; buf_len = strlen(str_buf); com_size = shoco_compress(str_buf, buf_len, com_buf, 4096); cout << "text size:" << buf_len << endl; cout << "compress text size:" << com_size << endl; cout << "compress ratio:" << (float)buf_len / (float)com_size << endl << endl; decom_size = shoco_decompress(com_buf, com_size, decom_buf, 4096); cout << "decompress text size:" << decom_size << endl; if(strncmp(str_buf, decom_buf, buf_len)) { cout << "decompress text is not equal to source text" << endl; } return 0; }
執行結果如下:
通過shoco壓縮後的短字符串長度為86,和源字符串相比,減少瞭21Byte。壓縮率比smaz要低。
2、Shoco的壓縮和解壓縮性能
#include <stdio.h> #include <string.h> #include <iostream> #include <sys/time.h> #include "shoco.h" using namespace std; int main() { int cnt = 0; int buf_len; int com_size; int decom_size; timeval st, et; char *com_ptr = NULL; char* decom_ptr = NULL; char str_buf[1024] = "Narrator: It is raining today. So, Peppa and George cannot play outside.Peppa: Daddy, it's stopped raining."; buf_len = strlen(str_buf); gettimeofday(&st, NULL); while(1) { com_ptr = (char *)malloc(buf_len); com_size = shoco_compress(str_buf, buf_len, com_ptr, buf_len); free(com_ptr); cnt++; gettimeofday(&et, NULL); if(et.tv_sec - st.tv_sec >= 10) { break; } } cout << endl <<"compress per second:" << cnt/10 << " times" << endl; cnt = 0; com_ptr = (char *)malloc(buf_len); com_size = shoco_compress(str_buf, buf_len, com_ptr, buf_len); gettimeofday(&st, NULL); while(1) { // decompress length not more than origin buf length decom_ptr = (char *)malloc(buf_len + 1); decom_size = shoco_decompress(com_ptr, com_size, decom_ptr, buf_len + 1); // check decompress length if(buf_len != decom_size) { cout << "decom error" << endl; } free(decom_ptr); cnt++; gettimeofday(&et, NULL); if(et.tv_sec - st.tv_sec >= 10) { break; } } cout << "decompress per second:" << cnt/10 << " times" << endl << endl; free(com_ptr); return 0; }
執行結果如何呢?
holy shit!壓縮和解壓縮居然都達到瞭驚人的百萬級。就像算法作者們自己說的一樣:“在長字符串壓縮領域,shoco不想與通用壓縮算法競爭,我們的優勢是短字符的快速壓縮,雖然壓縮率很爛!”。這樣說,好像也沒毛病。
(3)Unisox2
我們再來看看unisox2呢。
1、Unisox2的壓縮和解壓縮
#include <stdio.h> #include <string.h> #include "unishox2.h" int main() { int buf_len; int com_size; int decom_size; char com_buf[4096] = {0}; char decom_buf[4096] = {0}; char str_buf[1024] = "Narrator: It is raining today. So, Peppa and George cannot play outside.Peppa: Daddy, it's stopped raining."; buf_len = strlen(str_buf); com_size = unishox2_compress_simple(str_buf, buf_len, com_buf); printf("text size:%d\n", buf_len); printf("compress text size:%d\n", com_size); printf("compress ratio:%f\n\n", (float)buf_len / (float)com_size); decom_size = unishox2_decompress_simple(com_buf, com_size, decom_buf); printf("decompress text size:%d\n", decom_size); if(strncmp(str_buf, decom_buf, buf_len)) { printf("decompress text is not equal to source text\n"); } return 0; }
結果如下:
通過Unisox2壓縮後的短字符串長度為67,和源字符串相比,減少瞭40Byte,相當於是打瞭6折啊!不錯不錯。
2、Unisox2的壓縮和解壓縮性能
Unisox2的壓縮能力目前來看是三者中最好的,如果他的壓縮和解壓性能也不錯的話,那就真的就比較完美瞭。再一起看看Unisox2的壓縮和解壓性能吧!
#include <stdio.h> #include <string.h> #include <malloc.h> #include <sys/time.h> #include "unishox2.h" int main() { int cnt = 0; int buf_len; int com_size; int decom_size; struct timeval st, et; char *com_ptr = NULL; char* decom_ptr = NULL; char str_buf[1024] = "Narrator: It is raining today. So, Peppa and George cannot play outside.Peppa: Daddy, it's stopped raining."; buf_len = strlen(str_buf); gettimeofday(&st, NULL); while(1) { com_ptr = (char *)malloc(buf_len); com_size = unishox2_compress_simple(str_buf, buf_len, com_ptr); free(com_ptr); cnt++; gettimeofday(&et, NULL); if(et.tv_sec - st.tv_sec >= 10) { break; } } printf("\ncompress per second:%d times\n", cnt/10); cnt = 0; com_ptr = (char *)malloc(buf_len); com_size = unishox2_compress_simple(str_buf, buf_len, com_ptr); gettimeofday(&st, NULL); while(1) { // decompress length not more than origin buf length decom_ptr = (char *)malloc(buf_len + 1); decom_size = unishox2_decompress_simple(com_ptr, com_size, decom_ptr); // check decompress length if(buf_len != decom_size) { printf("decom error\n"); } free(decom_ptr); cnt++; gettimeofday(&et, NULL); if(et.tv_sec - st.tv_sec >= 10) { break; } } printf("decompress per second:%d times\n\n", cnt/10); free(com_ptr); return 0; }
執行結果如下:
事與願違,Unisox2雖然有三個算法中最好的壓縮率,可是卻也擁有最差的壓縮和解壓性能。跟前兩章分析的不謀而合:有高壓縮率,就會損失自身的壓縮性能,兩者不可兼得。
三、總結
本篇分享瞭smaz,shoco,unisox2三種短字符串壓縮算法,分別探索瞭它們各自的壓縮率與壓縮和解壓縮性能,結果如下表所示。
表1
shoco的壓縮率最低,但是擁有最高的壓縮和解壓速率;smaz居中;unisox2擁有最高的壓縮率,可是它的壓縮和解壓性能最低。
結論與前兩章有關長字符串壓縮的分析不謀而合:擁有高壓縮率,就會損失自身的壓縮性能,兩者不可兼得。
實際使用還是看自身需求和環境吧。如果適當壓縮就好,那就可以選用shoco,畢竟性能高;想要節約更多的空間,那就選擇smaz或者unisox2。
到此這篇關於C語言實現短字符串壓縮的三種方法詳解的文章就介紹到這瞭,更多相關C語言短字符串壓縮內容請搜索WalkonNet以前的文章或繼續瀏覽下面的相關文章希望大傢以後多多支持WalkonNet!
推薦閱讀:
- C語言字符串壓縮之ZSTD算法詳解
- C++字符數組、字符數組指針和string類
- 一文詳解C++中動態內存管理
- 詳解C/C++內存管理
- C/C++自主分配出現double free or corruption問題解決