1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
|
---
layout: post
title: 如何制作一个“完整”的博客压缩包?
tags: [压缩包, Quine, LZMA2, 7-Zip]
---
让AI做出真正的“完整”将不再是难事……<!--more-->
# 起因
在上次[用AI制作了各种东西](/2026/09/01/vibe-coding2.html)之后,我已经完全理解了AI确实是无所不能的。既然如此,那就让它帮我解决曾经未能解决的事情吧?
去年,我为了让下载全站压缩包的按钮不断链,让这个压缩包也包含它本身而研究了[ZIP Quine](/2025/09/01/quine.html),但限于DEFLATE的回溯窗口大小没能做到……但那是人做的东西,人还是太弱小了,现在换AI来试试,也许一切将变得不一样?
# 制作基于LZMA2的博客压缩包
首先,我把[Ruben Van Mello](https://github.com/ruvmello)写的那篇论文《[A Generator for Recursive Zip Files](https://www.mdpi.com/2076-3417/14/21/9797)》以及生成器[zip-quine-generator](https://github.com/ruvmello/zip-quine-generator)发给了AI,问它这个限制是不是真的,是不是真的无法创造出超过32KiB的ZIP Quine?没过多久它分析完了,告诉我确实存在这样的问题,但并不是无法解决的,最简单的办法就是换个回溯窗口更大的压缩算法。我看了一下它给我列出的几个算法,看起来LZMA2和zstd比较符合要求。不过zstd感觉不是很知名,一般的解压软件应该处理不了,所以我就选择让它基于LZMA2来制作了。
不过LZMA2只是算法,还得选个容器,考虑到7z算是最出名,而且LZMA2本来就是7-Zip的作者开发的,所以就直接让它做7z版本的了。另外因为[Nate Choe](https://github.com/NateChoe1)做了通过扩展欧几里得求逆元来计算CRC32的[PR](https://github.com/ruvmello/zip-quine-generator/pull/3),所以我也告诉它要用这种方法在多项式时间内解出CRC32的值,而不是爆破。就这样AI花了两个小时左右成功把代码写出来了,效果非常完美,另外它还偷懒地将计算CRC32的算法改成了高斯消元法😆,因为7z格式有3个相互影响的CRC32值,用扩展欧几里得会更复杂一点,不过对7z来说不写入文件的CRC32也不影响,只是会不校验罢了,至少它还是按我要求做了……但其实这个问题也并不是完全不能用扩展欧几里得算法,后来我又强烈要求了一下,AI还是给我写出来了。以下是代码片段,有兴趣的人可以参考一下(不过这个看起来写的就是很复杂,所以我自己只用了高斯消元法):
```python
RU_POLY = 0x104C11DB7 # x^32+x^26+x^23+...+1(含 x^32 首项)
def _find_probe(p):
ret = 0
for i in range(64):
if (1 << i) & p:
ret = i
return ret
def _mul_raw(p1, p2):
"""多项式乘法(无模)。"""
ret = 0
probe = _find_probe(p1)
for i in range(64):
if p2 & (1 << i):
assert probe + i < 64, "多项式乘法溢出"
ret ^= p1 << i
return ret
def _poly_divmod(dividend, divisor):
"""多项式长除法,返回 (商, 余)。"""
probe = _find_probe(divisor)
probe_bit = 1 << probe
quot = 0
rem = dividend
for i in range(63 - probe, -1, -1):
if rem & (probe_bit << i):
quot |= 1 << i
rem ^= divisor << i
return quot, rem
def _mul(p1, p2, mod):
"""p1*p2 % mod。"""
return _poly_divmod(_mul_raw(p1, p2), mod)[1]
def _xgcd(p1, p2):
"""扩展欧几里得:返回 (k1,k2,gcd),满足 p1*k1+p2*k2=gcd。"""
if _find_probe(p1) < _find_probe(p2):
k1, k2, g = _xgcd(p2, p1)
return k2, k1, g
if p2 == 0:
return p1, 0, p1
q, r = _poly_divmod(p1, p2)
c1, c2, g = _xgcd(p2, r)
return c2, c1 ^ _mul_raw(c2, q), g
def _minv(p, mod):
"""p 在环上的乘法逆元;不存在时返回 0。"""
k1, _, g = _xgcd(p, mod)
if g != 1:
return 0
return _poly_divmod(k1, mod)[1]
def _bitrev32(x):
"""32 位比特反转。CRC 的输出循环冗余(binascii 反射)↔ 本环(非反射)互为 bit-reverse。"""
return int(format(x & 0xFFFFFFFF, "032b")[::-1], 2)
def _ring_pow_x(k):
"""环内 x^k mod P(P=RU_POLY),二进制幂 O(log k)。变量列就是单个环幂。"""
res, base, mod, e = 1, 2, RU_POLY, k
while e:
if e & 1:
res = _mul(res, base, mod)
base = _mul(base, base, mod)
e >>= 1
return res
def solve_crc_system(files):
"""解析解 CRC 定点系统
- 常数列 :对“变量清零”的数据做一次 binascii.crc32,再 bit-reverse 32 位
(binascii 反射 CRC 与本环非反射 CRC 互为 bit-reverse,见 _bitrev32)。
- 变量列 :单个环幂 x^(32 + 8*(len-pos-4)),二进制幂 O(log N) 一步算出;
同一变量出现在多处时按位异或累加。
各自 O(#rows) 次 CRC + 少量环幂即可建出整矩阵,Gauss 消元不变。
"""
n = len(files)
mod = RU_POLY
matrix = [[0] * (n + 1) for _ in range(n)]
for file in range(n):
data = files[file][0]
offsets = files[file][1]
N = len(data)
zero = bytearray(data)
for pos in offsets: # 未知量占 4 字节,清零后求常数项
zero[pos:pos + 4] = b"\x00\x00\x00\x00"
matrix[file][n] = _bitrev32(crc32(bytes(zero)))
for pos, vid in offsets.items():
rest = N - pos - 4 # 变量被 4 字节(x^32)+ 其后字节(x^8/个)推到底
matrix[file][vid] ^= _ring_pow_x(32 + 8 * rest)
matrix[file][file] ^= 1
for sr in range(n):
ex = sr
while ex < n and matrix[ex][sr] == 0:
ex += 1
if ex == n:
raise RuntimeError("CRC 多项式系统奇异 @row%d" % sr)
if ex != sr:
matrix[sr], matrix[ex] = matrix[ex], matrix[sr]
inv = _minv(matrix[sr][sr], mod)
for er in range(sr + 1, n):
mult = _mul(inv, matrix[er][sr], mod)
for ec in range(n + 1):
matrix[er][ec] ^= _mul(mult, matrix[sr][ec], mod)
res = [0] * n
for row in range(n - 1, -1, -1):
v = matrix[row][n]
for col in range(row + 1, n):
v ^= _mul(res[col], matrix[row][col], mod)
res[row] = _mul(v, _minv(matrix[row][row], mod), mod)
return [_impl_val(res[i]) for i in range(n)]
def _impl_val(math_val):
imp = 0
for bit in range(32):
if math_val & (1 << bit):
imp |= 1 << (31 - bit)
return imp
class BlogQuine:
def solve_crc(self, F, lay, seed):
T, h, d = lay["T"], lay["h"], lay["d"]
k, n_sub, crc_base = lay["k"], lay["n_sub"], lay["crc_base"]
hoff = lay["total"] - h
vout_f = lay["total"] - 2 * T
hcopy = vout_f + (T - h)
# 已知 CRC:seed + 各文件(quine 的是未知量 D)
known = [crc32(seed)] + [crc32(data) for _, data in self.files]
assert len(known) == n_sub - 1
for i, e in enumerate(known):
struct.pack_into("<I", F, hoff + crc_base + 4 * i, e)
struct.pack_into("<I", F, hcopy + crc_base + 4 * i, e)
# 未知量 D N S:D=整个文件、N=头部 F[hoff:hoff+h]、S=签名 F[12:32]
dpos = crc_base + 4 * (n_sub - 1)
ncopy = 32 + 3 * k + d # C1k 载荷里 F[0:35) 副本起点
groups = [[hoff + dpos, hcopy + dpos], # D 出现两处
[28, ncopy + 28], # N
[8, ncopy + 8]] # S
# 构造 3 个“文件”喂给解析解系统:
# 文件0 = 整个 F,三个未知量各自出现在 groups 两处
# 文件1 = 头部块,只有 D 出现在相对 dpos(另一处 hcopy+dpos 在头外)
# 文件2 = 签名块,只有 N 出现在相对 16(28-12)
whole = bytes(F)
hdr = bytes(F[hoff:hoff + h])
sig = bytes(F[12:32])
f0_off = {hoff + dpos: 0, hcopy + dpos: 0,
28: 1, ncopy + 28: 1,
8: 2, ncopy + 8: 2}
f1_off = {dpos: 0}
f2_off = {28 - 12: 1}
vals = solve_crc_system([(whole, f0_off), (hdr, f1_off), (sig, f2_off)])
for g in range(3):
for p in groups[g]:
F[p:p + 4] = struct.pack("<I", vals[g])
real = (crc32(F), crc32(F[hoff:hoff + h]), crc32(F[12:32]))
assert real == tuple(vals), "CRC 定点失败: %s != %s" % (real, vals)
return vals
```
于是我第一时间就把原来的TGZ压缩命令换掉,换成了AI给我写的[blogquine.py](https://github.com/Mabbs/mabbs.github.io/blob/master/_tools/blogquine.py),现在就可以通过[这里](https://mayx.eu.org/MayxBlog.7z)下载到“完整”包含我博客所有内容的压缩包了。
不过唯一的问题就是这样做出来的压缩包并没有压缩😂,相当于给做成了普通的归档了。当然我的博客本身倒是不大,没压缩也多不了多少空间,但相比于能做出“完整”的效果来说,这点浪费的空间也是小问题了。
## 对TXZ格式的尝试
在做完7z格式的压缩包之后,我发现了一个问题,虽然7z确实很流行,但是在Linux下解压起来有点麻烦,7-Zip历史上主要面向Windows,Linux上长期以来更多依赖p7zip等第三方移植,因此生态集成度不如tar.xz,想要解压7z文件还得额外安装。
不过Linux下也有个支持LZMA2算法的压缩软件,那就是前些年出过[后门](https://tukaani.org/xz-backdoor/)的XZ Utils,配合tar就可以做出TXZ(tar.xz)文件,甚至用我[博客终端](https://mabbs.github.io/linux/)中的BusyBox也能解压。我想了一下反正有AI,干脆一句话让AI帮我把blogquine.py改成tar.xz格式的好了,结果倒也没费多少功夫,AI就这样写出来了:
<details markdown="1">
<summary markdown="span">
Show Code
</summary>
```python
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import argparse
import binascii
import io
import lzma
import os
import struct
import sys
import tarfile
import time
import ctypes
import ctypes.util
# ============================================================
# Part 1: 最小 LZMA1 range coder(只做编码)
# ============================================================
kNumBitModelTotalBits = 11
kBitModelTotal = 1 << kNumBitModelTotalBits # 2048
kNumMoveBits = 5
kTopValue = 1 << 24 # 0x1000000
kNumStates = 12
kNumLitStates = 7
kNumPosBitsMax = 4
kNumLenToPosStates = 4
kNumAlignBits = 4
kEndPosModelIndex = 14
kNumFullDistances = 1 << (kEndPosModelIndex >> 1) # 128
kMatchMinLen = 2
kNumLowLenBits = 3
kNumMidLenBits = 3
kNumHighLenBits = 8
kNumLowLenSymbols = 1 << kNumLowLenBits # 8
kNumMidLenSymbols = 1 << kNumMidLenBits # 8
kNumPosSlotBits = 6
PROB_INIT = kBitModelTotal >> 1 # 1024
class RangeEncoder:
"""LZMA 的区间编码器。low 是 64 位(要容纳进位),range 是 32 位。"""
def __init__(self):
self.low = 0
self.range = 0xFFFFFFFF
self.cache = 0
self.cache_size = 1 # 初值 1 -> 第一个输出字节恒为 0x00
self.buf = bytearray()
def _shift_low(self):
if (self.low & 0xFFFFFFFF) < 0xFF000000 or (self.low >> 32) != 0:
temp = self.cache
while True:
self.buf.append((temp + (self.low >> 32)) & 0xFF)
temp = 0xFF
self.cache_size -= 1
if self.cache_size == 0:
break
self.cache = (self.low >> 24) & 0xFF
self.cache_size += 1
self.low = ((self.low & 0xFFFFFFFF) << 8) & 0xFFFFFFFF
def encode_bit(self, probs, idx, bit):
p = probs[idx]
bound = (self.range >> kNumBitModelTotalBits) * p
if bit == 0:
self.range = bound
probs[idx] = p + ((kBitModelTotal - p) >> kNumMoveBits)
else:
self.low += bound
self.range -= bound
probs[idx] = p - (p >> kNumMoveBits)
while self.range < kTopValue:
self.range = (self.range << 8) & 0xFFFFFFFF
self._shift_low()
def encode_direct_bits(self, value, num_bits):
for i in range(num_bits - 1, -1, -1):
self.range >>= 1
if (value >> i) & 1:
self.low += self.range
while self.range < kTopValue:
self.range = (self.range << 8) & 0xFFFFFFFF
self._shift_low()
def bittree_encode(self, probs, off, num_bits, symbol):
m = 1
for i in range(num_bits - 1, -1, -1):
bit = (symbol >> i) & 1
self.encode_bit(probs, off + m, bit)
m = (m << 1) | bit
def bittree_reverse_encode(self, probs, off, num_bits, symbol):
m = 1
for i in range(num_bits):
bit = symbol & 1
symbol >>= 1
self.encode_bit(probs, off + m, bit)
m = (m << 1) | bit
def finish(self):
for _ in range(5):
self._shift_low()
return bytes(self.buf)
def _pos_slot_and_bits(d):
if d < 4:
return d, 0, 0
for slot in range(4, 64):
n = (slot >> 1) - 1
base = (2 | (slot & 1)) << n
if d < base + (1 << n):
return slot, n, d - base
raise ValueError("distance too large: %d" % d)
# pb 必须与写进每个 LZMA2 chunk 头的 PROPS_BYTE(0x5D) 一致。
# lc/lp 只影响 literal 编码的概率表索引,而本编码器不实现 literal 路径,故无需常量。
PB = 2
class LzmaEncoder:
"""只实现 quine 用到的两条路径:match 与 rep0-match。
literal / rep-g1 / rep-g2 三条分支从未被构造,因此它们对应的概率表
(p_lit、p_is_rep_g1、p_is_rep_g2)以及 prev_byte 都不必存在。
reps 同理:编码器从不回读(rep0 的距离由解码器自行维护),故不保存。
"""
def __init__(self):
self.pos_mask = (1 << PB) - 1
self.rc = RangeEncoder()
self.pos = 0
self.state = 0
n = PROB_INIT
self.p_is_match = [n] * (kNumStates << kNumPosBitsMax)
self.p_is_rep = [n] * kNumStates
self.p_is_rep_g0 = [n] * kNumStates
self.p_rep0_long = [n] * (kNumStates << kNumPosBitsMax)
self.p_pos_slot = [n] * (kNumLenToPosStates << kNumPosSlotBits)
self.p_spec_pos = [n] * (kNumFullDistances - kEndPosModelIndex)
self.p_align = [n] * (1 << kNumAlignBits)
self.p_len_choice = [n] * 2
self.p_len_low = [n] * (16 * kNumLowLenSymbols)
self.p_len_mid = [n] * (16 * kNumMidLenSymbols)
self.p_len_high = [n] * (1 << kNumHighLenBits)
self.p_rep_len_choice = [n] * 2
self.p_rep_len_low = [n] * (16 * kNumLowLenSymbols)
self.p_rep_len_mid = [n] * (16 * kNumMidLenSymbols)
self.p_rep_len_high = [n] * (1 << kNumHighLenBits)
@property
def pos_state(self):
return self.pos & self.pos_mask
def _encode_len(self, choice, low, mid, high, length):
ps = self.pos_state
l = length - kMatchMinLen
if l < kNumLowLenSymbols:
self.rc.encode_bit(choice, 0, 0)
self.rc.bittree_encode(low, ps << kNumLowLenBits, kNumLowLenBits, l)
else:
self.rc.encode_bit(choice, 0, 1)
l -= kNumLowLenSymbols
if l < kNumMidLenSymbols:
self.rc.encode_bit(choice, 1, 0)
self.rc.bittree_encode(mid, ps << kNumMidLenBits, kNumMidLenBits, l)
else:
self.rc.encode_bit(choice, 1, 1)
self.rc.bittree_encode(high, 0, kNumHighLenBits, l - kNumMidLenSymbols)
def _write_dist(self, dist, lts):
d = dist - 1
slot, n, low_bits = _pos_slot_and_bits(d)
self.rc.bittree_encode(self.p_pos_slot, lts << kNumPosSlotBits,
kNumPosSlotBits, slot)
if slot >= 4:
if slot < kEndPosModelIndex:
base = (2 | (slot & 1)) << n
off = base - slot - 1
self.rc.bittree_reverse_encode(self.p_spec_pos, off, n, low_bits)
else:
self.rc.encode_direct_bits(low_bits >> kNumAlignBits, n - kNumAlignBits)
self.rc.bittree_reverse_encode(self.p_align, 0, kNumAlignBits,
low_bits & ((1 << kNumAlignBits) - 1))
def match(self, dist, length):
assert kMatchMinLen <= length <= 273, length
ps = self.pos_state
self.rc.encode_bit(self.p_is_match, (self.state << kNumPosBitsMax) + ps, 1)
self.rc.encode_bit(self.p_is_rep, self.state, 0)
lts = min(length - kMatchMinLen, kNumLenToPosStates - 1)
self._encode_len(self.p_len_choice,
self.p_len_low, self.p_len_mid, self.p_len_high, length)
self._write_dist(dist, lts)
self.state = 7 if self.state < kNumLitStates else 10
self.pos += length
def rep_match(self, length):
assert kMatchMinLen <= length <= 273, length
ps = self.pos_state
self.rc.encode_bit(self.p_is_match, (self.state << kNumPosBitsMax) + ps, 1)
self.rc.encode_bit(self.p_is_rep, self.state, 1)
self.rc.encode_bit(self.p_is_rep_g0, self.state, 0)
self.rc.encode_bit(self.p_rep0_long, (self.state << kNumPosBitsMax) + ps, 1)
self._encode_len(self.p_rep_len_choice,
self.p_rep_len_low, self.p_rep_len_mid, self.p_rep_len_high, length)
self.state = 8 if self.state < kNumLitStates else 11
self.pos += length
def finish(self):
return self.rc.finish()
# ============================================================
# Part 2: tar + xz 格式原语
# ============================================================
MAX_MATCH = 273 # LZMA1 单个 match 长度上限
CHUNK = 65536 # LZMA2 单 chunk 解压上限
PROPS_BYTE = 0x5D # lc=3, lp=0, pb=2
SEED_NOTE = ("\nMayx's Blog!").encode("utf-8")
# xz 格式常量
XZ_MAGIC = b"\xFD7zXZ\x00"
XZ_FOOTER_MAGIC = b"YZ"
XZ_CHECK_CRC64 = 0x04
LZMA2_FILTER_ID = 0x21
CHECK_SIZE = 8 # CRC64 占 8 字节
_XZ_FLAGS = bytes([0x00, XZ_CHECK_CRC64]) # Stream Flags: reserved + check_type
# tar 格式常量
TAR_BLOCK = 512
def round_up_512(n):
"""长度按 tar 块 512 字节向上取整。"""
return (n + TAR_BLOCK - 1) // TAR_BLOCK * TAR_BLOCK
def crc32(b):
return binascii.crc32(b) & 0xFFFFFFFF
# ---------- CRC64 (ECMA-182, reflected) ----------
_CRC64_POLY = 0xC96C5795D7870F42 # reflected ECMA-182
_CRC64_INIT = 0xFFFFFFFFFFFFFFFF
_CRC64_XOROUT = 0xFFFFFFFFFFFFFFFF
_CRC64_NON_REFLECTED = 0x142F0E1EBA9EA3693 # 非反射多项式 (65 bit)
_CRC64_MOD_BIT = 1 << 64 # x^64 对应的 bit
_liblzma = None
try:
_lzma_name = ctypes.util.find_library('lzma')
if _lzma_name:
_liblzma = ctypes.CDLL(_lzma_name)
_liblzma.lzma_crc64.argtypes = [ctypes.c_char_p, ctypes.c_size_t, ctypes.c_uint64]
_liblzma.lzma_crc64.restype = ctypes.c_uint64
# 验证:CRC64("123456789") 应为 0x995dc9bbdf1939fa
if _liblzma.lzma_crc64(b'123456789', 9, 0) != 0x995dc9bbdf1939fa:
_liblzma = None
except (OSError, AttributeError):
_liblzma = None
def _make_crc64_table():
table = []
for i in range(256):
crc = i
for _ in range(8):
crc = (crc >> 1) ^ _CRC64_POLY if (crc & 1) else crc >> 1
table.append(crc)
return table
_CRC64_TABLE = _make_crc64_table()
def _crc64_pure(data):
"""纯 Python CRC64(ECMA-182 反射,init/xorout = 0xFFFF...FFFF)。"""
crc = _CRC64_INIT
for byte in data:
crc = (crc >> 8) ^ _CRC64_TABLE[(crc ^ byte) & 0xFF]
return crc ^ _CRC64_XOROUT
def _crc64_lib(data):
"""liblzma CRC64(init=0,内部处理 init/xorout)。"""
return _liblzma.lzma_crc64(data, len(data), 0)
crc64 = _crc64_lib if _liblzma else _crc64_pure
# ---------- GF(2^64) 多项式运算 ----------
# 用于 CRC64 自引用求解。CRC64 的线性贡献可表示为
# bit_reverse(contribution(D)) = D_poly * P (mod G_non)
# 其中 D_poly = bit_reverse(D),P 是位置决定的多项式。
# 方程 D_poly * (1 ^ P1 ^ P2) = bit_reverse(CRC64(W)) 通过多项式逆元求解。
def _bit_reverse64(v):
r = 0
for _ in range(64):
r = (r << 1) | (v & 1)
v >>= 1
return r
def _poly_mul_mod(a, b):
"""GF(2) 多项式乘法 mod G(非反射)。"""
result = 0
while b:
if b & 1:
result ^= a
b >>= 1
a <<= 1
if a & _CRC64_MOD_BIT:
a ^= _CRC64_NON_REFLECTED
return result
def _poly_pow(base, exp):
"""多项式幂:base^exp mod G。"""
result = 1
while exp:
if exp & 1:
result = _poly_mul_mod(result, base)
base = _poly_mul_mod(base, base)
exp >>= 1
return result
def _poly_minv(p):
"""多项式逆元 p^(-1) mod G,用扩展欧几里得算法。"""
if p == 0:
return 0
old_r, r = p, _CRC64_NON_REFLECTED
old_s, s = 1, 0
while r:
# poly divmod(old_r, r) -> (q, rem)
probe_r = r.bit_length() - 1
rem = old_r
q = 0
while rem.bit_length() - 1 >= probe_r and rem:
shift = rem.bit_length() - 1 - probe_r
q |= 1 << shift
rem ^= r << shift
old_r, r = r, rem
# s = old_s ^ q*s (raw 多项式乘法)
prod = 0
a, b = q, s
while b:
if b & 1:
prod ^= a
b >>= 1
a <<= 1
old_s, s = s, old_s ^ prod
if old_r != 1:
return 0
# old_s mod G
probe_m = _CRC64_NON_REFLECTED.bit_length() - 1
rem = old_s
while rem.bit_length() - 1 >= probe_m and rem:
shift = rem.bit_length() - 1 - probe_m
rem ^= _CRC64_NON_REFLECTED << shift
return rem
def store_hdr(payload_len, first=False):
"""LZMA2 uncompressed chunk 头(3 字节,大端 size-1)。"""
assert 1 <= payload_len <= 65536
return bytes([0x01 if first else 0x02]) + (payload_len - 1).to_bytes(2, "big")
def matches_for(dist, total):
"""dist 固定、总长 total 的 token 序列:首个 match + 后续 rep0。"""
assert total >= 2
toks = []
first = min(MAX_MATCH, total)
if total - first == 1:
first -= 1
toks.append(("m", dist, first))
total -= first
while total > 0:
l = min(MAX_MATCH, total)
if total - l == 1:
l -= 1
toks.append(("r0", l))
total -= l
return toks
def _add_gear(chunks, o, f, q):
"""追加一组标准 gear:store 载荷 x = slip+3,紧跟一个 lzma 把那 x 字节复制一遍。
lzma chunk 解压出的字节数正好等于 x,所以齿轮转完之后 slip (f - (o-q)) 精确
归约为 len(cb)(很小的正数,x 被 CHUNK 截断时则按 CHUNK 缩减)。
_layout_iterate 反复套用把 slip 压到 <= 16,_trim_layouts 用它做相位微调。
返回追加后的 (o, f)。
"""
x = min(f - (o - q) + 3, CHUNK)
chunks.append({"kind": "store", "foff": f, "size": 3 + x,
"ooff": o, "olen": x})
o += x
f += 3 + x
cb, cu = lzma_chunk(matches_for(x, x), o)
chunks.append({"kind": "lzma", "foff": f, "size": len(cb),
"ooff": o, "olen": cu, "bytes": cb})
return o + cu, f + len(cb)
def _forward_vac(F, vac, q):
"""vac_store:把紧随其后的等长字节块前移覆盖自身。"""
fo = vac["ooff"] - q
olen = vac["olen"]
F[fo:fo + olen] = F[fo + olen:fo + 2 * olen]
def dict_prop_for(maxdist):
"""选最小的 LZMA2 dict prop 使字典 >= maxdist。"""
for p in range(41):
if (2 | (p & 1)) << (p // 2 + 11) >= maxdist:
return p, (2 | (p & 1)) << (p // 2 + 11)
raise ValueError("distance too large")
def lzma_chunk(tokens, out_pos):
"""编一个 LZMA chunk(0xC0: state+props reset,无 dict reset,无 end marker)。"""
enc = LzmaEncoder()
enc.pos = out_pos
total = 0
for t in tokens:
if t[0] == "m":
enc.match(t[1], t[2])
total += t[2]
else:
enc.rep_match(t[1])
total += t[1]
data = enc.finish()
assert total - 1 < 65536 and len(data) - 1 < 65536
hdr = bytes([0xC0]) + (total - 1).to_bytes(2, "big") + \
(len(data) - 1).to_bytes(2, "big") + bytes([PROPS_BYTE])
return hdr + data, total
# ---------- tar 原语 ----------
def tar_header(name, size, mtime, mode=0o644, typeflag=b'0', uid=0, gid=0):
"""构建 512 字节 POSIX ustar tar header。
注意:tar 数值字段包含 null 终止符,slice 范围必须精确匹配字段宽度。
mode/uid/gid 各 8 字节(7 octal + NUL),size/mtime 各 12 字节(11 octal + NUL)。
"""
h = bytearray(512)
name_bytes = name.encode('utf-8')[:100]
h[0:len(name_bytes)] = name_bytes
h[100:108] = b'%07o\x00' % mode # 8 bytes (field 100-107)
h[108:116] = b'%07o\x00' % uid # 8 bytes (field 108-115)
h[116:124] = b'%07o\x00' % gid # 8 bytes (field 116-123)
h[124:136] = b'%011o\x00' % size # 12 bytes (field 124-135)
h[136:148] = b'%011o\x00' % mtime # 12 bytes (field 136-147)
h[148:156] = b' ' # checksum placeholder (8 spaces)
h[156:157] = typeflag
h[257:263] = b'ustar\x00'
h[263:265] = b'00'
assert len(h) == 512, "header length %d != 512" % len(h)
# 计算校验和:所有字节之和,checksum 字段视为空格
chksum = sum(h) & 0o7777777
h[148:156] = b'%06o\x00 ' % chksum # 6 octal + NUL + space = 8 bytes
assert len(h) == 512, "header length %d != 512 after checksum" % len(h)
return bytes(h)
# ---------- xz 原语 ----------
def xz_varint(v):
"""xz 变长整数编码:每字节 7 位数据(LSB first),MSB 为续位。"""
b = []
while v >= 0x80:
b.append((v & 0x7F) | 0x80)
v >>= 7
b.append(v & 0x7F)
return bytes(b)
def xz_stream_header():
"""12 字节 xz stream header (CHECK_CRC64)。
Stream Flags = [reserved(0x00), check_type(CRC64=0x04)]。
"""
flags = _XZ_FLAGS
crc = crc32(flags)
return XZ_MAGIC + flags + struct.pack('<I', crc)
def xz_block_header(dict_size):
"""xz block header(单 LZMA2 filter)。
返回 (header_bytes, 实际字典大小);header 长度即 block header 总长度。
"""
prop, real_dict = dict_prop_for(dict_size)
# filter flags: filter_id(1) + props_size(1) + props(1=prop byte)
filter_flags = bytes([LZMA2_FILTER_ID, 1, prop])
# block header content: flags(1, 0x00=1 filter) + filter_flags
content = bytes([0x00]) + filter_flags
# 填充到 (1 + len(content) + 4) 是 4 的倍数
total = 1 + len(content) + 4
pad = (4 - total % 4) % 4
content += b'\x00' * pad
bh = 1 + len(content) + 4
assert bh % 4 == 0, "bh not 4-aligned: %d" % bh
crc = crc32(bytes([bh // 4 - 1]) + content)
header = bytes([bh // 4 - 1]) + content + struct.pack('<I', crc)
assert len(header) == bh
return header, real_dict
def xz_index(records):
"""xz index。records = [(unpadded_size, uncompressed_size), ...]。"""
data = bytes([0x00]) # index indicator
data += xz_varint(len(records))
for unpadded, uncompressed in records:
data += xz_varint(unpadded)
data += xz_varint(uncompressed)
# 填充到 (len(data) + 4) 是 4 的倍数
pad = (4 - (len(data) + 4) % 4) % 4
data += b'\x00' * pad
return data + struct.pack('<I', crc32(data))
def xz_stream_footer(backward_size):
"""12 字节 xz stream footer (CHECK_CRC64)。
backward_size = (index_size / 4) - 1。
Stream Flags = [reserved(0x00), check_type(CRC64=0x04)]。
CRC32 覆盖 Backward Size + Stream Flags(6 字节,不含 CRC32 和 Footer Magic)。
"""
flags = _XZ_FLAGS
crc_data = struct.pack('<I', backward_size) + flags # 6 bytes, no magic
crc = crc32(crc_data)
return struct.pack('<I', crc) + crc_data + XZ_FOOTER_MAGIC
# ============================================================
# Part 3: 多文件 tar.xz quine 构造器
# ============================================================
class TarXzQuine:
def __init__(self, root, quine_name="quine.tar.xz", seed_name=".this_is_mayx_blog",
src_dir=""):
self.quine_name = quine_name
self.seed_name = seed_name
self.src_dir = src_dir.strip("/")
self.mtime = int(time.time())
# walk 源目录
self.walk_entries = []
self.files = []
if self.src_dir:
parts = self.src_dir.split("/")
for i in range(1, len(parts) + 1):
self.walk_entries.append(("/".join(parts[:i]), True))
for dirpath, dirnames, filenames in os.walk(root):
dirnames.sort()
for dn in dirnames:
rel = os.path.relpath(os.path.join(dirpath, dn), root).replace(os.sep, "/")
self.walk_entries.append((self._prefixed(rel), True))
for fn in sorted(filenames):
p = os.path.join(dirpath, fn)
rel = os.path.relpath(p, root).replace(os.sep, "/")
with open(p, "rb") as fp:
data = fp.read()
self.walk_entries.append((self._prefixed(rel), False))
self.files.append((self._prefixed(rel), data))
# rel -> data 映射,避免下面三处按 rel 回头线性搜索整个 files 列表
self.file_map = dict(self.files)
self.dirs = [r for r, isd in self.walk_entries if isd]
self.content = b"".join(data for _, data in self.files)
def _prefixed(self, rel):
return self.src_dir + "/" + rel if self.src_dir else rel
def tar_entries(self, d_seed, xz_size):
"""计算 tar 归档中每个条目的 (header, data_size, is_dir)。
顺序:seed, walk_entries (interleaved files+dirs), quine。
"""
entries = []
# seed 文件
seed_hdr = tar_header(self.seed_name, d_seed, self.mtime)
entries.append({"name": self.seed_name, "hdr": seed_hdr,
"data_size": d_seed, "is_dir": False})
# walk 顺序的文件和目录
for rel, isd in self.walk_entries:
if isd:
hdr = tar_header(rel, 0, self.mtime, mode=0o755, typeflag=b'5')
entries.append({"name": rel, "hdr": hdr,
"data_size": 0, "is_dir": True})
else:
# file_map 与 walk_entries 在 __init__ 里同步构造,这里必有
data = self.file_map[rel]
entries.append({"name": rel, "hdr": tar_header(rel, len(data), self.mtime),
"data_size": len(data), "is_dir": False})
# quine 文件(放在最后)
qhdr = tar_header(self.quine_name, xz_size, self.mtime)
entries.append({"name": self.quine_name, "hdr": qhdr,
"data_size": xz_size, "is_dir": False})
return entries
# ---------- 结构计算 ----------
def _compute_q(self, d_seed):
"""计算 xz_bytes 在 tar 中的偏移 q。
q = seed + files + dirs 的 tar 占用 + quine tar header。
q 不依赖 xz_size(quine 的 data 在 q 之后)。"""
q = 0
# seed 文件
q += TAR_BLOCK # seed tar header
q += round_up_512(d_seed) # seed data + pad
# walk 顺序的文件和目录
for rel, isd in self.walk_entries:
q += TAR_BLOCK # tar header
if not isd:
q += round_up_512(len(self.file_map[rel]))
# quine tar header
q += TAR_BLOCK
return q
def _trim_layouts(self, base, o_g, f_g, q, w_plant_off, d_seed):
"""枚举 trim 配置,产出一批「相位」不同的候选布局。
trim = [可选的裸 store chunk,载荷长度 x0] + 一组标准 gear
(store 载荷 x = s+3,紧接着一个 lzma 复制这 x 字节)。
裸 store 把 slip 抬高 3、把 f 抬高 3+x0;随后这组 gear 把 slip 重新归一到
len(cb),jump 再归一到 -3。净效果是 (o_pre, f_pre) 被整体平移
δ ≈ s_g + 6 + len(cb) + 3 + x0,而 δ 决定了自洽方程的模 4 相位。
旧实现 δ 恒为 0,相位一旦不对就永远无解 —— 而 d_seed += 1 根本不改变
布局(q 只在跨 512 字节块时才变),所以那 80 次重试是纯粹的无效空转。
注意:不能只插一个裸的 lzma 复制块。lzma 复制块的输出必须等于
F[o-q : o-q+olen],而这只能靠前面那个「载荷 = F[o-q : o-q+x] 且 x = s+3」
的 store chunk 把字节搬到正确位置来实现;裸插复制块解压出来和 F 对不上。
返回 [(chunks, o_before_jump, f_before_jump, y, jb), ...],按文件体积升序。
"""
s_g = f_g - (o_g - q)
out = []
seen = set()
for x0 in range(0, s_g + 4):
chunks = list(base)
o, f = o_g, f_g
if x0:
# 裸 store:载荷 = F[o-q : o-q+x0],要求 x0 <= s+3 以免读到尚未定稿的字节
chunks.append({"kind": "store", "foff": f, "size": 3 + x0,
"ooff": o, "olen": x0})
o += x0
f += 3 + x0
o, f = _add_gear(chunks, o, f, q) # 一组标准 gear
# jump 不动点:y = s + 3 + len(jb)
s = f - (o - q)
y = max(2, s + 17)
jb = ju = None
for _ in range(64):
try:
jb, ju = lzma_chunk([("m", o - w_plant_off, y)], o)
except AssertionError:
jb = None
break
y2 = s + 3 + len(jb)
if y2 == y:
break
if y2 < 2 or y2 > 273:
jb = None
break
y = y2
if jb is None or ju != y:
continue
if y + w_plant_off - TAR_BLOCK > d_seed:
continue
key = (o + y, f + len(jb))
if key in seen:
continue
seen.add(key)
out.append((chunks, o, f, y, jb))
out.sort(key=lambda t: t[2] + len(t[4]))
return out
def _solve_gadget(self, chunks_pre, o_j, f_j, y, jb, q, bh, FEW,
block_hdr, dict_size, hdrA, hdrB, d_seed, k):
"""在一个给定相位的布局上求解 gadget(T / gb / suffix / tz / k)。
关键简化:tz 只依赖 u = gb + suffix 与 k;T = u + tz + 4 也只依赖 (u, k);
而 gb 又由 T 唯一确定(len(lzma_chunk(matches_for(T, T))))。
因此只需一维枚举 u 即可遍历全部解,既不漏解也远快于原来的三重暴力。
"""
o_pre = o_j + y
f_pre = f_j + len(jb)
if f_pre != (o_pre - q) - 3: # jump 应把 slip 归一到 -3
return None
chunks = list(chunks_pre)
chunks.append({"kind": "lzma", "foff": f_j, "size": len(jb),
"ooff": o_j, "olen": y, "bytes": jb, "jump": True})
best = None
for u in range(8, 512):
# 512k 必须落在 [f_pre + 2u + 2059, f_pre + 2u + 3595)
lo = f_pre + 2 * u + 2059
hi = f_pre + 2 * u + 3595
for k_cand in range(max(1, (lo - 512) // 512), (hi + 512) // 512 + 2):
num = k_cand * 512 - (o_pre + 2 * u + 8 - q - 1024)
if num % 3 != 0:
continue
tz_cand = num // 3
if tz_cand < 1024:
continue
T = u + tz_cand + 4
if T < 2:
continue
gb, gu = lzma_chunk(matches_for(T, T), o_pre + T)
gb_len = len(gb)
suffix = u - gb_len
if suffix < 1:
continue
# 精确自检(不再用带 9 字节偏差的预筛公式)
f_final = f_pre + 3 + T + gb_len + 3 + tz_cand + 1
n = f_final - (12 + bh)
bp = (4 - (12 + bh + n) % 4) % 4
unpadded = bh + n + CHECK_SIZE
idx = xz_index([(unpadded, q + k_cand * 512 + 1024)])
ix = len(idx)
suffix_actual = bp + CHECK_SIZE + ix + 12
if suffix_actual != suffix:
continue
xz_size = 12 + bh + n + bp + CHECK_SIZE + ix + 12
xz_pad_actual = round_up_512(xz_size)
if xz_pad_actual != k_cand * 512:
continue
total_w_actual = q + xz_pad_actual + 1024
o_final = o_pre + 2 * T + tz_cand
if o_final != total_w_actual:
continue
if best is not None and xz_size >= best["xz_size"]:
continue # 已找到更小的解,跳过
backward_size = ix // 4 - 1
final = list(chunks)
final.append({"kind": "store", "foff": f_pre, "size": 3 + T,
"ooff": o_pre, "olen": T, "vac": True})
final.append({"kind": "lzma", "foff": f_pre + 3 + T,
"size": gb_len, "ooff": o_pre + T, "olen": gu,
"bytes": gb})
final.append({"kind": "store",
"foff": f_pre + 3 + T + gb_len,
"size": 3 + tz_cand, "ooff": o_pre + 2 * T,
"olen": tz_cand, "trailing": True})
final.append({"kind": "term",
"foff": f_pre + 3 + T + gb_len + 3 + tz_cand,
"size": 1, "ooff": o_final, "olen": 0,
"bytes": b"\x00"})
best = {
"q": q, "d_seed": d_seed, "k": k,
"chunks": final, "block_hdr": block_hdr, "bh": bh,
"n": n, "xz_size": xz_size,
"T": T, "suffix": suffix_actual,
"bp": bp, "ix": ix,
"dict_size": dict_size,
"FEW": FEW, "hdrA": hdrA,
"hdrB": hdrB, "jump_y": y, "index": idx,
"stream_hdr": xz_stream_header(),
"footer": xz_stream_footer(backward_size),
"entries": self.tar_entries(d_seed, xz_size),
}
return best
def _layout_iterate(self, d_seed, maxdist_hint):
"""核心布局。
阶段 1a:C1 / repro / gears(只依赖 q,所有 trim 配置共用,只算一次)
阶段 1b:插入 trim 微调块,平移 (o_pre, f_pre) 改变模 4 相位
阶段 2 :按 u = gb + suffix 一维搜索自洽解
"""
block_hdr, dict_size = xz_block_header(maxdist_hint)
bh = len(block_hdr)
FEW = 12 + bh + 3
w_hdrA_off = TAR_BLOCK + 12 + bh
w_hdrB_off = TAR_BLOCK + 12 + bh + 3
w_plant_off = TAR_BLOCK + 12 + bh + 6
q = self._compute_q(d_seed)
# ---------- 阶段 1a:C1 + repro + gears ----------
base = []
o, f = 0, 12 + bh
Ls = []
rem = q + FEW
while rem > CHUNK:
Ls.append(CHUNK)
rem -= CHUNK
Ls.append(rem)
assert Ls[-1] >= 2, "末 C1 chunk 载荷 %d 太小" % Ls[-1]
k = len(Ls)
hdrA = store_hdr(CHUNK, first=False)
hdrB = store_hdr(Ls[-1], first=False)
S = 0
for j, L in enumerate(Ls):
base.append({"kind": "store", "foff": f, "size": 3 + L,
"ooff": o, "olen": L, "first": j == 0,
"c1": True, "S": S})
o += L
f += 3 + L
S += L
assert o == q + FEW
S = 0
for j, L in enumerate(Ls):
if j > 0:
srcpos = w_hdrA_off if (j < k - 1 or L == CHUNK) else w_hdrB_off
dist_h = (q + FEW + S + 3 * (j - 1)) - srcpos
hb, hu = lzma_chunk([("m", dist_h, 3)], o)
base.append({"kind": "lzma", "foff": f, "size": len(hb),
"ooff": o, "olen": hu, "bytes": hb})
o += hu
f += len(hb)
cb, cu = lzma_chunk(matches_for(q + FEW + 3 * j, L), o)
base.append({"kind": "lzma", "foff": f, "size": len(cb),
"ooff": o, "olen": cu, "bytes": cb})
o += cu
f += len(cb)
S += L
# gears:把 slip 压到 <= 16
while f - (o - q) > 16:
o, f = _add_gear(base, o, f, q)
o_g, f_g = o, f
# ---------- 阶段 1b + 阶段 2 ----------
# 相位按文件体积升序枚举;第一个成功的相位通常已经足够好,但不同相位的
# (u, tz) 不同,xz_size 仍可能相差几百字节,所以再多看几个取最小的。
tried = 0
best = None
since_hit = 0
for chunks_pre, o_j, f_j, y, jb in self._trim_layouts(
base, o_g, f_g, q, w_plant_off, d_seed):
lay = self._solve_gadget(chunks_pre, o_j, f_j, y, jb, q, bh, FEW,
block_hdr, dict_size, hdrA, hdrB,
d_seed, k)
tried += 1
if lay is not None and (best is None
or lay["xz_size"] < best["xz_size"]):
best = lay
if best is not None:
since_hit += 1
if since_hit >= 10:
break
elif tried >= 80: # 一直无解,放弃后续相位
break
if best is not None:
return best
raise RuntimeError("layout 不收敛: 无自洽解 (q=%d, f_pre=%d, o_pre=%d)"
% (q, f_g, o_g))
# ---------- 装配 ----------
def assemble(self, lay):
"""构建 xz 文件 F 和 seed。"""
F = bytearray(lay["xz_size"])
q = lay["q"]
d_seed = lay["d_seed"]
bh = lay["bh"]
FEW = lay["FEW"]
# bytearray 已全零初始化:block padding、CRC64 占位、trailing-zero 载荷
# 都不必再显式写 0(后面几处赋值只会写到各自独立的区间,不会交叉覆盖)。
# pass A: 写 xz stream header + block header + 所有 chunk bytes + suffix
F[0:12] = lay["stream_hdr"]
F[12:12+bh] = lay["block_hdr"]
jump_c = None
for c in lay["chunks"]:
if c["kind"] == "store":
F[c["foff"]:c["foff"] + 3] = store_hdr(c["olen"],
first=c.get("first", False))
else:
F[c["foff"]:c["foff"] + c["size"]] = c["bytes"]
if c.get("jump"):
jump_c = c
# block padding + CRC64(pass D 填) + index + footer
bp_off = 12 + bh + lay["n"]
ix_off = bp_off + lay["bp"] + CHECK_SIZE
F[ix_off:ix_off + lay["ix"]] = lay["index"]
sf_off = ix_off + lay["ix"]
F[sf_off:sf_off + 12] = lay["footer"]
assert jump_c is not None
# pass B: 构建 seed
y = lay["jump_y"]
plant = bytes(F[jump_c["ooff"] - q: jump_c["ooff"] - q + y])
seed = bytearray(d_seed)
seed[0:12] = lay["stream_hdr"]
seed[12:12+bh] = lay["block_hdr"]
seed[12+bh:12+bh+3] = lay["hdrA"]
seed[12+bh+3:12+bh+6] = lay["hdrB"]
seed[12+bh+6:12+bh+6+y] = plant
pad = (SEED_NOTE * (d_seed // len(SEED_NOTE) + 2))[:d_seed - 12 - bh - 6 - y]
seed[12+bh+6+y:] = pad
# 构建 tar 前缀 T(含 seed data)
T = self._build_tar_prefix_with_seed(lay, seed)
F_prefix = bytes(F[0:FEW])
P = T + F_prefix
assert len(P) == q + FEW, "P 长度 %d != q+FEW %d" % (len(P), q + FEW)
# pass C: 填充 store payloads
for c in lay["chunks"]:
if c["kind"] != "store":
continue
ooff, olen = c["ooff"], c["olen"]
if c.get("c1"):
F[c["foff"] + 3:c["foff"] + 3 + olen] = P[c["S"]:c["S"] + olen]
elif c.get("trailing"):
# 载荷是尾部零:F 全零初始化,且唯一会写到 chunk 之外的 vac_store
# 目标是 [f_pre+3, f_pre+3+T)(f_pre == (o_pre-q)-3 已由 _solve_gadget
# 断言),与本区块不重叠,因此无需写入。
continue
elif c.get("vac"):
_forward_vac(F, c, q)
else:
# 其余 store 的载荷 = 输出流自身对应位置的字节(自引用复制)
fo = ooff - q
F[c["foff"] + 3:c["foff"] + 3 + olen] = F[fo:fo + olen]
# pass D: 求解 CRC64 自引用
self._solve_crc64(F, lay, T)
return bytes(F), bytes(seed)
def _build_tar_prefix_with_seed(self, lay, seed):
"""构建 tar 前缀 T(含 seed data)。"""
entries = lay["entries"]
T = bytearray()
for e in entries[:-1]:
T += e["hdr"]
if not e["is_dir"] and e["data_size"] > 0:
data = seed if e["name"] == self.seed_name else self.file_map[e["name"]]
T += data
T += b'\x00' * (round_up_512(e["data_size"]) - e["data_size"])
T += entries[-1]["hdr"] # quine tar header
return bytes(T)
def _solve_crc64(self, F, lay, T_prefix):
"""求解 CRC64 自引用。
CRC64 值 D = CRC64(W),W = T + F + 尾部零。
D 在 W 中出现两次:F 的 CRC64 字段 + vac_store 副本。
解法(多项式逆元):
CRC64 的线性贡献可表示为 GF(2^64) 多项式乘法:
bit_reverse(contribution(D)) = D_poly * P (mod G)
其中 D_poly = bit_reverse(D),P = x^(64+trailing) mod G。
方程 D_poly * (1 ^ P1 ^ P2) = bit_reverse(CRC64(W)) (mod G)
用扩展欧几里得求 (1 ^ P1 ^ P2) 的逆元,一步求解。
"""
q = lay["q"]
bh = lay["bh"]
n = lay["n"]
bp = lay["bp"]
xz_size = lay["xz_size"]
T = lay["T"]
# CRC64 位置 1:在 F 中(block padding 后)
crc_f_off = 12 + bh + n + bp
crc_w1 = q + crc_f_off
# CRC64 位置 2:vac_store 副本
crc_f_copy = crc_f_off - T
crc_w2 = q + crc_f_copy
# 构造 W = T + F + 尾部零(先把两处 CRC64 置 0)
xz_pad = round_up_512(xz_size)
tz = xz_pad - xz_size + 1024
W = bytearray(T_prefix) + F + b'\x00' * tz
W[crc_w1:crc_w1 + CHECK_SIZE] = b'\x00' * CHECK_SIZE
W[crc_w2:crc_w2 + CHECK_SIZE] = b'\x00' * CHECK_SIZE
# CRC64(W) 的多项式表示
crc_poly = _bit_reverse64(crc64(bytes(W)))
# P1 = x^(64 + 位置1之后的bit数) mod G
trailing1 = (len(W) - crc_w1 - 8) * 8
P1 = _poly_pow(2, 64 + trailing1)
# P2 = x^(64 + 位置2之后的bit数) mod G
trailing2 = (len(W) - crc_w2 - 8) * 8
P2 = _poly_pow(2, 64 + trailing2)
# D_poly = crc_poly * inv(1 ^ P1 ^ P2)
coeff = 1 ^ P1 ^ P2
inv = _poly_minv(coeff)
if inv == 0:
raise RuntimeError("CRC64 多项式无逆元")
D_poly = _poly_mul_mod(crc_poly, inv)
D = _bit_reverse64(D_poly)
# 写入 D 到 F 的 CRC64 位置
struct.pack_into('<Q', F, crc_f_off, D)
# 重新执行 vac_store 副本
_forward_vac(F, next(c for c in lay["chunks"] if c.get("vac")), q)
# 验证
W2 = bytearray(T_prefix) + F + b'\x00' * tz
assert crc64(bytes(W2)) == D, "CRC64 求解验证失败"
def build(self, maxdist_hint=None):
if maxdist_hint is None:
maxdist_hint = 2 * (len(self.content) + 128) + (1 << 21)
last_err = None
# d_seed 在同一 512 字节块内不改变 q(seed 数据按 512 对齐),所以旧的
# "d_seed += 1" 重试完全不会改变布局,只是空转 80 次。真正需要 d_seed
# 变化的情况只有两种:seed 装不下 jump 种植串,或需要跨块改变 q 作为兜底。
for d_seed in (64, 96, 128, 160, 192, 256, 320, 384, 448,
576, 704, 832, 960, 1088, 1600, 2112, 2624, 3648):
try:
lay = self._layout_iterate(d_seed, maxdist_hint)
F, seed = self.assemble(lay)
return F, lay, seed
except (RuntimeError, AssertionError, ValueError) as e:
last_err = e
raise RuntimeError("build 多次失败: %s" % last_err)
# ============================================================
# Part 4: 验证 + main
# ============================================================
def verify(F, lay, files, quine_name):
"""验证:解压 xz → 解析 tar → 检查文件 + quine 自复制。"""
# 1. 解压 xz
try:
dec = lzma.LZMADecompressor(format=lzma.FORMAT_XZ)
tar_data = dec.decompress(F)
except Exception as e:
print("[verify] xz 解压失败: %s" % e)
return False
# 2. 检查 quine 自复制
if tar_data[lay["q"]:lay["q"] + lay["xz_size"]] != F:
print("[verify] quine 自复制不匹配")
a = tar_data[lay["q"]:lay["q"] + lay["xz_size"]]
for i in range(min(len(a), len(F))):
if a[i] != F[i]:
print("[verify] 首个不匹配在偏移 %d: tar=%02x vs xz=%02x" % (i, a[i], F[i]))
break
return False
# 3. 解析 tar
try:
tf = tarfile.open(fileobj=io.BytesIO(tar_data))
members = tf.getmembers()
except Exception as e:
print("[verify] tar 解析失败: %s" % e)
return False
print("[verify] tar 条目: %s" % [m.name for m in members])
# 4. 检查源文件
ok = True
for rel, data in files:
try:
f = tf.extractfile(rel)
if f is None:
print("[verify] 文件 %s 是目录或不存在" % rel)
ok = False
continue
extracted = f.read()
if extracted != data:
print("[verify] 文件 %s 内容不匹配" % rel)
ok = False
except KeyError:
print("[verify] 文件 %s 不在 tar 中" % rel)
ok = False
# 5. 检查 quine 文件
try:
qf = tf.extractfile(quine_name)
if qf:
qdata = qf.read()
if qdata != F:
print("[verify] quine 文件内容不匹配 (%d vs %d)" % (len(qdata), len(F)))
ok = False
else:
print("[verify] quine 文件匹配!")
except KeyError:
print("[verify] quine 文件不在 tar 中")
ok = False
print("[verify] 结果: %s" % ("通过" if ok else "失败"))
return ok
def main():
ap = argparse.ArgumentParser(description="多文件 tar.xz quine 生成器")
ap.add_argument("srcdir")
ap.add_argument("output")
ap.add_argument("--quine-name", default=None,
help="归档内 quine 文件名(默认 = output 的文件名)")
ap.add_argument("--seed-name", default=".this_is_mayx_blog")
ap.add_argument("--src-dir", default="",
help="源文件在归档内的前缀目录")
ap.add_argument("--quine-dir", default="",
help="quine 在归档内所在目录")
args = ap.parse_args()
quine_name = args.quine_name or os.path.basename(args.output)
if args.quine_dir:
quine_dir = args.quine_dir.strip("/")
quine_name = quine_dir + "/" + quine_name
bq = TarXzQuine(args.srcdir, quine_name=quine_name, seed_name=args.seed_name,
src_dir=args.src_dir)
print("[blogquine-tarxz] %d 个文件, %d 个目录, 内容 %d 字节"
% (len(bq.files), len(bq.dirs), len(bq.content)))
F, lay, seed = bq.build()
with open(args.output, "wb") as fp:
fp.write(F)
print("[blogquine-tarxz] written %s: %d 字节 (q=%d, k=%d chunks, n=%d, xz_size=%d)"
% (args.output, len(F), lay["q"], lay["k"], lay["n"], lay["xz_size"]))
print("[blogquine-tarxz] T=%d, suffix=%d (bp=%d, ix=%d), dict_size=%d"
% (lay["T"], lay["suffix"], lay["bp"], lay["ix"], lay["dict_size"]))
ok = verify(F, lay, bq.files, quine_name)
sys.exit(0 if ok else 1)
if __name__ == "__main__":
main()
```
</details>
不过对我来说,相比于TXZ格式我还是更喜欢7z一点,一是因为LZMA2本来就是7-Zip的作者发明的,XZ感觉像是摘桃子的,二是TXZ这个名字听起来有点怪,感觉不像压缩包,三是XZ Utils出过后门,尽管整个Linux社区都在使用XZ,但是我还是稍微有点偏见,所以这份TXZ的生成器我就粘贴出来给需要的人吧,我博客用7z格式就好了。
# 感想
以前总是有人说AI没有创新能力,只是对曾经在网络上出现的东西进行重组,但人何尝不是这样呢?像这次制作的7z/TXZ Quine生成器在整个网络上没有任何公开信息,当然我也知道这并不是理论上的创新,但谁说组合创新不是创新呢?再看看最近OpenAI又解决了一大堆数学难题,完全可以相信AI是真的拥有智能,所以我相信总有一天AI将能完成人类能做的所有事情,人类将不再需要额外的思考,只需要做自己想做的事情吧。
|