Overview
Request 888046 accepted
- Add patch file-5.40-ascii.patch
* fix regressions on dection of smaller ASCII files (boo#1184899)
- Add upstream comitts as patches
* file-5.40-9b0459af.patch
put attributes inside the xz magic. (boo#1184888, boo#1184891)
* file-5.40-749e1ecf.patch
If the file is less than 3 bytes, use the file length to determine type
* file-5.40-f0601504.patch
Fix regression after unsigned/signed printing changes
* file-5.40-f7705dca.patch
fix previous (cast >>)
* file-5.40-3096f87f.patch
Correct return values to exptexted
* file-5.40-4c5fe1ad.patch
Add missing parens
- Port patch
* file-5.28-btrfs-image.dif
due patch file-5.40-f0601504.patch
- Add upstream commits as patches
* file-5.40-1c677c04.patch
Don't count each byte encounter as 1, count the total number
of bytes found (Anatol Belski). This makes it behave like 5.39
* file-5.40-6b34436a.patch
remove "u" from the pattern (Joerg Jenderek)
* file-5.40-9e2becec.patch
Encoding bug fix
- Fix offsets of patches
* file-5.17-option.dif
- Created by WernerFink
- In state accepted
- Supersedes 883179
Likely candidate for failures in subversion:
[ 7946s] W: DIFF STDOUT (unordered): [ 7946s] W: | --- EXPECTED STDOUT (unordered) [ 7946s] W: | +++ ACTUAL STDOUT [ 7946s] W: | @@ -3,6 +3,8 @@ [ 7946s] W: | key [ 7946s] W: | key [ 7946s] W: | svn:mergeinfo [ 7946s] W: | + svn:mime-type [ 7946s] W: | + svn:mime-type [ 7946s] W: | Properties on 'G': [ 7946s] W: | Properties on 'G/M': [ 7946s] W: | Properties on 'G/M/N':
createrepo_c:
[ 34s] ====================================================================== [ 34s] FAIL: test_detect_compression (tests.test_compression_wrapper.TestCaseCompressionWrapper) [ 34s] ---------------------------------------------------------------------- [ 34s] Traceback (most recent call last): [ 34s] File "/home/abuild/rpmbuild/BUILD/createrepo_c-0.16.0/tests/python/tests/test_compression_wrapper.py", line 64, in test_detect_compression [ 34s] self.assertEqual(comtype, cr.XZ) [ 34s] AssertionError: 1 != 5
libmodulemd:
[ 124s] (/home/abuild/rpmbuild/BUILD/modulemd-2.12.0/x86_64-suse-linux/modulemd/module_index:2185): libmodulemd-DEBUG: 18:53:36.079: modulemd_detect_compression: Detected mime type: application/octet-stream; charset=binary (/home/abuild/rpmbuild/BUILD/modulemd-2.12.0/modulemd/tests/test_data/compression/xzipped) [ 124s] (/home/abuild/rpmbuild/BUILD/modulemd-2.12.0/x86_64-suse-linux/modulemd/module_index:2185): libmodulemd-DEBUG: 18:53:36.079: Parser error [ 124s] (/home/abuild/rpmbuild/BUILD/modulemd-2.12.0/x86_64-suse-linux/modulemd/module_index:2185): libmodulemd-DEBUG: 18:53:36.079: Error: Parser error [ 124s] Bail out! libmodulemd:ERROR:../modulemd/tests/test-modulemd-moduleindex.c:1420:test_module_index_read_compressed: assertion failed (error == NULL): Parser error (modulemd-yaml-error-quark, 2) [ 124s] --- stderr --- [ 124s] ** [ 124s] libmodulemd:ERROR:../modulemd/tests/test-modulemd-moduleindex.c:1420:test_module_index_read_compressed: assertion failed (error == NULL): Parser error (modulemd-yaml-error-quark, 2) [ 124s] -------
python-python-magic:
[ 20s] ======================================================================
[ 20s] FAIL: test_descriptions (__main__.MagicTest)
[ 20s] ----------------------------------------------------------------------
[ 20s] Traceback (most recent call last):
[ 20s] File "test/test.py", line 106, in test_descriptions
[ 20s] }, buf_equals_file=False)
[ 20s] File "test/test.py", line 49, in assert_values
[ 20s] self.assertIn(value, expected_value)
[ 20s] AssertionError: 'PDF document, version 1.2, 2 pages' not found in ('PDF document, version 1.2',)
All of those builds succeeded after this sr got moved away, so it's confirmed.
Breaks some packages, see comment
Then give me something readable ... no python stack traces but real the cause ... does the changes in file-5.40.tar.gz/python/magic.py really cause this break?
Two failures look like it's not detecting XZ compression and another looks like it lost the ability to read the page count in PDFs
The file command fresh build in a build environment can detect XZ compression:
noether:/ # cat /etc/hosts | xz - > /etc/hosts.xz noether:/ # file /etc/hosts.xz /etc/hosts.xz: XZ compressed data, checksum CRC64 noether:/ # file -z /etc/hosts.xz /etc/hosts.xz: ASCII text (XZ compressed data, checksum CRC64)
also detecting pages within pdf files never worked with file command. That would require grepping throughout the PDF file to detect the meta informations within the comments
/suse/werner> file --version file-5.39 magic file from /etc/magic:/usr/share/misc/magic /suse/werner> file Dokumente/*.pdf | grep -i page /suse/werner>
this means that python-python-magic uses more than libmagic. Beside this even if not spotted in the changelogs there is a change in python-magic (aka python API in file tar ball) ... maybe this is the reason in behaviour change of python-python-magic
--- file-5.39/python/magic.py 2018-07-09 16:24:36.000000000 +0200
+++ file-5.40/python/magic.py 2021-02-23 01:49:28.000000000 +0100
@@ -50,6 +50,14 @@ MAGIC_NO_CHECK_ENCODING = NO_CHECK_ENCOD
MAGIC_NO_CHECK_BUILTIN = NO_CHECK_BUILTIN = 4173824
+MAGIC_PARAM_INDIR_MAX = PARAM_INDIR_MAX = 0
+MAGIC_PARAM_NAME_MAX = PARAM_NAME_MAX = 1
+MAGIC_PARAM_ELF_PHNUM_MAX = PARAM_ELF_PHNUM_MAX = 2
+MAGIC_PARAM_ELF_SHNUM_MAX = PARAM_ELF_SHNUM_MAX = 3
+MAGIC_PARAM_ELF_NOTES_MAX = PARAM_ELF_NOTES_MAX = 4
+MAGIC_PARAM_REGEX_MAX = PARAM_REGEX_MAX = 5
+MAGIC_PARAM_BYTES_MAX = PARAM_BYTES_MAX = 6
+
FileMagic = namedtuple('FileMagic', ('mime_type', 'encoding', 'name'))
@@ -106,6 +114,14 @@ _errno = _libraries['magic'].magic_errno
_errno.restype = c_int
_errno.argtypes = [magic_t]
+_getparam = _libraries['magic'].magic_getparam
+_getparam.restype = c_int
+_getparam.argtypes = [magic_t, c_int, c_void_p]
+
+_setparam = _libraries['magic'].magic_setparam
+_setparam.restype = c_int
+_setparam.argtypes = [magic_t, c_int, c_void_p]
+
class Magic(object):
def __init__(self, ms):
@@ -231,6 +247,24 @@ class Magic(object):
"""
return _errno(self._magic_t)
+ def getparam(self, param):
+ """
+ Returns the param value if successful and -1 if the parameter
+ was unknown.
+ """
+ v = c_int()
+ i = _getparam(self._magic_t, param, byref(v))
+ if i == -1:
+ return -1
+ return v.value
+
+ def setparam(self, param, value):
+ """
+ Returns 0 if successful and -1 if the parameter was unknown.
+ """
+ v = c_int(value)
+ return _setparam(self._magic_t, param, byref(v))
+
def open(flags):
"""
OK ... looks like file 5.40 does grep throughout PDF files to detect not only if password protected but also for page counter /Count ... leads me to the question if the test in python-python-magic does not expect page numbers
werner/file> diff -up file-5.39/magic/Magdir/pdf file-5.40/magic/Magdir/pdf --- file-5.39/magic/Magdir/pdf 2020-05-31 12:34:40.000000000 +0200 +++ file-5.40/magic/Magdir/pdf 2021-02-23 01:49:24.000000000 +0100 @@ -1,15 +1,18 @@ #------------------------------------------------------------------------------ -# $File: pdf,v 1.12 2020/01/30 01:48:44 christos Exp $ +# $File: pdf,v 1.15 2020/08/23 00:30:45 christos Exp $ # pdf: file(1) magic for Portable Document Format # 0 name pdf +>8 search /Count +>>&0 regex [0-9]+ \b, %s pages >8 search/512 /Filter/FlateDecode/ (password protected) 0 string %PDF- PDF document !:mime application/pdf !:strength +60 +!:ext pdf >5 byte x \b, version %c >7 byte x \b.%c >0 use pdf @@ -17,6 +20,7 @@ 0 string \012%PDF- PDF document !:mime application/pdf !:strength +60 +!:ext pdf >6 byte x \b, version %c >8 byte x \b.%c >0 use pdf @@ -24,6 +28,7 @@ 0 string \xef\xbb\xbf%PDF- PDF document (UTF-8) !:mime application/pdf !:strength +60 +!:ext pdf >6 byte x \b, version %c >8 byte x \b.%c >0 use pdf @@ -33,12 +38,14 @@ 0 string %FDF- FDF document !:mime application/vnd.fdf !:strength +60 +!:ext pdf >5 byte x \b, version %c >7 byte x \b.%c 0 search/256 %PDF- PDF document !:mime application/pdf !:strength +60 +!:ext pdf >&0 byte x \b, version %c >&2 byte x \b.%c >0 use pdf
Indeed ..
werner/file> file --version file-5.39 magic file from /etc/magic:/usr/share/misc/magic werner/file> file test.pdf test.pdf: PDF document, version 1.2
versus
noether:/ # file --version file-5.40 magic file from /etc/magic:/usr/share/misc/magic noether:/ # file home/abuild/rpmbuild/SOURCES/test.pdf home/abuild/rpmbuild/SOURCES/test.pdf: PDF document, version 1.2, 2 pages
that is python-python-magic has to be fixed for file 5.40
The test.pdf is https://raw.githubusercontent.com/ahupp/python-magic/master/test/testdata/test.pdf
Similar to XZ detection ... looks like the assumptions for the return string of libmagic of file command has to be changed:
--- file-5.39/magic/Magdir/compress 2020-05-31 12:34:40.000000000 +0200 +++ file-5.40/magic/Magdir/compress 2021-03-31 01:47:28.000000000 +0200 @@ -1,5 +1,5 @@ #------------------------------------------------------------------------------ -# $File: compress,v 1.79 2020/05/30 23:53:04 christos Exp $ +# $File: compress,v 1.80 2021/03/15 17:49:24 christos Exp $ # compress: file(1) magic for pure-compression formats (no archives) # # compress, gzip, pack, compact, huf, squeeze, crunch, freeze, yabba, etc. @@ -264,7 +264,12 @@ >>5 lequad !0xffffffffffffffff non-streamed, size %lld # http://tukaani.org/xz/xz-file-format.txt -0 ustring \xFD7zXZ\x00 XZ compressed data +0 ustring \xFD7zXZ\x00 XZ compressed data, checksum +>7 byte&0xf 0x0 NONE +>7 byte&0xf 0x1 CRC32 +>7 byte&0xf 0x4 CRC64 +>7 byte&0xf 0xa SHA-256 + !:strength * 2 !:mime application/x-xz
the only thing left is subversion ... how does subversion use file or libmagic?
Fabian ? Do you have some news or more info?
No news, I suggest you file bug reports for the components which fail with the new version of file.
I don't have the full log of the failed subversion build, but you could start such a build yourself to reproduce that.
Unignored: returned to active backlog.
Request History
WernerFink created request
- Add patch file-5.40-ascii.patch
* fix regressions on dection of smaller ASCII files (boo#1184899)
- Add upstream comitts as patches
* file-5.40-9b0459af.patch
put attributes inside the xz magic. (boo#1184888, boo#1184891)
* file-5.40-749e1ecf.patch
If the file is less than 3 bytes, use the file length to determine type
* file-5.40-f0601504.patch
Fix regression after unsigned/signed printing changes
* file-5.40-f7705dca.patch
fix previous (cast >>)
* file-5.40-3096f87f.patch
Correct return values to exptexted
* file-5.40-4c5fe1ad.patch
Add missing parens
- Port patch
* file-5.28-btrfs-image.dif
due patch file-5.40-f0601504.patch
- Add upstream commits as patches
* file-5.40-1c677c04.patch
Don't count each byte encounter as 1, count the total number
of bytes found (Anatol Belski). This makes it behave like 5.39
* file-5.40-6b34436a.patch
remove "u" from the pattern (Joerg Jenderek)
* file-5.40-9e2becec.patch
Encoding bug fix
- Fix offsets of patches
* file-5.17-option.dif
factory-auto added opensuse-review-team as a reviewer
Please review sources
factory-auto accepted review
Check script succeeded
licensedigger accepted review
ok
dimstar accepted review
dimstar_suse set openSUSE:Factory:Staging:B as a staging project
Being evaluated by staging project "openSUSE:Factory:Staging:B"
dimstar_suse accepted review
Picked "openSUSE:Factory:Staging:B"
dimstar_suse accepted review
Staging Project openSUSE:Factory:Staging:B got accepted.
dimstar_suse approved review
Staging Project openSUSE:Factory:Staging:B got accepted.
dimstar_suse accepted request
Staging Project openSUSE:Factory:Staging:B got accepted.
