Sign Up
Log In
Log In
or
Sign Up
Places
All Projects
Status Monitor
Collapse sidebar
home:huakim:matrix
cpanspec
_service:obs_scm:cpanspec.obscpio
Overview
Repositories
Revisions
Requests
Users
Attributes
Meta
File _service:obs_scm:cpanspec.obscpio of Package cpanspec
07070100000000000041ED000000000000000000000003663D00E300000000000000000000000000000000000000000000001100000000cpanspec/.github07070100000001000041ED000000000000000000000002663D00E300000000000000000000000000000000000000000000001B00000000cpanspec/.github/workflows07070100000002000081A4000000000000000000000001663D00E300000368000000000000000000000000000000000000002500000000cpanspec/.github/workflows/test.yamlname: Test on: push: pull_request: branches: [ master ] jobs: suse: runs-on: ubuntu-latest container: image: registry.opensuse.org/opensuse/leap:15.4 steps: - run: | zypper -n install tar gzip - uses: actions/checkout@v3 - run: > zypper -n install perl-App-cpanminus make gcc cpanm -n Archive::Zip Pod::POM Parse::CPAN::Packages Text::Autoformat YAML::XS LWP::UserAgent Perl::PrereqScanner Algorithm::Diff - name: test run: perl -c cpanspec ubuntu: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - run: > sudo apt-get install cpanminus sudo cpanm -n Archive::Zip Pod::POM Parse::CPAN::Packages Text::Autoformat YAML::XS LWP::UserAgent Perl::PrereqScanner Algorithm::Diff - name: test run: perl -c cpanspec 07070100000003000081A4000000000000000000000001663D00E300000023000000000000000000000000000000000000001400000000cpanspec/.gitignoreMakefile blib pm_to_blib obs_cache 07070100000004000081A4000000000000000000000001663D00E300001CC3000000000000000000000000000000000000001200000000cpanspec/ArtisticNAME perlartistic - the Perl Artistic License SYNOPSIS You can refer to this document in Pod via "L<perlartistic>" Or you can see this document by entering "perldoc perlartistic" DESCRIPTION This is "The Artistic License". It's here so that modules, programs, etc., that want to declare this as their distribution license, can link to it. It is also one of the two licenses Perl allows itself to be redistributed and/or modified; for the other one, the GNU General Public License, see the perlgpl. The "Artistic License" Preamble The intent of this document is to state the conditions under which a Package may be copied, such that the Copyright Holder maintains some semblance of artistic control over the development of the package, while giving the users of the package the right to use and distribute the Package in a more-or-less customary fashion, plus the right to make reasonable modifications. Definitions "Package" refers to the collection of files distributed by the Copyright Holder, and derivatives of that collection of files created through textual modification. "Standard Version" refers to such a Package if it has not been modified, or has been modified in accordance with the wishes of the Copyright Holder as specified below. "Copyright Holder" is whoever is named in the copyright or copyrights for the package. "You" is you, if you're thinking about copying or distributing this Package. "Reasonable copying fee" is whatever you can justify on the basis of media cost, duplication charges, time of people involved, and so on. (You will not be required to justify it to the Copyright Holder, but only to the computing community at large as a market that must bear the fee.) "Freely Available" means that no fee is charged for the item itself, though there may be fees involved in handling the item. It also means that recipients of the item may redistribute it under the same conditions they received it. Conditions 1. You may make and give away verbatim copies of the source form of the Standard Version of this Package without restriction, provided that you duplicate all of the original copyright notices and associated disclaimers. 2. You may apply bug fixes, portability fixes and other modifications derived from the Public Domain or from the Copyright Holder. A Package modified in such a way shall still be considered the Standard Version. 3. You may otherwise modify your copy of this Package in any way, provided that you insert a prominent notice in each changed file stating how and when you changed that file, and provided that you do at least ONE of the following: a) place your modifications in the Public Domain or otherwise make them Freely Available, such as by posting said modifications to Usenet or an equivalent medium, or placing the modifications on a major archive site such as uunet.uu.net, or by allowing the Copyright Holder to include your modifications in the Standard Version of the Package. b) use the modified Package only within your corporation or organization. c) rename any non-standard executables so the names do not conflict with standard executables, which must also be provided, and provide a separate manual page for each non-standard executable that clearly documents how it differs from the Standard Version. d) make other distribution arrangements with the Copyright Holder. 4. You may distribute the programs of this Package in object code or executable form, provided that you do at least ONE of the following: a) distribute a Standard Version of the executables and library files, together with instructions (in the manual page or equivalent) on where to get the Standard Version. b) accompany the distribution with the machine-readable source of the Package with your modifications. c) give non-standard executables non-standard names, and clearly document the differences in manual pages (or equivalent), together with instructions on where to get the Standard Version. d) make other distribution arrangements with the Copyright Holder. 5. You may charge a reasonable copying fee for any distribution of this Package. You may charge any fee you choose for support of this Package. You may not charge a fee for this Package itself. However, you may distribute this Package in aggregate with other (possibly commercial) programs as part of a larger (possibly commercial) software distribution provided that you do not advertise this Package as a product of your own. You may embed this Package's interpreter within an executable of yours (by linking); this shall be construed as a mere form of aggregation, provided that the complete Standard Version of the interpreter is so embedded. 6. The scripts and library files supplied as input to or produced as output from the programs of this Package do not automatically fall under the copyright of this Package, but belong to whoever generated them, and may be sold commercially, and may be aggregated with this Package. If such scripts or library files are aggregated with this Package via the so-called "undump" or "unexec" methods of producing a binary executable image, then distribution of such an image shall neither be construed as a distribution of this Package nor shall it fall under the restrictions of Paragraphs 3 and 4, provided that you do not represent such an executable image as a Standard Version of this Package. 7. C subroutines (or comparably compiled subroutines in other languages) supplied by you and linked into this Package in order to emulate subroutines and variables of the language defined by this Package shall not be considered part of this Package, but are the equivalent of input as in Paragraph 6, provided these subroutines do not change the language in any way that would cause it to fail the regression tests for the language. 8. Aggregation of this Package with a commercial distribution is always permitted provided that the use of this Package is embedded; that is, when no overt attempt is made to make this Package's interfaces visible to the end user of the commercial distribution. Such use shall not be construed as a distribution of this Package. 9. The name of the Copyright Holder may not be used to endorse or promote products derived from this software without specific prior written permission. 10. THIS PACKAGE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE. The End 07070100000005000081A4000000000000000000000001663D00E30000028B000000000000000000000000000000000000000E00000000cpanspec/BUGS* This script is known to fail on the following: - perlmenu 4.0 (*horrible* file name, paths in the tar file, etc.) - Data::Dump::Streamer 2.08-40 (error: line 2: Illegal char '-' in version: Version: 2.08-40) - Mail::ClamAV 0.22 (uses Inline, so it isn't noarch) * PAR 0.92 is arch-specific because of stuff in %{_bindir}, not %{perl_vendorarch}. There's probably a way to detect that. * There is logic to exclude directories from %doc, but it doesn't seem to work all that often. * Scripts aren't added to %files reliably. Man pages for scripts are never added. * We should exclude zero-length items from %doc. 07070100000006000081A4000000000000000000000001663D00E300004D67000000000000000000000000000000000000001100000000cpanspec/COPYINGNAME perlgpl - the GNU General Public License, version 2 SYNOPSIS You can refer to this document in Pod via "L<perlgpl>" Or you can see this document by entering "perldoc perlgpl" DESCRIPTION This is "The GNU General Public License, version 2". It's here so that modules, programs, etc., that want to declare this as their distribution license, can link to it. It is also one of the two licenses Perl allows itself to be redistributed and/or modified; for the other one, the Perl Artistic License, see the perlartistic. GNU GENERAL PUBLIC LICENSE GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc. 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Library General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. -- GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) -- These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. -- 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. -- 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS -- Appendix: How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. <one line to give the program's name and a brief idea of what it does.> Copyright (C) 19yy <name of author> This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: Gnomovision version 69, Copyright (C) 19yy name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker. <signature of Ty Coon>, 1 April 1989 Ty Coon, President of Vice This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Library General Public License instead of this License. [End.] 07070100000007000081A4000000000000000000000001663D00E3000014D9000000000000000000000000000000000000001100000000cpanspec/Changes1.84.00 2023-07-09 * Ignore toml and yml files (#43) * Exlude scripts/ from removing executable bit (#45) * Fix code for Module::Build(::Tiny) (#46) 1.83.00 2023-01-23 * Fix reading META.yml (#37) * fix --debug option by adding 'use Data::Dumper' (#31) * Strip pod from summary (#38) * Switch to faster CPAN mirror * Ensure non-interactive mode (#41) * Add blank line after %autosetup (#42) 1.82.02 2022-05-05 * Deal with missing MYMETA files (#34) 1.82.01 2022-03-11 * Drop perl markup in summary line (#32) * Read from MYMETA.{json,yml} (#33) 1.81.01 2021-05-07 18:32:43+02:00 * Also read from META.json * Read 'dynamic_config' and 'provides' from META files * Move Intrusive.pm into its own subprocess * Add --debug option * Add statistics output * Add batch processing script for testing changes on a number of packages 1.80.01 * support opensuse patch comments (PATCH-FIX-UPSTREAM) * add misc block, e.g. for subpackages 1.79.1 * new release for openSUSE, changes see in git repository https://github.com/openSUSE/cpanspec * smaller bug-fixes * add add_doc and skip_doc to allow changing docs handling * ignore some more unwanted files in doc handling * fix package requires * reduce changelog space wasting, document option --old-file * update copyrights 1.78.5 * convert from Build.PL to Makefile.PL 1.78 2009-01-16 * Fix up the license list some more. * Send STDIN to /dev/null in the child when executing Makefile.PL. (Bug report from Peter Willis.) * Ignore pax_global_header file. * Assume that *.inl is a hint that this isn't noarch. * It's 2009. Update Copyright. 1.77 2008-06-16 * Only use --nodeps if we're only building a source rpm. 1.76 2008-06-16 * Drop dependency on Module::CoreList and just fetch the list from rpm. 1.75 2008-05-05 * Try $] as-is and numeric when we use it with Module::CoreList. * Switch from wget to curl in cpanget since wget is broken in Fedora 9. * The best README is probably the one with the shortest filename. 1.74 2007-12-11 * Maybe finally handle "v" in version number correctly. * Continue on YAML errors. * "GPL or Artistic" is now "GPL+ or Artistic". Other License tag-related changes will follow. 1.73 2007-07-22 * Filter repoquery results better. 1.72 2007-07-13 * Get rid of the word "Extras". * Filter out "Loading ... plugin" when running repoquery. * Properly handle "requires: perl: 0" in META.yml. 1.71 2007-06-29 * It's 2007 now, so update the copyright. * Handle .bz2 files. * Exclude config.guess, config.sub, and install.sh (usually seen near configure.) * Add option processing to cpanget. It now accepts the following options: -h Print usage message -n Don't download, only print URLs -q Work quietly 1.70 2007-03-12 * Delete pm_to_blib if it exists. * Don't include the MODULE_COMPAT magic with --old. * Exclude *.cfg from %doc. * Add BuildRequires: perl(ExtUtils::MakeMaker) when using Makefile.PL. 1.69.1 2006-10-16 * Oops, it's "OPTIMIZE", not "optimize", when running Makefile.PL. 1.69 2006-10-03 * Exclude autobuild.sh. * Change regex to also drop leading "an" or "the" from Summary. * Tiny whitespace fix. * Add patch from Chris Weyl to add --prefer-macros and handle modules specified as Foo-Bar instead of Foo::Bar (SF#1546966). * Originally -V was going to be the same as --version, but for some reason it doesn't really work. The documentation has been updated. 1.68 2006-07-20 * Fix find option ordering (patch from Ville Skyttä, #199442). * Random documentation updates. 1.67 2006-07-13 * Exclude NINJA. * Do a case-insensitive match on the possible licenses. * Add a patch from Ian Burrell to support .tgz archives. * Handle files that aren't in the current directory properly. 1.66 2006-05-16 * Fix $summary modification. * Strip leading [Vv]\.? from spec Version. * Add --epoch option. * rpm is in /bin, not /usr/bin. * Update list of licenses, and fix some to agree with rpmlint. * Use "$dep" instead of "$module" in a lot of loops to not conflict with $module that stores the name of the module we're working on. * Add --follow and some simple code to fetch build dependencies. 1.65 2006-04-26 * Exclude inc (suggested by Ian Burrell). * Massage $summary (capitalize, remove trailing ".", remove leading "A"). * Use %{__perl} instead of just "perl" when running Build.PL. 1.64 2006-03-24 * Look a couple more places for %description/Summary. * Avoid adding common directories to @doc. * If there is a Build.PL and a Makefile.PL, use Build.PL. 1.63 2006-03-24 * Look in $name.pod for %description and Summary also. * Avoid useless "Summary: SYNOPSIS". 1.62 2006-03-22 * Improve auto-detection of %description and Summary. 1.61 2006-03-11 * Add this file. * Add some command-line options: --filter-requires Specify Requires to remove --filter-provides Specify Provides to remove --add-requires Add Requires for this item --add-provides Add Provides for this item --add-buildrequires Add BuildRequires for this item --version Print the version and exit * Fix path check to allow for a directory without a trailing /. (Found in Module::Install 0.59.) * Force en_US.UTF-8. https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=185192 # vi: set ai et: 07070100000008000081A4000000000000000000000001663D00E30000039D000000000000000000000000000000000000001400000000cpanspec/DockerfileFROM opensuse/leap:15.2 RUN zypper refresh \ && zypper install -y \ osc \ perl \ perl-YAML-LibYAML \ perl-XML-Simple \ perl-Parse-CPAN-Packages \ procmail \ wget \ vim \ git \ perl-Text-Autoformat \ perl-YAML \ perl-Pod-POM \ perl-libwww-perl \ perl-Class-Accessor-Chained \ perl-Perl-PrereqScanner \ perl-Algorithm-Diff \ perl-Module-Build-Tiny \ perl-ExtUtils-Depends \ perl-ExtUtils-PkgConfig \ obs-service-format_spec_file \ && true RUN cd /tmp && wget http://www.cpan.org/modules/02packages.details.txt.gz ENV LANG=en_US.UTF-8 \ LC_CTYPE="en_US.UTF-8" \ LC_NUMERIC="en_US.UTF-8" \ LC_TIME="en_US.UTF-8" \ LC_COLLATE="en_US.UTF-8" # perl /cpanspec/cpanspec -v -f --skip-changes --pkgdetails /tmp/02packages.details.txt.gz tarball 07070100000009000081A4000000000000000000000001663D00E300000078000000000000000000000000000000000000001200000000cpanspec/MANIFESTcpanget cpanspec MANIFEST This list of files META.yml COPYING Artistic BUGS TODO Changes Makefile.PL lib/Intrusive.pm 0707010000000A000081A4000000000000000000000001663D00E3000002D9000000000000000000000000000000000000001200000000cpanspec/META.yml--- abstract: 'Generate a spec file for a CPAN module' author: - 'openSUSE perl maintainers' license: perl meta-spec: url: http://module-build.sourceforge.net/META-spec-v1.4.html version: 1.4 name: cpanspec requires: Archive::Tar: 0 Archive::Zip: 0 Carp: 0 Cwd: 0 File::Basename: 0 File::Path: 0 File::Temp: 0 FileHandle: 0 Getopt::Long: 0 LWP::UserAgent: 0 Module::CoreList: 0 POSIX: 0 PPI::Document: 0 Parse::CPAN::Packages: 0 Perl::PrereqScanner: 0 Pod::POM: 0 Pod::POM::View::Text: 0 Pod::Simple::TextContent: 0 Pod::Usage: 0 Text::Autoformat: 0 YAML: 0 locale: 0 recommends: IO::Uncompress::Bunzip2: 0 resources: license: http://dev.perl.org/licenses/ version: 1.83.00 0707010000000B000081A4000000000000000000000001663D00E300000470000000000000000000000000000000000000001500000000cpanspec/Makefile.PL# Note: this file was auto-generated by Module::Build::Compat version 0.3603 use ExtUtils::MakeMaker; WriteMakefile ( 'PL_FILES' => {}, 'INSTALLDIRS' => 'site', 'NAME' => 'cpanspec', 'EXE_FILES' => [ 'cpanget', 'cpanspec' ], 'VERSION_FROM' => 'cpanspec', 'PREREQ_PM' => { 'Parse::CPAN::Packages' => 0, 'Getopt::Long' => 0, 'YAML' => 0, 'Pod::Usage' => 0, 'Archive::Zip' => 0, 'locale' => 0, 'FileHandle' => 0, 'Text::Autoformat' => 0, 'LWP::UserAgent' => 0, 'POSIX' => 0, 'Pod::Simple::TextContent' => 0, 'File::Basename' => 0, 'Archive::Tar' => 0, 'IO::Uncompress::Bunzip2' => 0 } ) ; 0707010000000C000081A4000000000000000000000001663D00E300000EFD000000000000000000000000000000000000001700000000cpanspec/OBS-Update.md# Updating the perl modules repositories in OBS There are a number of scripts that can update the following repositories: * https://build.opensuse.org/project/show/devel:languages:perl:autoupdate * https://build.opensuse.org/project/show/devel:languages:perl:CPAN-A (B, C, D etc.) ## Usage This is how a crontab could look like: 0 2 * * * $HOME/cpanspec/bin/fetch-cpan.sh > $HOME/obs-mirror/logs/cpan.log 2>&1 0 3 * * * $HOME/cpanspec/bin/update-autoupdate.sh > $HOME/obs-mirror/logs/autoupdate.log 2>&1 0 4 * * * $HOME/cpanspec/bin/update-dlp-cpan.sh > $HOME/obs-mirror/logs/cpanupdate.log 2>&1 First you need to create `~/obs-mirror` and `~/obs-mirror/logs`. The `fetch-cpan.sh` will fetch the list of currently indexed modules from CPAN. `update-autoupdate.sh` will update the [autoupdate](https://build.opensuse.org/project/show/devel:languages:perl:autoupdate) repository. It will branch projects from [`devel:languages:perl`](https://build.opensuse.org/project/show/devel:languages:perl) and build the newest version from CPAN. Requests for submitting the new versions into `devel:languages:perl` have to be created manually. See below. `update-dlp-cpan.sh` will update the `CPAN-A`, B, C repositories. To get the current status you can use these commands: ./bin/status-perl --project devel:languages:perl:autoupdate --data ~/obs-mirror This will show how many modules in `devel:languages:perl` are outdated. ./bin/status --data ~/obs-mirror --project devel:languages:perl:CPAN- [W] the same for the `CPAN-A`, B, C repositories. ## Exclude certain modules Some modules can't be updated automatically via this script for different reasons. To prevent them from showing up in `autoupdate`, you can remove them and set the status from `todo` to `error` in `~/obs-mirror/status/perl.tsv`. For example, the latest Nagios::Plugin release is just a placeholder to say that there will be no more releases, so `perl.tsv` contains: Nagios-Plugin\terror\t... ## Usage of update scripts You can run the `bin/update` and `bin/update-perl` scripts with certain options: --max 150 This will end the script after 150 updates --ask This will prompt for every module if you want to update it --ask-commit This will generate the spec, but ask before committing ## Cache For the `CPAN-<letter>` projects there is a cache in: `~/obs-mirror/obs-cache/` It caches the versions that are already in OBS, because the returned version via the OBS API is not reliable. The format is Perl Storable. If you are moving this project to a different server or directory, you should keep the cache directory, becase refilling the cache can take days and will do a lot of API requests. ## Creating submit requests Go to https://build.opensuse.org/project/show/devel:languages:perl:autoupdate and click on the package you want to submit to devel:languages:perl. Click on "link diff" to see the changes. This link might not be available in every case (I don't know why). In that case click on the revisions to see the changes. If the build is passing and the diff looks ok, click on "Submit package". The changes will automatically be filled in. Check "Remove local package if request is accepted" and click on "Create". ## Example for updating just one module # Get newest modules from CPAN # It might take a while until a module is mirrored ./bin/fetch-cpan --data ~/obs-mirror # Update status which perl modules need to be updated ./bin/status-perl --data ~/obs-mirror --project devel:languages:perl:autoupdate --update # Update ./bin/update-perl --data ~/obs-mirror --project devel:languages:perl:autoupdate --package Mojolicious Now go to https://build.opensuse.org/package/show/devel:languages:perl:autoupdate/perl-Mojolicious and create a Submit Request. 0707010000000D000081A4000000000000000000000001663D00E3000003E3000000000000000000000000000000000000001800000000cpanspec/PodViewSpec.pmpackage PodViewSpec; use parent qw( Pod::POM::View::Text ); use Pod::POM::View; use Text::Wrap; # overwrite the link default - we don't want # to reference to "the manpage" sub view_seq_link { my ($self, $link) = @_; $link =~ s,^/,,; if ($link =~ m/^(.*)\|(.*)/) { my $ltext = $1; my $lurl = $2; if ($ltext eq 'here' || $lurl =~ m/^http/) { return "at $lurl"; } return $ltext; } return $link; } sub view_item { my ($self, $item) = @_; my $indent = ref $self ? \$self->{ INDENT } : \$INDENT; my $pad = ' ' x $$indent; local $Text::Wrap::unexpand = 0; my $title = $item->title->present($self); if ($title !~ m/^\s*$/ && $title ne '*') { $title = wrap($pad . '* ', $pad . ' ', $title); $$indent += 2; my $content = $item->content->present($self); $$indent -= 2; return "$title\n\n$content"; } else { my $content = $item->content->present($self); chomp $content; return " * $content\n"; } } 1; 0707010000000E000081A4000000000000000000000001663D00E30000022C000000000000000000000000000000000000001000000000cpanspec/READMETool to create spec files for perl packages. INSTALL: -------- Requires: * perl(Algorithm::Diff) * perl(Archive::Zip) * perl(Class::Accessor::Chained) * perl(File::ShareDir::Install) * perl(LWP::UserAgent) * perl(Module::Build) * perl(Module::Build::Tiny) * perl(Parse::CPAN::Packages) * perl(Perl::PrereqScanner) * perl(Pod::POM) * perl(Text::Autoformat) * perl(YAML::XS) * obs-service-format_spec_file EXAMPLE USE: ------------ * Call "cpan" at least once to initialize * call "./cpanspec -v -f WWW::Mechanize" to build perl-WWW-Mechanize package 0707010000000F000081A4000000000000000000000001663D00E300000277000000000000000000000000000000000000000E00000000cpanspec/TODO* Planned features (as of 2005-09-19): - Download from CPAN automatically when executed as "cpanspec Foo::Bar". + DONE! + Add --download-only or something similar to replace cpanget. * It would be useful to be able to write the spec to stdout. * Add options to filter out Provides/Requires. - Done, but the documentation needs work. * Do a better job of detecting scripts in the package. * Check the search path for rpm, rpmbuild, etc. instead of hard-coding paths. * Try to get cpan_home from CPAN::MyConfig or CPAN::Config instead of hard-coding $pkgdetails. * Add an option to honor %_sourcedir and such. 07070100000010000041ED000000000000000000000002663D00E300000000000000000000000000000000000000000000000D00000000cpanspec/bin07070100000011000081A4000000000000000000000001663D00E30000067F000000000000000000000000000000000000001600000000cpanspec/bin/batch.pl#!/usr/bin/perl use strict; use warnings; use 5.010; use Data::Dumper; use Getopt::Long; GetOptions( "debug" => \my $debug, "dry" => \my $dry, "help|h" => \my $help, ) # flag or die "Error in command line arguments"; if ($help) { print <<'EOM'; Usage: batch.pl /path/to/packages 0 50 # process the first 51 packages EOM exit; } my ($dir, $from, $to) = @ARGV; $from ||= 0; $to ||= $from; my @skip = qw/ perl-AcePerl perl-Acme-MetaSyntactic perl-Acme-Ook perl-Algorithm-Munkres perl-Alien-LibGumbo perl-Alien-SVN perl-Alien-Tidyp perl-Apache-AuthNetLDAP perl-Apache-Filter perl-Apache-SessionX perl-Apache-Gallery perl-App-ProcIops perl-App-Nopaste perl-App-SVN-Bisect perl-App-gcal perl-Array-Dissect perl-Audio-CD perl-Authen-SASL-Cyrus perl-BIND-Conf_Parser perl-BSXML perl-Boost-Geometry-Utils perl-Class-Accessor-Chained Class-Multimethods perl-Crypt-HSXKPasswd perl-Crypt-Rot13 /; my %skip; @skip{ @skip } = (); opendir my $dh, $dir or die $!; my @pkgs = sort grep { -d "$dir/$_" && m/^perl-/ and not exists $skip{ $_ } } readdir $dh; closedir $dh; my $count = @pkgs; say "Total: $count"; my $opt_debug = $debug ? '--debug' : ''; for my $i ($from .. $to) { my $pkg = $pkgs[ $i ]; chdir $dir; say "=========== ($i) $pkg"; next if $dry; chdir $pkg; my $mod = $pkg; $mod =~ s/^perl-//; my @glob = glob("$mod*"); my $cmd = qq{cpanspec $opt_debug -v -f --skip-changes --pkgdetails /tmp/02packages.details.txt.gz @glob 2>&1}; say "Cmd: $cmd"; my $out = qx{$cmd}; say $out; } 07070100000012000081ED000000000000000000000001663D00E300000B38000000000000000000000000000000000000001A00000000cpanspec/bin/build-status#!/usr/bin/env perl # Shows the OBS build status statistics for a certain project. use strict; use warnings; use 5.010; use FindBin '$Bin'; use lib "$Bin/../lib"; use CPAN2OBS qw/ debug info prompt /; use XML::Simple qw/ XMLin /; use YAML::XS; use Getopt::Long; GetOptions( "project=s" => \my $project, "project-prefix=s" => \my $project_prefix, "yaml" => \my $as_yaml, "lastbuild" => \my $lastbuild, "repo=s" => \my $repo, "arch=s" => \my $arch, "help|h" => \my $help, ); usage(), exit if $help; $arch ||= 'x86_64'; $repo ||= 'openSUSE_Tumbleweed'; my $apiurl = "https://api.opensuse.org"; my $cpan2obs = CPAN2OBS->new({ apiurl => $apiurl, }); my %counts; if ($project_prefix) { my @letters = ('A' .. 'Z'); if (@ARGV) { @letters = map { uc } @ARGV; } for my $letter (@letters) { my $project = "$project_prefix$letter"; build_status($cpan2obs, $project, $letter, \%counts); } } elsif ($project) { my ($key) = @ARGV; $key //= "key"; build_status($cpan2obs, $project, $key, \%counts); } if ($as_yaml) { say YAML::XS::Dump \%counts; exit; } my @states = qw/ total building finished scheduled blocked broken succeeded failed unresolvable disabled excluded /; my @states_title = qw/ total building finished scheduled blocked broken succeeded failed unres. disabled excluded /; my $fmt = join ' | ', ('%9s') x @states; info sprintf "%10s | $fmt", '', @states_title; for my $letter (sort keys %counts) { my $localcounts = $counts{ $letter }; info sprintf "%10s | $fmt", $letter, map { $_ || 0 } @$localcounts{ @states }; } sub build_status { my ($self, $project, $letter, $counts) = @_; my $apiurl = $self->apiurl; my $args = "repository=$repo&arch=$arch"; if ($lastbuild) { $args .= '&lastbuild=1'; } my $cmd = sprintf "osc -A %s api '/build/%s/_result?$args'", $apiurl, $project; debug("CMD $cmd"); open my $fh, "-|", $cmd; my $res = XMLin($fh, forcearray => [qw/status/]); close $fh; my %localcounts; if ($res && $res->{result}) { my $result = $res->{result}; if (ref $result ne 'ARRAY') { $result = [$result]; } for my $r (@$result) { my $packages = $r->{status}; for my $pkg (@$packages) { my $code = $pkg->{code}; $localcounts{ $code }++; $localcounts{total}++; } } } $counts->{ $letter } = \%localcounts; } sub usage { info <<"EOM"; Usage: $0 --project devel:languages:perl:autoupdate --yaml autoupdate --repo standard $0 --project devel:languages:perl --yaml perl $0 --project devel:languages:perl perl --repo SLE_15 $0 --project devel:languages:perl --yaml perl --repo openSUSE_Tumbleweed --arch ppc EOM } 07070100000013000081ED000000000000000000000001663D00E30000031A000000000000000000000000000000000000001800000000cpanspec/bin/fetch-cpan#!/usr/bin/env perl # Will fetch module list from CPAN and update status files under # $data/cpan/$letter.tsv use strict; use warnings; use 5.010; use FindBin '$Bin'; use lib "$Bin/../lib"; use Getopt::Long; use CPAN2OBS qw/ debug info prompt /; GetOptions( "data=s" => \my $data, "help|h" => \my $help, ); usage(), exit if $help; die "--data missing" unless $data; my $details_url = "http://www.cpan.org/modules/02packages.details.txt.gz"; my @skip = qw/ Acme-DependOnEverything Acme-Shining /; my %skip; @skip{ @skip } = (1) x @skip; my $cpan2obs = CPAN2OBS->new({ data => $data, skip => \%skip, cpandetails => $details_url, }); $cpan2obs->fetch_cpan_list(); sub usage { print <<"EOM"; Usage: $0 --data <data-dir> $0 --data ~/obs-mirror EOM } 07070100000014000081ED000000000000000000000001663D00E3000000C4000000000000000000000000000000000000001B00000000cpanspec/bin/fetch-cpan.sh#!/bin/bash case "$1" in -h|--help) echo "Wrapper for bin/fetch-cpan" exit ;; esac DIR="$( dirname ${BASH_SOURCE[0]} )/.." cd $DIR ./bin/fetch-cpan --data ~/obs-mirror 07070100000015000081ED000000000000000000000001663D00E3000007B2000000000000000000000000000000000000001400000000cpanspec/bin/status#!/usr/bin/env perl use strict; use warnings; use 5.010; use FindBin '$Bin'; use lib "$Bin/../lib"; use CPAN2OBS qw/ debug info prompt /; use Getopt::Long; GetOptions( "update" => \my $update, "data=s" => \my $data, "project=s" => \my $project, "help|h" => \my $help, ); usage(), exit if $help; die "--data missing" unless $data; $project ||= "home:tinita:cpan-mirror:CPAN-"; #$project ||= "devel:languages:perl:CPAN-"; my $apiurl = "https://api.opensuse.org"; my @letters = ('A' .. 'Z'); if (@ARGV) { @letters = map { uc } @ARGV; } my $cpan2obs = CPAN2OBS->new({ data => $data, apiurl => $apiurl, project_prefix => $project, }); my @states = qw/ total done todo new error disabled /; my @status_list; for my $letter (@letters) { if ($update) { $cpan2obs->update_status($letter); } my $states = $cpan2obs->fetch_status($letter); my %counts; for my $dist (sort keys %$states) { my $dist_status = $states->{ $dist }; my $status = $dist_status->[0]; my $obs_ok = $dist_status->[4]; if ($status eq 'done' and not $obs_ok) { $status = 'disabled'; } if ($status =~ m/^error/) { $status = 'error'; } $counts{ $status }++; $counts{total}++; next; } push @status_list, [ $letter, map { $counts{ $_ } || 0 } @states, ]; } info sprintf " | %10s | %10s | %10s | %10s | %15s | %10s", @states; for my $item (@status_list) { info sprintf "%s | %10s | %10s | %10s | %10s | %15s | %10s", @$item; } exit; sub usage { print <<"EOM"; Usage: $0 --data ~/obs-mirror --project devel:languages:perl:CPAN- A B $0 --data ~/obs-mirror --project devel:languages:perl:CPAN- $0 --data ~/obs-mirror --project devel:languages:perl:CPAN- --update Report status of modules to update. With --update it will look into the latest data fetched with fetch-cpan` and update the status. EOM } 07070100000016000081ED000000000000000000000001663D00E30000063F000000000000000000000000000000000000001900000000cpanspec/bin/status-perl#!/usr/bin/env perl use strict; use warnings; use 5.010; use FindBin '$Bin'; use lib "$Bin/../lib"; use CPAN2OBS qw/ debug info prompt /; use Getopt::Long; GetOptions( "update" => \my $update, "data=s" => \my $data, "project=s" => \my $project, "help|h" => \my $help, ); usage(), exit if $help; die "--data missing" unless $data; $project ||= "home:tinita:CPAN:autoupdate"; #$project ||= "devel:languages:perl:autoupdate"; my $apiurl = "https://api.opensuse.org"; my $cpan2obs = CPAN2OBS->new({ data => $data, apiurl => $apiurl, project_prefix => $project, }); my @states = qw/ total done todo new older error /; my @status_list; if ($update) { $cpan2obs->update_status_perl(); } my $states = $cpan2obs->fetch_status_perl(); my %counts; for my $dist (sort keys %$states) { my $dist_status = $states->{ $dist }; my $status = $dist_status->[0]; if ($status =~ m/^error/) { $status = 'error'; } $counts{ $status }++; $counts{total}++; next; } push @status_list, [ "perl", map { $counts{ $_ } || 0 } @states, ]; info sprintf " | %10s | %10s | %10s | %10s | %10s | %10s", @states; for my $item (@status_list) { info sprintf "%s | %10s | %10s | %10s | %10s | %10s | %10s", @$item; } exit; sub usage { print <<"EOM"; Usage: $0 --data ~/obs-mirror --project devel:languages:perl:autoupdate $0 --data ~/obs-mirror --project devel:languages:perl:autoupdate --update Report status of modules to update. With --update it will look into the latest data fetched with fetch-cpan` and update the status. EOM } 07070100000017000081ED000000000000000000000001663D00E300000705000000000000000000000000000000000000001400000000cpanspec/bin/update#!/usr/bin/env perl use strict; use warnings; use 5.010; use FindBin '$Bin'; use lib "$Bin/../lib"; use CPAN2OBS qw/ debug info prompt /; use Data::Dumper; use Getopt::Long; my $max = 50; GetOptions( 'ask' => \my $ask, 'ask-commit' => \my $ask_commit, 'max=i' => \$max, 'data=s' => \my $data, 'project=s' => \my $project, 'package=s@' => \my @packages, 'redo' => \my $redo, 'help|h' => \my $help, ); usage(), exit if $help; usage(), exit 1 if (not $project or not $data); #$project ||= "home:tinita:cpan-mirror:CPAN-"; #$project ||= "devel:languages:perl:CPAN-"; unless (@ARGV) { usage(); exit 1; } my ($letter) = @ARGV; unless ($letter =~ m/^[A-Za-z]\z/) { usage(); exit 1; } $letter = uc $letter; my $cpan2obs = CPAN2OBS->new({ data => $data, cpanmirror => 'http://cpan.noris.de', apiurl => 'https://api.opensuse.org', cpanspec => "$Bin/../cpanspec", project_prefix => $project, }); info("Updating status $letter"); $cpan2obs->update_status($letter); $cpan2obs->update_obs($letter, { ask => $ask, ask_commit => $ask_commit, max => $max, packages => \@packages, redo => $redo, }); info("Updating status $letter"); $cpan2obs->update_status($letter); exit; sub usage { print <<"EOM"; Usage: $0 update [options] first-letter $0 --data ~/obs-mirror --project devel:languages:perl:CPAN- --max 20 A # Only update one package $0 --data ... --project ... --ask --ask-commit --package YAML-LibYAML # Force if status is already done $0 --data ... --project ... --ask --ask-commit --package YAML-LibYAML --redo --max Only to \$max updates, even if there are more packages to update --ask Ask before trying to update --ask-commit Ask before committing EOM } 07070100000018000081ED000000000000000000000001663D00E300000177000000000000000000000000000000000000002200000000cpanspec/bin/update-autoupdate.sh#!/bin/bash case "$1" in -h|--help) echo "Wrapper for bin/status-perl && bin/update-perl" exit ;; esac DIR="$( dirname ${BASH_SOURCE[0]} )/.." cd $DIR ./bin/status-perl --data ~/obs-mirror --project devel:languages:perl:autoupdate --update ./bin/update-perl \ --data ~/obs-mirror \ --project devel:languages:perl:autoupdate \ --max 20 07070100000019000081ED000000000000000000000001663D00E3000001D6000000000000000000000000000000000000002000000000cpanspec/bin/update-dlp-cpan.sh#!/bin/bash case "$1" in -h|--help) echo "Wrapper for bin/status && bin/update" exit ;; esac DIR="$( dirname ${BASH_SOURCE[0]} )/.." cd $DIR ./bin/status --data ~/obs-mirror --project devel:languages:perl:CPAN- --update for LETTER in A B C D E F G H I J K L M N O P Q R S T U V W X Y Z do echo $LETTER ./bin/update \ --data ~/obs-mirror \ --project devel:languages:perl:CPAN- \ --max 20 \ $LETTER done 0707010000001A000081ED000000000000000000000001663D00E30000066E000000000000000000000000000000000000001900000000cpanspec/bin/update-perl#!/usr/bin/env perl use strict; use warnings; use 5.010; use FindBin '$Bin'; use lib "$Bin/../lib"; use CPAN2OBS qw/ debug info prompt /; use Data::Dumper; use Getopt::Long; my $max = 50; GetOptions( 'ask' => \my $ask, 'ask-commit' => \my $ask_commit, 'max=i' => \$max, 'data=s' => \my $data, 'project=s' => \my $project, 'package=s@' => \my @packages, 'redo' => \my $redo, 'help|h' => \my $help, ); usage(), exit if $help; usage(), exit 1 if (not $project or not $data); #$project ||= "home:tinita:CPAN:autoupdate"; #$project ||= "devel:languages:perl:CPAN-"; my $cpan2obs = CPAN2OBS->new({ data => $data, cpanmirror => 'http://cpan.metacpan.org', apiurl => 'https://api.opensuse.org', cpanspec => "$Bin/../cpanspec", project_prefix => $project, }); my $letter = 'perl'; info("Updating status $letter"); $cpan2obs->update_status_perl; $cpan2obs->update_obs_perl({ ask => $ask, ask_commit => $ask_commit, max => $max, packages => \@packages, redo => $redo, }); info("Updating status $letter"); $cpan2obs->update_status_perl; exit; sub usage { print <<"EOM"; Usage: $0 update [options] $0 --data ~/obs-mirror --project devel:languages:perl:autoupdate --max 20 # Only update one package $0 --data ... --project ... --ask --ask-commit --package YAML-LibYAML # Force if status is already done $0 --data ... --project ... --ask --ask-commit --package YAML-LibYAML --redo --max Only to \$max updates, even if there are more packages to update --ask Ask before trying to update --ask-commit Ask before committing EOM } 0707010000001B000081ED000000000000000000000001663D00E30000027A000000000000000000000000000000000000001600000000cpanspec/checkallcpan#! /bin/bash n=`mktemp` s=`mktemp` for l in {A..Z}; do curl -s "https://api.opensuse.org/public/build/devel:languages:perl:CPAN-$l/_result?repository=openSUSE_Leap_42.2&arch=x86_64" | sed -e "s,nothing,\nnothing,g" >> $n curl -s "https://api.opensuse.org/public/build/devel:languages:perl:CPAN-$l/_result?repository=openSUSE_Leap_42.2&arch=x86_64&code=succeeded" | grep package= | sed -e 's,.*package=",,; s,".*,,' >> $s done n2=`mktemp` grep nothing $n | sed -e 's,.*nothing provides perl(,,; s,).*,,' | sort | uniq -c | sort -n > $n2 for i in `sed -e 's,^perl-,,; s,-,::,g' $s` ; do grep " $i\$" $n2 ; done | sort -nr | head 0707010000001C000081ED000000000000000000000001663D00E300000695000000000000000000000000000000000000001100000000cpanspec/cpanget#!/bin/sh set -e CPAN=${CPAN:-"http://www.cpan.org"} packages=$HOME/.cpan/sources/modules/02packages.details.txt.gz npackages=$HOME/.local/share/.cpan/sources/modules/02packages.details.txt.gz if test -e $npackages; then packages=$npackages fi quiet='' only_url=0 help() { cat <<END Usage: $( basename $0 ) [options] [module [module [...]]] Options: -h Print this message -n Don't download, only print URLs -q Work quietly END } for n in "$@" ; do if [ "$( echo '' "$n" | sed 's/^ //;s/^\(.\).*$/\1/' )" = "-" ] ; then shift for arg in $( echo '' "$n" | sed 's/^ //;s/^-//;s/./ &/g' ) ; do case $arg in [h?]) help exit 0 ;; n) only_url=1 ;; q) quiet='-s' ;; *) echo "Unknown option '$arg'." help exit 1 ;; esac done else break fi done if [ $# -eq 0 ] ; then help exit 1 fi mkdir -p $( dirname $packages ) (cd $(dirname $packages) && wget -q -N -nd $CPAN/modules/$( basename $packages ) ) for module in "$@" ; do tar=$( zgrep '^'$module' ' $packages | awk '{print $3}' ) if [ -z "$tar" ] ; then echo "Can't find $module, skipping..." >&2 continue fi if [ $only_url -eq 0 ] ; then curl $quiet -R -o "$( basename $tar )" $CPAN/authors/id/$tar else echo "$module: $CPAN/authors/id/$tar" fi done # vi: set ai et: 0707010000001D000081ED000000000000000000000001663D00E30000DE61000000000000000000000000000000000000001200000000cpanspec/cpanspec#!/usr/bin/perl # # cpanspec - Generate a spec file for a CPAN module # # Copyright (C) 2004-2009 Steven Pritchard <steve@kspei.com> # This program is free software; you can redistribute it # and/or modify it under the same terms as Perl itself. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. =head1 NAME cpanspec - Generate a spec file for a CPAN module =head1 SYNOPSIS cpanspec [options] [file [...]] Options: --help -h Help message --old -o Be more compatible with old RHL/FC releases --license -l Include generated license texts if absent in source --noprefix -n Don't add perl- prefix to package name --force -f Force overwriting existing spec --packager -p Name and email address of packager (for changelog) --release -r Release of package (defaults to 0) --epoch -e Epoch of package --disttag -d Disttag (defaults to %{?dist}) --srpm -s Build a source rpm --build -b Build source and binary rpms --cpan -c CPAN mirror URL --verbose -v Be more verbose --prefer-macros -m Prefer macros over environment variables in the spec Long options: --follow Process build dependencies --filter-requires Specify Requires to remove --filter-provides Specify Provides to remove --add-requires Add Requires for this item --add-provides Add Provides for this item --add-buildrequires Add BuildRequires for this item --old-file=s Old archive file for extraction of changelog difference --skip-changes Do not create or update .changes file --generate-changes Create or update .changes file --version Print the version number and exit =head1 DESCRIPTION B<cpanspec> will generate a spec file to build a rpm from a CPAN-style Perl module distribution. =head1 OPTIONS =over 4 =item B<-h>, B<--help> Print a brief help message and exit. =item B<-o>, B<--old> Be more compatible with old RHL/FC releases. With this option enabled, the generated spec file =over 4 =item * Defines perl_vendorlib or perl_vendorarch. =item * Includes explicit dependencies for core Perl modules. =item * Uses C<%check || :> instead of just C<%check>. =item * Includes a hack to remove LD_RUN_PATH from Makefile. =back =item B<-l>, B<--license> Generate COPYING and Artistic license texts if the source doesn't seem to include them. =item B<-n>, B<--noprefix> Don't add I<perl-> prefix to the name of the package. This is useful for perl-based applications (such as this one), so that the name of the rpm is simply B<cpanspec> instead of B<perl-cpanspec>. =item B<-f>, B<--force> Force overwriting an existing spec file. Normally B<cpanspec> will refuse to overwrite an existing spec file for safety. This option removes that safety check. Please use with caution. =item B<-p>, B<--packager> The name and email address of the packager. Overrides the C<%packager> macro in C<~/.rpmmacros>. =item B<-r>, B<--release> The release number of the package. Defaults to 0. =item B<-e>, B<--epoch> The epoch number of the package. By default, this is undefined, so no epoch will be used in the generated spec. =item B<-d>, B<--disttag> Disttag (a string to append to the release number), used to differentiate builds for various releases. Defaults to the semi-standard (for Fedora) string C<%{?dist}>. =item B<-s>, B<--srpm> Build a source rpm from the generated spec file. =item B<-b>, B<--build> Build source and binary rpms from the generated spec file. B<Please be aware that this is likely to fail!> Even if it succeeds, the generated rpm will almost certainly need some work to make rpmlint happy. =item B<-c>, B<--cpan> The URL to a CPAN mirror. If not specified with this option or the B<CPAN> environment variable, defaults to L<https://cpan.metacpan.org/>. =item B<-v>, B<--verbose> Be more verbose. =item B<-m>, B<--prefer-macros> Prefer the macro form of common spec constructs over the environment variable form (e.g. %{buildroot} vs $RPM_BUILD_ROOT). =item B<--follow> Add build dependencies to the list of modules to process. =item B<--filter-requires> Specify Requires to remove. =item B<--filter-provides> Specify Provides to remove. =item B<--add-requires> Add Requires for this item. =item B<--add-provides> Add Provides for this item. =item B<--add-buildrequires> Add BuildRequires for this item. =item B<--version> Print the version number and exit. =back =head1 AUTHOR Steven Pritchard <steve@kspei.com> =head1 SEE ALSO L<perl(1)>, L<cpan2rpm(1)>, L<cpanflute2(1)> =cut use strict; use warnings; use 5.010; our $NAME = "cpanspec"; our $VERSION = '1.84.00'; my $script = __FILE__; use Cwd; BEGIN { my ($wd) = Cwd::abs_path($0) =~ m-(.*)/-; $wd ||= '.'; unshift @INC, "$wd"; unshift @INC, "$wd/lib"; } use Module::CoreList; use FileHandle; use Archive::Tar; use Archive::Tar::Constant qw/ &DIR /; use Archive::Zip qw(:ERROR_CODES); use POSIX; use locale; use Text::Autoformat; use YAML::XS (); use Getopt::Long; use Pod::POM; use PodViewSpec; use Pod::Usage; use File::Basename; use LWP::UserAgent; use Parse::CPAN::Packages; use File::Temp; use File::Path qw(rmtree); use Perl::PrereqScanner; use Encode qw/ decode_utf8 /; use Encode::Guess; use JSON::PP (); use Data::Dumper; use Algorithm::Diff; require Carp; my $url="https://metacpan.org/release/\%{cpan_name}"; my $bin = Cwd::abs_path(dirname($script)); my $coder = JSON::PP->new->utf8; our %opt; our $help = 0; our $compat = 0; our $addlicense = 0; our $noprefix = 0; our $force = 0; our $packager; our $release = 0; our $epoch; our $disttag = '%{?dist}'; our $buildsrpm = 0; our $buildrpm = 0; our $verbose = 0; our $follow = 0; our $macros = 1; our $skip_changes = 0; our $cmdperl = "perl"; # \%{__perl} our $cmdrm = "rm"; # \%{__rm} our $cmdmake = "make"; # \%{__make} our $perllicense = "Artistic-1.0 or GPL-1.0-or-later"; our $config_file; our $old_file; our $config = {}; our $cpan = $ENV{'CPAN'} || 'https://cpan.metacpan.org'; my $debug; our $home = $ENV{'HOME'} || (getpwuid($<))[7]; die "Can't locate home directory. Please define \$HOME.\n" if (!defined($home)); our $pkgdetails = (-d "$home/.local/share/.cpan" ? "$home/.local/share" : "$home") . "/.cpan/sources/modules/02packages.details.txt.gz"; our $updated = 0; our $basedir = mkdtemp("/tmp/cpanspecXXXXXX") . "/"; our $packages; our @filter_requires; our @filter_provides; our @add_requires; our @add_provides; our @add_buildrequires; our ($file, $name, $source, $version); our ($content, $summary, $description, $author); # env. vars and their macro analogues my @MACROS = ( # 0 is for the full expansions.... { 'optimize' => '$RPM_OPT_FLAGS', 'buildroot' => '$RPM_BUILD_ROOT', }, # 1 is for the macros. { 'optimize' => '%{optflags}', 'buildroot' => '%{buildroot}', }, ); # this is set after the parameters are passed our %macro; sub print_version { print "$NAME version $VERSION\n"; exit 0; } sub verbose(@) { print STDERR @_, "\n" if ($verbose); } sub fetch($$) { my ($url, $file) = @_; my @locations = (); verbose("Fetching $file from $url..."); my $ua = LWP::UserAgent->new('env_proxy' => 1) or die "LWP::UserAgent->new() failed: $!\n"; my $request; LOOP: $request = HTTP::Request->new('GET' => $url) or die "HTTP::Request->new() failed: $!\n"; my @buf = stat($file); $request->if_modified_since($buf[9]) if (@buf); # FIXME - Probably should do $ua->request() here and skip loop detection. my $response = $ua->simple_request($request) or die "LWP::UserAgent->simple_request() failed: $!\n"; push(@locations, $url); if ($response->code eq "301" or $response->code eq "302") { $url = $response->header('Location'); die "Redirect loop detected! " . join("\n ", @locations, $url) . "\n" if (grep { $url eq $_ } @locations); goto LOOP; } if ($response->is_success) { my $fh = new FileHandle ">$file" or die "Can't write to $file: $!\n"; print $fh $response->content; $fh->close(); my $last_modified = $response->last_modified; verbose("Set last modified to $last_modified"); utime(time, $last_modified, $file) if ($last_modified); } elsif ($response->code eq "304") { verbose("$file is up to date."); } else { die "Failed to get $url: " . $response->status_line . "\n"; } } sub mkdir_p($) { my $dir = shift; my @path = split '/', $dir; for (my $n = 0; $n < @path; $n++) { my $partial = "/" . join("/", @path[0 .. $n]); if (!-d $partial) { verbose("mkdir($partial)"); mkdir $partial or die "mkdir($partial) failed: $!\n"; } } } sub update_packages() { return 1 if ($updated); return 1; verbose("Updating $pkgdetails..."); mkdir_p(dirname($pkgdetails)) if (!-d dirname($pkgdetails)); fetch("$cpan/modules/" . basename($pkgdetails), $pkgdetails); $updated = 1; } sub get_file($) { my $ofile = shift; my ($file, $name, $version, $type); # Look up $ofile in 02packages.details.txt. verbose("Get file $ofile"); update_packages(); if (!defined($packages)) { verbose "parsing packages"; $packages = Parse::CPAN::Packages->new($pkgdetails); verbose "done"; } die "Parse::CPAN::Packages->new() failed: $!\n" unless defined $packages; my ($m, $d); if ($m = $packages->package($ofile) and $d = $m->distribution()) { $source = $cpan . "/authors/id/" . $d->prefix(); $file = basename($d->filename()); fetch($source, $file) unless -f $file; $name = $d->dist(); $version = $d->version(); $version =~ s/^v\.?//; $type = $d->prefix() =~ /\.zip/ ? "zip" : "tar"; } else { warn "Failed to parse '$file' or find a module by that name, skipping...\n"; } return ($file, $name, $version, $type); } sub get_source($) { my $file = shift; # keep things happy if we get "Foo-Bar" instead of "Foo::Bar" verbose("Get source $file"); # Look up $file in 02packages.details.txt. update_packages(); if (!defined($packages)) { $packages = Parse::CPAN::Packages->new($pkgdetails); } die "Parse::CPAN::Packages->new() failed: $!\n" if (!defined($packages)); my ($m, $d); if ($d = $packages->latest_distribution($file)) { $source = $cpan . "/authors/id/" . $d->prefix(); $source =~ s/$name/\%{cpan_name}/; } else { warn "Failed to parse '$file' or find a module by that name in $pkgdetails, skipping...\n"; $source = ''; return; } } sub build_rpm($) { my $spec = shift; my $dir = getcwd(); my $rpmbuild = (-x "/usr/bin/rpmbuild" ? "/usr/bin/rpmbuild" : "/bin/rpm"); verbose("Building " . ($buildrpm ? "rpms" : "source rpm") . " from $spec"); # From Fedora CVS Makefile.common. if (system($rpmbuild, "--define", "_sourcedir $dir", "--define", "_builddir $dir", "--define", "_srcrpmdir $dir", "--define", "_rpmdir $dir", ($buildrpm ? "-ba" : ("-bs", "--nodeps")), $spec) != 0) { if ($? == -1) { die "Failed to execute $rpmbuild: $!\n"; } elsif (WIFSIGNALED($?)) { die "$rpmbuild died with signal " . WTERMSIG($?) . (($? & 128) ? ", core dumped\n" : "\n"); } else { die "$rpmbuild exited with value " . WEXITSTATUS($?) . "\n"; } } } sub is_in_core($$) { my ($module, $version) = (@_); return 1 if ($module eq 'perl'); return 0 if Module::CoreList::removed_from($module); my $release = Module::CoreList::first_release($module, $version); return 0 unless $release; # 10.1 is the minimum we care for my $ret = $release <= 5.008008; return $ret; } sub readfile { my ($filename, $encoding) = @_; $encoding //= ''; local $/ = undef; die "empty filename" unless length($filename); open my $fh, "<", $basedir . $filename or return undef; binmode $fh; my $string = <$fh>; if ($encoding eq 'guess') { $string = decode_latin_or_utf8($string); } close $fh; return $string; } sub get_content(%) { my %args = @_; my $pm = ""; my $cont; my $path = $args{module}; # YAML::PP::LibYAML $path =~ s,::,/,g; # YAML/PP/LibYAML my @pmfiles = ("lib/$path.pod", "lib/$path.pm"); # lib/YAML/PP/LibYAML.{pm,pod} if ($args{module} =~ /::/) { my @tmp = split '/', $path; # YAML PP LibYAML my $last = pop @tmp; # LibYAML push(@pmfiles, "lib/$last.pod", "lib/$last.pm"); # lib/LibYAML.{pm,pod} } do { push(@pmfiles, "$path.pod", "$path.pm"); } while ($path =~ s,^[^/]+/,,); # PP/LibYAML -> LibYAML push(@pmfiles, "$args{module}") # DateTime if ($args{module} !~ /::/); for my $file (@pmfiles) { $pm = (grep { $_ eq $file or $_ eq "./$file" } @{$args{files}})[0]; last if $pm; } return (undef, undef) if (!length($pm)); if (my $cont = readfile("$args{path}/$pm")) { return $cont; } else { warn "Failed to read $args{path}/$pm from $args{filename}\n"; return (undef, undef); } } sub get_description { my $cont = shift; my $title = shift || 'DESCRIPTION'; my $parser = Pod::POM->new; # extract pod; the file may contain no pod, that's ok my $pom = $parser->parse_text($cont); HEAD1: foreach my $head1 ($pom->head1) { next HEAD1 unless $head1->title eq $title; $description = ''; foreach my $item ($head1->content()) { last if ($item->type() eq 'head2'); eval { $description .= $item->present('PodViewSpec'); }; } return $description = undef unless length($description); my @paragraphs = (split /\n\n/, $description); my $limit = $config->{description_paragraphs}; if ($limit) { @paragraphs = @paragraphs[0 .. ($limit - 1)]; } $description = join "\n\n", @paragraphs; # autoformat and return... return autoformat $description, {all => 1}; } return $description = undef; } sub get_summary { my ($summary, $cont, $mod) = @_; my $parser = Pod::POM->new; # extract pod; the file may contain no pod, that's ok my $pom = $parser->parse_text($cont); HEAD1: foreach my $head1 ($pom->head1) { next HEAD1 unless $head1->title eq 'NAME'; my $pom = $head1->content; $pom =~ /^[^-]+ -* (.*)$/m; my $s = $1; if($s) { # strip markup $s =~ s/C<([^>]+)>/$1/g; return $s; } } return $summary; } sub get_author($) { my $cont = shift; my @lines = (); my $parser = Pod::POM->new; my $author; # extract pod; the file may contain no pod, that's ok my $pom = $parser->parse_text($cont); HEAD1: foreach my $head1 ($pom->head1) { next HEAD1 unless $head1->title eq 'AUTHOR'; my $pom = $head1->content; eval { $author = $pom->present('Pod::POM::View::Text'); }; my @paragraphs = (split /\n/, $author); foreach my $line (@paragraphs) { next if $line eq ""; $line =~ s/^/ /; push(@lines, $line); } $author = join "\n", @lines; # return... return $author; } return 'sorry, no author found'; } sub get_license_from_content($) { my $cont = shift; my $license; my @lines = (); my $parser = Pod::POM->new; # extract pod; the file may contain no pod, that's ok my $pom = $parser->parse_text($cont); HEAD1: foreach my $head1 ($pom->head1) { next HEAD1 unless $head1->title =~ /LICEN[CS]E/i || $head1->title =~ /COPYRIGHT/i; my $pom = $head1->content; eval { $license = $pom->present('Pod::POM::View::Text'); }; my @paragraphs = (split /\n/, $license); foreach my $line (@paragraphs) { next if $line eq ""; next if $line =~ /Copyright/i; $line =~ s/^/ /; push(@lines, $line); } $license = join " ", @lines; $license =~ s,\s+, ,g; if ($license && ( $license =~ /under the same terms (and conditions )?as Perl/ || $license =~ /same terms as the Perl 5/ || $license =~ /under the terms of the Perl artistic license/ || $license =~ qr/free software.*dev.perl.org.* for more information/)) { $license = $perllicense; } # return... return $license; } return undef; } sub check_rpm($) { my $dep = shift; my $rpm = "/bin/rpm"; return undef if (!-x $rpm); my @out = `$rpm -q --whatprovides "$dep"`; if ($? != 0) { #warn "backtick (rpm) failed with return value $?"; return undef; } return @out; } sub check_repo($) { my $dep = shift; my $repoquery = "/usr/bin/repoquery"; return undef if (!-x $repoquery); verbose("Running $repoquery to check for $dep. This may take a while..."); my @out = `$repoquery --whatprovides "$dep"`; if ($? != 0) { #warn "backtick (repoquery) failed with return value $?"; return undef; } return grep { /^\S+-[^-]+-[^-]+$/ } @out; } sub check_dep($) { my $module = shift; return (check_rpm("perl($module)") || check_repo("perl($module)")); } sub map_version($$) { my ($module, $version) = @_; if (grep { $module eq $_ } qw/Module::Build CPAN::Meta::Requirements ExtUtils::PkgConfig Test::Exception Test::Number::Delta Time::Duration DateTime::Locale File::Path/ ) { my $nv = $version . "000000"; if ($nv =~ /^(.*\.[0-9]{6})/) { return $1; } } return $version; } sub extract_old_changes($$) { my ($archive, $filename) = @_; my $tar = Archive::Tar->new; $tar->read($archive, 1, {filter => qr/\Q$filename\E$/}); my @cfile = $tar->list_files(); return $tar->get_content($cfile[0]); } sub diff_changes($$) { my @old_changes = split qr/\n/, shift; my @changes = split qr/\n/, shift; #print $old_changes, $changes; my $diff = Algorithm::Diff->new(\@old_changes, \@changes); # ignore common prefix while ($diff->Next()) { last unless $diff->Same(); } return '' unless $diff->Next(0); if( $diff->Items(2) && ! $diff->Items(1) ) { my $changesdiff = ''; $changesdiff .= " $_\n" for $diff->Items(2); # some cleanups $changesdiff =~ s/[ \t\r]+\n/\n/g; # no trailing spaces or line feeds $changesdiff =~ s/\n\n\n+/\n\n/g; # not more than a single empty line # sanitize diffs with too many empty lines my $emptylines = 0; while($changesdiff =~ /\n\n/g) { ++$emptylines; } $changesdiff =~ s/\n\n/\n/g if $emptylines > 3; return $changesdiff; } return ''; } sub add_custom_spec_section($$) { my ($spec, $section) = @_; if ($config->{$section}) { print $spec "# MANUAL BEGIN\n"; chomp $config->{$section}; print $spec $config->{$section}; print $spec "\n# MANUAL END\n"; } } sub is_license_file { my $file = shift; $file = lc $file; return 1 if $file =~ /^copying/; return 1 if $file =~ /^licen[sc]e/; return 1 if $file =~ /^artistic/; return 1 if $file =~ /^mit/; return 1 if $file =~ /^gpl/; return; } # Set locale to en_US.UTF8 so that dates in changelog will be correct # if using another locale. Also ensures writing out UTF8. (Thanks to # Roy-Magne Mo for pointing out the problem and providing a solution.) setlocale(LC_ALL, "en_US.UTF-8"); my $generate_changes = 0; GetOptions( 'help|h' => \$help, 'old|o' => \$compat, 'license|l' => \$addlicense, 'noprefix|n' => \$noprefix, 'force|f' => \$force, 'packager|p=s' => \$packager, 'release|r=i' => \$release, 'epoch|e=i' => \$epoch, 'disttag|d=s' => \$disttag, 'srpm|s' => \$buildsrpm, 'build|b' => \$buildrpm, 'cpan|c=s' => \$cpan, 'verbose|v' => \$verbose, 'follow' => \$follow, 'filter-requires=s' => \@filter_requires, 'filter-provides=s' => \@filter_provides, 'add-requires=s' => \@add_requires, 'add-provides=s' => \@add_provides, 'add-buildrequires=s' => \@add_buildrequires, 'skip-changes' => \$skip_changes, 'generate-changes' => \$generate_changes, 'old-file=s' => \$old_file, 'config=s' => \$config_file, 'version' => \&print_version, 'prefer-macros|m' => \$macros, 'pkgdetails=s' => \$pkgdetails, 'debug' => \$debug, ) or pod2usage({-exitval => 1, -verbose => 0}); if (!$skip_changes){ $skip_changes = !$generate_changes; } pod2usage({-exitval => 0, -verbose => 1}) if ($help); pod2usage({-exitval => 1, -verbose => 0}) if (!@ARGV); if ($follow and $buildrpm) { warn "Sorry, --follow and --build are mutually exclusive right now.\n" . "We can't build when tracking deps right now. Ignoring --build.\n"; $buildrpm = 0; } %macro = %{$MACROS[$macros]}; my $prefix = $noprefix ? "" : "perl-"; #$packager=$packager || `rpm --eval '\%packager'`; my $defaultdocdir = `rpm --eval '\%_defaultdocdir'`; chomp $defaultdocdir; $config_file ||= "cpanspec.yml" if -f "cpanspec.yml"; $config = YAML::XS::LoadFile($config_file) if $config_file; $config->{ignored_requires} = {}; for my $r (split(/ /, $config->{ignore_requires} || '')) { $config->{ignored_requires}->{$r} = 1; } my @args = @ARGV; my @processed = (); local $Data::Dumper::Sortkeys = 1; for my $ofile (@args) { my $type = undef; my $download = undef; my $summary; ($file, $name, $source, $version) = (undef, undef, undef, undef); my $license; ($content, $description, $author) = (undef, undef, undef); if ($ofile =~ /^(?:.*\/)?(.+)-(?:v\.?)?([^-]+)\.(tar)\.(?:gz|bz2)$/i) { $file = $ofile; $name = $1; $version = $2; $type = $3; } elsif ($ofile =~ /^(?:.*\/)?(.+)-(?:v\.?)?([^-]+)\.tgz$/i) { $file = $ofile; $name = $1; $version = $2; $type = 'tar'; } elsif ($ofile =~ /^(?:.*\/)?(.+)-(?:v\.?)?([^-]+)\.(zip)$/i) { $file = $ofile; $name = $1; $version = $2; $type = $3; } else { # keep things happy if we get "Foo-Bar" instead of "Foo::Bar" $ofile =~ s/-/::/g; # Look up $file in 02packages.details.txt. ($file, $name, $version, $type) = get_file($ofile); $download = 1; } next unless $name; my $module = $name; $module =~ s/-/::/g; my $archive; my $path; my $ext = '.gz'; my @archive_files = (); if ($type eq 'tar') { my $f = $file; if ($file =~ /\.bz2$/) { #eval { # use IO::Uncompress::Bunzip2; #}; if ($@) { warn "Failed to load IO::Uncompress::Bunzip2: $@\n"; warn "Skipping $file...\n"; next; } $f = IO::Uncompress::Bunzip2->new($file); if (!defined($f)) { warn "IO::Uncompress::Bunzip2->new() failed on $file: $!\n"; next; } $ext = '.bz2'; } my $next = Archive::Tar->iter($f, 1, {prefix => "/tmp/tar"}); while (my $f = $next->()) { next if $f->type == DIR; push(@archive_files, $f->full_path); $f->extract($basedir . $f->full_path) or warn "Extraction failed " . $f->full_path; } } elsif ($type eq 'zip') { $archive = Archive::Zip->new() or die "Archive::Zip->new() failed: $!\n"; die "Read error on $file\n" unless ($archive->read($file) == AZ_OK); $ext = '.zip'; $archive->extractTree('', $basedir); push(@archive_files, $archive->memberNames()); } my %stats; my @files = (); my $bogus = 0; my $execs = 0; my $changes; # Sort files, so that Changes comes before Changes.foo @archive_files = sort @archive_files; my $changesfile; foreach my $entry (@archive_files) { my $version0=$version; $version0=~s/0*$/0*/; # pathnames may not contain as many trailing zeros if ( $entry !~ m{^(?:./)?($name-(?:v\.?)?$version0)(?:/|$)} and $entry !~ m{^(?:./)?($name)(?:\/|$)} ) { warn "BOGUS PATH DETECTED: $entry\n"; $bogus++; next; } $path //= $1; unless ($entry =~ s{^(?:./)?$name-(?:v\.?)?$version0/}{}) { $entry =~ s{^(?:./)?$name/}{}; } next if (!$entry); push(@files, $entry); my $candidate = lc $entry; if (!$changes && ($candidate =~ m/^changes/ || $candidate =~ m/^changelog/ || $candidate =~ m/^history/)) { $changesfile = $entry; $changes = readfile("$path/$entry", 'guess'); } if (-x "$basedir$path/$entry" && -f "$basedir$path/$entry") { if ($entry !~ m/.pl$/ && $entry !~ m,^bin/,) { verbose("disable executables because of $entry"); $execs = 1; } chmod 0644, "$basedir$path/$entry"; } } if ($bogus) { warn "Skipping $file with $bogus path elements!\n"; next; } get_source($name) if(!defined $source && -d dirname($pkgdetails)); $content = get_content( filename => $file, name => $name, module => $module, version => $version, files => \@files, path => $path, ); $description = $config->{description} if $config->{description}; if (!$description) { get_description($content) || get_description($content, 'OVERVIEW'); } # $author //= get_author($content); # my $authors="Authors:\n--------\n$author"; my @doc = find_doc($config, $path, @files); my $date = strftime("%a %b %d %Y", localtime); my $noarch=!grep /\.(c|h|xs|inl)$/i, @files; my $vendorlib=($noarch ? "vendorlib" : "vendorarch"); my $lib="\%{perl_$vendorlib}"; filter_requires($name) if @filter_requires; filter_provides($name) if @filter_provides; my $specfile = "$prefix$name.spec"; verbose "Writing $specfile..."; my $spec; if ($force) { rename($specfile, "$specfile~") if (-e $specfile); $spec = new FileHandle ">$specfile"; } else { $spec = new FileHandle "$specfile", O_WRONLY | O_CREAT | O_EXCL; } unless ($spec) { warn "Failed to create $specfile: $!\n"; next; } print $spec qq[\%{!?perl_$vendorlib: \%define perl_$vendorlib \%(eval "\`$cmdperl -V:install$vendorlib\`"; echo \$install$vendorlib)}\n\n] if ($compat); my $scripts = 0; my (%build_requires, %requires, %recommends, %provides); my $dynamic = 1; my $got_prereqs = 0; if (-e "$basedir/$path/META.json") { my $results = read_meta_json("$basedir/$path/META.json", \%stats); $got_prereqs = $results->{got_prereqs}; $dynamic = $results->{dynamic}; $summary //= $results->{abstract}; if ($results->{build}) { %build_requires = %{ $results->{build_requires} }; %requires = %{ $results->{requires} }; %recommends = %{ $results->{recommends} }; } if (my $prov = $results->{provides}) { @provides{ keys %$prov } = values %$prov; } $got_prereqs = $results->{got_prereqs}; } if (-e "$basedir/$path/META.yml") { my $results = read_meta_yaml("$path/META.yml", \%stats); $dynamic = $results->{dynamic}; $summary //= $results->{abstract}; if (not $got_prereqs and $results->{build}) { %build_requires = %{ $results->{build_requires} }; %requires = %{ $results->{requires} }; %recommends = %{ $results->{recommends} }; } if (not keys %provides and my $prov = $results->{provides}) { @provides{ keys %$prov } = values %$prov; } $got_prereqs = $results->{got_prereqs}; $scripts = $results->{scripts}; $license = $results->{license}; } $stats{dynamic} = $dynamic; $debug and warn __PACKAGE__.':'.__LINE__.$".Data::Dumper->Dump([\$dynamic], ['dynamic']); my $usebuildpl = 0; if (grep /^Build\.PL$/, @files) { $usebuildpl = 1; } else { $build_requires{'ExtUtils::MakeMaker'} ||= 0; } $ENV{PERL_MM_USE_DEFAULT} = 1; if ($got_prereqs and not $dynamic) { $debug and warn "Got prereqs, no need to run Makefile.PL/Build.PL"; } else { # Run Makefile.PL/Build.PL and read from generated MYMETA.{json,yml} my $cmd = "perl Makefile.PL"; if ($usebuildpl) { $cmd = "perl Build.PL"; } my $out = qx{cd $basedir/$path; $cmd 2>&1}; if ($?) { warn "Error when running $cmd: >>$out<<\n"; } else { my $dynamic_results; if (-e "$basedir/$path/MYMETA.json") { $dynamic_results = read_meta_json("$basedir/$path/MYMETA.json", \%stats); } elsif (-e "$basedir/$path/MYMETA.yml") { $dynamic_results = read_meta_yaml("$path/MYMETA.yml", \%stats); $license //= $dynamic_results->{license}; } else { warn "No MYMETA files, output from build:>>>\n$out<<<"; } if ($dynamic_results) { $summary //= $dynamic_results->{abstract}; # Add possible new requirements to existing static ones my %newrequires = %{ $dynamic_results->{requires} }; foreach my $dep (keys(%newrequires)) { $requires{$dep} = $newrequires{$dep}; } %newrequires = %{ $dynamic_results->{build_requires} }; foreach my $dep (keys(%newrequires)) { if (defined $build_requires{$dep}) { next if version->parse($build_requires{$dep}) > version->parse($newrequires{$dep}); } $build_requires{$dep} = $newrequires{$dep}; } } } } $license //= get_license_from_content($content); $license = "CHECK($perllicense)" unless $license; $stats{license}->{spec} = $license; $summary =~ s/[CL]<([^>]+)>/$1/g if defined $summary; $summary //= get_summary($summary, $content, $module); $description //= $summary; $stats{summary} = $summary; $summary =~ s,\.$,,; $summary =~ s,^[aA] ,,; $summary = ucfirst $summary; if (length($summary) > 79) { $summary = substr($summary, 0, 72) . "[cut]"; } $summary = $config->{summary} if $config->{summary}; if (not length $summary) { $summary = "$module Perl module"; } $stats{provides} = keys %provides; dump_statistics($module, $version, \%stats); unless (%provides) { verbose("No 'provides' info in meta, parsing code.\n"); %provides = parse_provides($basedir . $path, \@files); } delete @build_requires{ keys %provides }; my %hdoc; foreach my $d (@doc) { $hdoc{$d} = 1; } rmtree($basedir); my $rpm_version = map_version($module, $version); my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time); print $spec <<END; # # spec file for package $prefix$name (Version $rpm_version) # # Copyright (c) $year SUSE LLC # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed # upon. The license for this file, and modifications and additions to the # file, is the same license as for the pristine package itself (unless the # license for the pristine package is not an Open Source License, in which # case the license is the MIT License). An "Open Source License" is a # license that conforms to the Open Source Definition (Version 1.9) # published by the Open Source Initiative. # Please submit bugfixes or comments via https://bugs.opensuse.org/ # END my $version_string = '%{version}'; print $spec <<END; %define cpan_name $name Name: $prefix$name Version: $rpm_version Release: $release END if ($rpm_version ne $version) { print $spec "%define cpan_version $version\n"; $version_string = '%{cpan_version}'; print $spec "Provides: perl($module) = $rpm_version\n"; } print $spec "Epoch: $epoch\n" if (defined($epoch)); if ($config->{license}) { if($config->{license} =~ /^Perl( License)?$/i) { $config->{license} = $perllicense; } if("CHECK($config->{license})" ne $license && $config->{license} ne $license) { print $spec "#Upstream: $license\n"; } print $spec "License: $config->{license}\n"; } else { print $spec "License: $license\n"; } print $spec <<END; Summary: $summary Url: $url END my $sfile = basename($ofile); $sfile =~ s/$name/\%{cpan_name}/; $sfile =~ s/$version/%{version}/; $source =~ s/$name/\%{cpan_name}/; $source =~ s/$version/%{version}/; my $counter = 0; if ($source) { # if (basename($source) ne $sfile && $sfile !~ /::/) { # the current source URL is not the version we're looking for # just take a guess - authors don't change daily # $source = dirname($source) . "/$sfile"; # } # $source =~ s,$version,$version_string,; print $spec "Source$counter: $source\n"; } else { print $spec "Source$counter: $sfile\n"; } $counter++; if ($config->{sources}) { for my $s (@{$config->{sources}}) { print $spec "Source$counter: $s\n"; $counter++; } } if ($config_file) { print $spec "Source$counter: $config_file\n"; } if ($config->{patches}) { $counter = 0; for my $p (sort keys %{$config->{patches}}) { my $args = $config->{patches}->{$p} || ''; print $spec "# $1\n" if $args =~ / ?(PATCH-FIX.*)/; print $spec "Patch$counter: $p\n"; $counter++; } } if ($config->{skip_noarch}) { printf $spec "# MANUAL\n#%-15s%s\n", "BuildArch:", "noarch" if $noarch; $noarch = undef; } else { printf $spec "%-16s%s\n", "BuildArch:", "noarch" if $noarch; } printf $spec "%-16s%s\n", "BuildRequires:", "perl"; printf $spec "%-16s%s\n", "BuildRequires:", "perl-macros"; for my $dep (keys(%requires)) { next if ($dep eq 'perl'); $build_requires{$dep} = $build_requires{$dep} || $requires{$dep}; } my @treqs = sort(keys(%build_requires)); for my $dep (@treqs) { my $iscore = 0; eval { $iscore = is_in_core($dep, $build_requires{$dep}); }; next if $iscore; if ($follow) { if ($dep ne $module and !(grep { $_ eq $dep } @processed, @args)) { if (check_dep($dep)) { verbose("$dep is available, skipping."); } else { verbose("$dep is not available, adding it to the list."); push(@args, $dep); } } } if (defined $build_requires{$dep}) { if ($config->{ignored_requires}->{$dep}) { printf $spec "#"; } printf $spec "%-16s%s", "BuildRequires:", "perl($dep)"; print $spec (" >= " . map_version($dep, $build_requires{$dep})) if ($build_requires{$dep}); print $spec "\n"; } } for my $dep (sort @add_buildrequires) { printf $spec "%-16s%s\n", "BuildRequires:", $dep if (length($dep)); } for my $dep (sort(keys(%requires))) { next if (is_in_core($dep, $requires{$dep})); if ($config->{ignored_requires}->{$dep}) { printf $spec "#"; } printf $spec "%-16s%s", "Requires:", "perl($dep)"; print $spec (" >= " . map_version($dep, $requires{$dep})) if ($requires{$dep}); print $spec "\n"; } for my $dep (@add_requires) { printf $spec "%-16s%s\n", "Requires:", $dep; } for my $prov (@add_provides) { printf $spec "%-16s%s\n", "Provides:", $prov; } for my $dep (sort(keys(%recommends))) { next if (is_in_core($dep, $recommends{$dep})); next if ($dep eq 'perl'); printf $spec "%-16s%s", "Recommends:", "perl($dep)"; print $spec (" >= " . $recommends{$dep}) if ($recommends{$dep}); print $spec "\n"; } if (@filter_requires) { print $spec <<END Source98: $name-filter-requires.sh \%global real_perl_requires \%{__perl_requires} \%define __perl_requires \%{_tmppath}/\%{name}-\%{version}-\%{release}-\%(\%{__id_u} -n)-filter-requires END } if (@filter_provides) { print $spec <<END Source99: $name-filter-provides.sh \%global real_perl_provides \%{__perl_provides} \%define __perl_provides \%{_tmppath}/\%{name}-\%{version}-\%{release}-\%(\%{__id_u} -n)-filter-provides END } print $spec "%{perl_requires}\n"; add_custom_spec_section($spec, "preamble"); my $buildpath = $path; $buildpath =~ s/$name/\%{cpan_name}/; $buildpath =~ s/$version/$version_string/; print $spec <<END; \%description $description \%prep END my $all_patch_args_same = 1; my $common_patch_args = undef; if ($config->{patches}) { for my $p (sort keys %{$config->{patches}}) { my $args = $config->{patches}->{$p} || undef; $args =~ s/ ?PATCH-FIX.*// if defined($args); if (!defined($common_patch_args) && defined($args)) { $common_patch_args = $args; } $all_patch_args_same = ($args // '') eq ($common_patch_args // ''); last if !$all_patch_args_same; } } my $autosetup_arg = (!$config->{patches} || $all_patch_args_same) ? (defined($common_patch_args) ? " $common_patch_args" : "") : " -N"; print $spec <<END; \%autosetup @{[($noprefix ? "" : " -n $buildpath")]}$autosetup_arg END if ($execs) { print $spec qq{find . -type f ! -path "*/t/*" ! -name "*.pl" ! -path "*/bin/*" ! -path "*/script/*" ! -path "*/scripts/*" ! -name "configure" -print0 | xargs -0 chmod 644\n}; } if ($config->{patches} && !$all_patch_args_same) { my $counter = 0; for my $p (sort keys %{$config->{patches}}) { my $args = $config->{patches}->{$p} || undef; if (defined($args)) { $args =~ s/ ?PATCH-FIX.*//; print $spec "%patch -P$counter $args\n"; } else { print $spec "%patch -P$counter\n"; } $counter++; } } if (@filter_requires) { print $spec <<'END'; sed -e 's,@@PERL_REQ@@,%{real_perl_requires},' %{SOURCE98} > %{__perl_requires} chmod +x %{__perl_requires} END } if (@filter_provides) { print $spec <<'END'; sed -e 's,@@PERL_PROV@@,%{real_perl_provides},' %{SOURCE99} > %{__perl_provides} chmod +x %{__perl_provides} END } if (grep { $_ eq "pm_to_blib" } @files) { print $spec <<'END'; rm -f pm_to_blib END } add_custom_spec_section($spec, "post_prep"); print $spec <<END; \%build END my $makefile_env = ''; if (grep { $_ eq 'inc/Module/Install.pm' } @archive_files) { # Since perl 5.26, . was removed from @INC, but several modules # do "use inc::Module::Install" in Makefile.PL $makefile_env = "PERL_USE_UNSAFE_INC=1 "; } if ($config->{custom_build}) { print $spec $config->{custom_build} . "\n"; } else { if ($usebuildpl) { print $spec <<END; $makefile_env$cmdperl Build.PL --installdirs=vendor@{[$noarch ? '' : qq{ optimize="$macro{optimize}"} ]} ./Build build --flags=\%{?_smp_mflags} END } else { print $spec <<END; $makefile_env$cmdperl Makefile.PL INSTALLDIRS=vendor@{[$noarch ? '' : qq{ OPTIMIZE="$macro{optimize}"}]} END print $spec "$cmdperl -pi -e 's/^\\tLD_RUN_PATH=[^\\s]+\\s*/\\t/' Makefile\n" if ($compat and !$noarch); print $spec <<END; \%make_build END } } add_custom_spec_section($spec, "post_build"); print $spec <<END; \%check@{[($compat ? ' || :' : '')]} END if ($config->{custom_test}) { print $spec $config->{custom_test} . "\n"; } else { my $noprefix = ''; if ($config->{no_testing}) { print $spec "# MANUAL no testing ($config->{no_testing})\n"; $noprefix = '#'; } if ($usebuildpl) { print $spec "$noprefix./Build test\n"; } else { print $spec "$noprefix$cmdmake test\n"; } } print $spec <<END; \%install END if ($usebuildpl) { print $spec "./Build install --destdir=$macro{buildroot} --create_packlist=0\n"; } else { print $spec <<END; %perl_make_install %perl_process_packlist END } add_custom_spec_section($spec, "post_install"); print $spec "%perl_gen_filelist\n"; if ($addlicense and !grep /copying|artistic|copyright|license/i, @doc) { print $spec <<END; perldoc -t perlgpl > COPYING perldoc -t perlartistic > Artistic END $hdoc{"COPYING"} = 1; $hdoc{"Artistic"} = 1; } if (@filter_requires || @filter_provides) { print $spec <<END; \%clean $cmdrm -rf $macro{buildroot}@{[ (@filter_requires ? ' %{__perl_requires}' : '') . (@filter_provides ? ' %{__perl_provides}' : '')]} END } else { print $spec "\n"; } print $spec "\%files -f \%{name}.files\n"; if (%hdoc) { my (@ldoc, @hdoc); for my $f (keys %hdoc) { if (is_license_file($f)) { push(@ldoc, $f); } else { push(@hdoc, $f); } } if (@hdoc) { print $spec "%doc " . join(' ', sort(@hdoc)) . "\n"; } if (@ldoc) { print $spec "%license " . join(' ', sort(@ldoc)) . "\n"; } } if ($scripts) { print $spec "\%{_bindir}/*\n"; # FIXME - How do we auto-detect man pages? } print $spec $config->{"misc"} if $config->{"misc"}; print $spec <<END; \%changelog END $spec->close(); my ($fh, $filename) = File::Temp::tempfile; if (-x "/usr/lib/obs/service/format_spec_file.files/prepare_spec") { if (!system("/usr/lib/obs/service/format_spec_file.files/prepare_spec '$specfile' > '$filename'")) { # don't want to reimplement cross-device rename system("mv '$filename' '$specfile'"); } } else { print STDERR "please install obs-service-format_spec_file\n"; } build_rpm($specfile) if ($buildsrpm or $buildrpm); push(@processed, $module); if (!$skip_changes) { (my $basename = $specfile) =~ s{\.spec$}{}; my $changelogfile = "$basename.changes"; my $skip = 0; my $changes_diff; my ($tfh, $tmpfile) = File::Temp::tempfile; binmode $tfh, ':encoding(UTF-8)'; my $cltxt = ""; if (-f $changelogfile) { my $txt = "- updated to $version\n see $defaultdocdir/$basename/$changesfile\n\n"; $cltxt .= $txt; if ($old_file && $changes) { my $old_changes = extract_old_changes($old_file, $changesfile); $old_changes = decode_latin_or_utf8($old_changes); $old_changes =~ s,\r\n,\n,g; $cltxt .= diff_changes($old_changes, $changes) if $old_changes; } local $/; undef $/; if(open CHANGES,"<",$changelogfile) { my $log = <CHANGES>; if($log =~ $txt) { $skip = 1; print STDERR "Skip changelog entry, already exists.\n"; } close CHANGES; } } else { $cltxt .= "- initial package $version\n * created by $NAME $VERSION"; } $cltxt .= "\n"; # ensure line termination $cltxt =~ s/\n+$/\n/; # drop multi-lines at the end, osc adds one again print $tfh $cltxt; system("osc vc -F $tmpfile $basename.changes") if !$skip; die "osc vc failed with $?" if $?; close($tfh); } } sub prereqs_from_metayaml { my ($meta) = @_; my (%provides, %build_requires, %requires, %recommends); if ($meta->{provides}) { for my $pkg (keys %{ $meta->{provides} || {} }) { $provides{ $pkg } = 1; } } %build_requires = %{$meta->{build_requires}} if ($meta->{build_requires}); if ($meta->{configure_requires}) { while (my ($key, $value) = each(%{$meta->{configure_requires}})) { if (defined $build_requires{$key}) { next if version->parse($build_requires{$key}) > version->parse($value); } $build_requires{$key} = $value; } } if ($meta->{test_requires}) { while (my ($key, $value) = each(%{$meta->{test_requires}})) { $build_requires{$key} = $value; } } %requires = %{$meta->{requires}} if ($meta->{requires}); %recommends = %{$meta->{recommends}} if ($meta->{recommends}); return (\%provides) if (not %build_requires and not %requires and not %recommends); return (\%provides, \%build_requires, \%requires, \%recommends); } sub prereqs_from_metajson { my ($metajson) = @_; my (%provides, %build, %run, %rec); if ($metajson->{provides}) { for my $pkg (keys %{ $metajson->{provides} || {} }) { $provides{ $pkg } = 1; } } my $prereqs = $metajson->{prereqs}; return (\%provides) unless keys %$prereqs; for my $phase (qw/ build configure test /) { if (my $build = $prereqs->{ $phase }) { my $req = $build->{requires} || {}; for my $module (sort keys %$req) { next if exists $build{ $module } and version->parse($build{ $module }) > version->parse( $req->{ $module }); $build{ $module } = $req->{ $module }; } } } for my $phase (qw/ runtime /) { if (my $build = $prereqs->{ $phase }) { my $req = $build->{requires} || {}; @run{ keys %$req } = values %$req; my $rec = $build->{recommends} || {}; @rec{ keys %$rec } = values %$rec; } } return (\%provides, \%build, \%run, \%rec); } sub parse_provides { my ($path, $files) = @_; my %provides; my $scanner = Perl::PrereqScanner->new; foreach my $test (grep /\.(pm|t|PL|pl)/, @$files) { my $doc = PPI::Document->new($path . "/" . $test); next unless ($doc); # Get the name of the main package my $pkg = $doc->find_first('PPI::Statement::Package'); if ($pkg) { $provides{$pkg->namespace} = 1; } } return %provides; } sub decode_latin_or_utf8 { my ($string) = @_; my $enc = guess_encoding($string, qw/ utf8 latin1 /); unless (ref $enc) { return decode_utf8 $string; } #my $name = $enc->name; $string = $enc->decode($string); return $string; } sub read_meta_json { $debug and warn __PACKAGE__.':'.__LINE__.": =============== read_meta_json\n"; my ($file, $stats) = @_; my %results = ( dynamic => 1 ); open my $fh, '<', $file or die $!; my $json = do { local $/; <$fh> }; close $fh; my $metajson = eval { $coder->decode($json) }; if ($@) { warn "Error decoding META.json, ignoring ($@)"; $stats->{metajson} = 'error'; return; } $stats->{metajson} = 1; if (exists $metajson->{dynamic_config} and not $metajson->{dynamic_config}) { $results{dynamic} = 0; } my ($prov, $build, $run, $rec) = prereqs_from_metajson($metajson); if (keys %$prov) { @{ $results{provides} }{keys %$prov } = values %$prov; } if ($build) { $results{build} = 1; $results{build_requires} = $build; $results{requires} = $run; $results{recommends} = $rec; $results{got_prereqs} = 1; if ($debug) { warn __PACKAGE__.':'.__LINE__.$".Data::Dumper->Dump([\%results], ['results']); } } $results{license} = $metajson->{license}; if ($metajson->{abstract} && $metajson->{abstract} ne 'unknown') { my $abstract = $metajson->{abstract}; $stats->{abstract}->{metajson} = $abstract; $results{abstract} = $abstract; } $stats->{license}->{metajson} = $metajson->{license}; if ($metajson->{abstract}) { $stats->{abstract}->{metajson} = $metajson->{abstract}; } return \%results; } sub read_meta_yaml { $debug and warn __PACKAGE__.':'.__LINE__.": =============== read_meta_yaml\n"; my ($file, $stats) = @_; my $yml = readfile($file); my $meta = eval { YAML::XS::Load($yml); }; if ($@) { warn "Error parsing $file: $@"; $stats->{metayaml} = 'error'; return; } my %results = ( dynamic => 1 ); $stats->{metayaml} = 1; if (exists $meta->{dynamic_config} and not $meta->{dynamic_config}) { $results{dynamic} = 0; } if ($meta->{abstract} && $meta->{abstract} ne 'unknown') { my $abstract = $meta->{abstract}; $stats->{abstract}->{metayaml} = $abstract; $results{abstract} = $abstract; } my ($prov, $build, $run, $rec) = prereqs_from_metayaml($meta); if (keys %$prov) { @{ $results{provides} }{keys %$prov } = values %$prov; } if ($build) { $results{build} = 1; $results{build_requires} = $build; $results{requires} = $run; $results{recommends} = $rec; $results{got_prereqs} = 1; if ($debug) { warn __PACKAGE__.':'.__LINE__.$".Data::Dumper->Dump([\%results], ['results']); } } $stats->{got_prereqs} = $results{got_prereqs}; # did we get static dependencies? # FIXME - I'm not sure this is sufficient... if ($meta->{script_files} or $meta->{scripts}) { $results{scripts} = 1; } my $license; $stats->{license}->{metayaml} = $meta->{license}; if ($meta->{license}) { # This list of licenses is from the Module::Build::API # docs, cross referenced with the list of licenses in # /usr/share/rpmlint/config. if ($meta->{license} =~ /^perl$/i) { $license = $perllicense; } elsif ($meta->{license} =~ /^apache$/i) { $license = "Apache-2.0"; } elsif ($meta->{license} =~ /^artistic$/i) { $license = "Artistic-1.0"; } elsif ($meta->{license} =~ /^artistic_?2$/i) { $license = "Artistic-2.0"; } elsif ($meta->{license} =~ /^bsd$/i) { $license = "BSD-3-Clause"; } elsif ($meta->{license} =~ /^gpl$/i) { $license = "GPL-1.0-or-later"; } elsif ($meta->{license} =~ /^gpl2$/i) { $license = "GPL-2.0-or-later"; } elsif ($meta->{license} =~ /^lgpl$/i) { $license = "LGPL-2.1-or-later"; } elsif ($meta->{license} =~ /^mit$/i) { $license = "MIT"; } elsif ($meta->{license} =~ /^mozilla$/i) { $license = "MPL"; } elsif ($meta->{license} =~ /^gpl3$/i) { $license = "GPL-3.0-or-later"; } elsif ($meta->{license} =~ /^open_source$/i || $meta->{license} =~ /^unrestricted$/i) { $license = "SUSE-Public-Domain"; # rpmlint will complain } elsif ($meta->{license} =~ /^restrictive$/i) { $license = "SUSE-NonFree"; warn "License is 'restrictive'." . " This package should not be redistributed.\n"; } elsif ($meta->{license} =~ /^unknown$/i) { # do nothing, it's unknown and we know } else { warn "Unknown license in meta '" . $meta->{license} . "'!\n"; } } $results{license} = $license; return \%results; } sub find_doc { my ($config, $path, @files) = @_; my @skipdoc = split ' ', ($config->{skip_doc} || ''); my $doskip = sub { for my $s (@skipdoc) { return 1 if m/$s/ } return 0; }; my @doc = sort { $a cmp $b } grep { !m{/} and !/\.(pl|xs|h|c|pm|ini?|pod|cfg|inl|bak|spec|yml|toml)$/i and !/^\./ and !/~$/ and $_ ne $path and $_ ne "MANIFEST" and $_ ne "MANIFEST.SKIP" and $_ ne "SIGNATURE" and $_ ne "NINJA" and $_ ne "c" and $_ ne "configure" and $_ ne "config.guess" and $_ ne "config.sub" and $_ ne "typemap" and $_ ne "bin" and $_ ne "lib" and $_ ne "t" and $_ ne "xt" and $_ ne "inc" and $_ ne "dist.ini.meta" and $_ ne "autobuild.sh" and $_ ne "debian" and $_ ne "cpanfile" and $_ ne "pm_to_blib" and $_ ne "install.sh" and !/^INSTALL/i and !/^META\..+$/i and !/^MYMETA\..+$/i and !/^perlcritic/ and !/^perltidy/ and !/\.tar\./ and !/~$/ and !$doskip->($_) } @files; push @doc, split ' ', ($config->{add_doc} || ''); # special subdir push(@doc, "examples") if grep(/^examples\//, @files); push(@doc, "doc") if grep(/^doc\//, @files); push(@doc, "docs") if grep(/^docs\//, @files); push(@doc, "util") if grep(/^util\//, @files); push(@doc, "example") if grep(/^example\//, @files); push(@doc, "samples") if grep(/^samples\//, @files); push(@doc, "license") if grep(/^license\//, @files); return @doc; } sub dump_statistics { my ($module, $version, $stats) = @_; $stats->{name} = $module; $stats->{version} = $version; my $yaml = YAML::XS::Dump($stats); $yaml =~ s/^/# STATS # /mg; $yaml = "# STATS # # $module, $version\n$yaml"; verbose($yaml); } sub filter_requires { my ($name) = @_; my $script="$name-filter-requires.sh"; verbose "Writing $script..."; my $sh; if ($force) { rename($script, "$script~") if (-e $script); $sh=new FileHandle ">$script"; } else { $sh=new FileHandle $script, O_WRONLY|O_CREAT|O_EXCL; } die "Failed to create $script: $!\n" if (!$sh); print $sh "#!/bin/sh\n\n" . "\@\@PERL_REQ\@\@ \"\$\@\" | sed -e '/^$filter_requires[0]\$/d'"; if (@filter_requires > 1) { for my $dep (@filter_requires[1..$#filter_requires]) { print $sh " \\\n -e '/^$dep\$/d'"; } } print $sh "\n"; } sub filter_provides { my ($name) = @_; my $script = "$name-filter-provides.sh"; verbose "Writing $script..."; my $sh; if ($force) { rename($script, "$script~") if (-e $script); $sh = new FileHandle ">$script"; } else { $sh = new FileHandle $script, O_WRONLY | O_CREAT | O_EXCL; } die "Failed to create $script: $!\n" if (!$sh); print $sh "#!/bin/sh\n\n" . "\@\@PERL_PROV\@\@ \"\$\@\" | sed -e '/^$filter_provides[0]\$/d'"; if (@filter_provides > 1) { for my $dep (@filter_provides[1 .. $#filter_provides]) { print $sh " \\\n -e '/^$dep\$/d'"; } } print $sh "\n"; } # vi: set ai et: 0707010000001E000081A4000000000000000000000001663D00E30000039B000000000000000000000000000000000000001600000000cpanspec/cpanspec.yml--- #description_paragraphs: 3 #description: |- # override description from CPAN #summary: override summary from CPAN #no_testing: broken upstream #sources: # - source1 # - source2 #patches: # foo.patch: -p1 # bar.patch: # baz.patch: PATCH-FIX-OPENSUSE #preamble: |- # BuildRequires: gcc-c++ #post_prep: |- # hunspell=`pkg-config --libs hunspell | sed -e 's,-l,,; s, *,,g'` # sed -i -e "s,hunspell-X,$hunspell," t/00-prereq.t Makefile.PL #post_build: |- # rm unused.files #post_install: |- # sed on %{name}.files #license: SUSE-NonFree #skip_noarch: 1 #custom_build: |- #./Build build flags=%{?_smp_mflags} --myflag #custom_test: |- #startserver && make test #ignore_requires: Bizarre::Module #skip_doc: regexp_to_skip_for_doc.* #add_doc: files to add to docs #misc: |- #anything else to be added to spec file #follows directly after %files section, so it can contain new blocks or also #changes to %files section 0707010000001F000041ED000000000000000000000002663D00E300000000000000000000000000000000000000000000000D00000000cpanspec/lib07070100000020000081A4000000000000000000000001663D00E300005FD5000000000000000000000000000000000000001900000000cpanspec/lib/CPAN2OBS.pmpackage CPAN2OBS; use strict; use warnings; use 5.010; use base 'Exporter'; our @EXPORT_OK = qw/ debug info prompt /; use Data::Dumper; use Term::ANSIColor qw/ colored /; use YAML::XS qw/ DumpFile /; use XML::Simple qw/ XMLin /; use Storable qw/ retrieve store /; use File::Basename qw/ basename /; use File::Copy qw/ copy move /; use File::Path qw/ remove_tree /; use File::Spec; use Parse::CPAN::Packages; use File::Glob qw/ bsd_glob /; use FindBin '$Bin'; use autodie qw/ mkdir chdir open close unlink /; use Moo; use namespace::clean; has data => ( is => 'rw', coerce => \&_coerce_data ); has cpandetails => ( is => 'ro' ); has skip => ( is => 'ro' ); has apiurl => ( is => 'ro' ); has cpanmirror => ( is => 'ro' ); has cpanspec => ( is => 'ro' ); has project_prefix => ( is => 'ro', coerce => \&_coerce_project_prefix ); has locked => ( is => 'rw' ); sub _coerce_data { my ($data) = @_; unless (File::Spec->file_name_is_absolute($data)) { $data = File::Spec->rel2abs($data); } return $data; } sub _coerce_project_prefix { my ($prefix) = @_; unless ($prefix =~ m/\A[A-Za-z][A-Za-z:-]+\z/) { die "Invalid project prefix '$prefix'"; } return $prefix; } sub debug { my ($msg) = @_; say STDERR colored([qw/ grey15 /], $msg); } sub info { my ($msg) = @_; say colored([qw/ magenta /], $msg); } sub prompt { my ($msg) = @_; print colored([qw/ cyan /], $msg); chomp(my $answer = <STDIN>); $answer = uc $answer; return $answer; } sub fetch_status { my ($self, $letter) = @_; my $data = $self->data; my $status_file = "$data/status/$letter.tsv"; my %states; unless (-e $status_file) { return \%states; } $self->lockdata; open my $fh, '<', $status_file; while (my $line = <$fh>) { chomp $line; my ($dist, $status, $version, $url, $obs_tar, $obs_ok) = split /\t/, $line; $states{ $dist } = [ $status, $version, $url, $obs_tar, $obs_ok ]; } close $fh; $self->unlockdata; return \%states; } sub fetch_status_perl { my ($self, $letter) = @_; my $data = $self->data; my $status_file = "$data/status/perl.tsv"; my %states; unless (-e $status_file) { return \%states; } $self->lockdata; open my $fh, '<', $status_file; while (my $line = <$fh>) { chomp $line; my ($dist, $status, $version, $url, $obs_version) = split /\t/, $line; $states{ $dist } = [ $status, $version, $url, $obs_version ]; } close $fh; $self->unlockdata; return \%states; } sub write_status { my ($self, $letter, $states) = @_; my $data = $self->data; mkdir "$data/status" unless -d "$data/status"; my $status_file = "$data/status/$letter.tsv"; open my $fh, '>', $status_file; for my $dist (sort keys %$states) { my $status = $states->{ $dist }; say $fh join "\t", $dist, @$status; } close $fh; } sub fetch_cpan_list { my ($self) = @_; $self->lockdata; my $data = $self->data; my $cpan_modules_dir = "$data/cpan"; my $cpan_stats = "$cpan_modules_dir/stats.yaml"; my $details = "$data/02packages.details.txt.gz"; my $details_url = $self->cpandetails; my $details_mtime = -e $details ? (stat $details)[9] : 0; my $stats_mtime = -e $cpan_stats ? (stat $cpan_stats)[9] : 0; if ($details_mtime + 60 * 30 < time) { my $cmd = "wget -q $details_url -O $details"; debug("CMD $cmd"); system $cmd; $details_mtime = -e $details ? (stat $details)[9] : 0; } else { info("$details uptodate"); } if ($stats_mtime >= $details_mtime) { info("$cpan_stats uptodate"); $self->unlockdata; return 1; } info("Parsing $details"); my $p = Parse::CPAN::Packages->new($details); my %seen; my %upstream; for my $m ($p->packages) { $m = $m->distribution; my $url = $m->prefix; my $dist = $m->dist; unless ($dist) { next; } if ($self->skip->{ $dist }) { info("Ignoring $dist"); next; } my $version = $m->version; if (not $version) { #warn sprintf "Distribution %s has no version defined", $url; next; } my $first = uc substr $dist, 0, 1; if ($dist eq 'XML-Xerces') { # versions like '2.7.0-1' are not parseable. # XML-Xerces is in the module index multiple times with different # versions and that leads to a different version almost every day $version =~ s/-//g; } my $v = eval { version->parse($version); }; $v ||= version->declare('0'); if (not exists $seen{ $dist } or ($seen{ $dist } ) < $v) { $seen{ $dist } = $v; $upstream{ $first }->{ $dist } = [$v, $url]; } } my %stats; my $total = 0; for my $letter ('A' .. 'Z') { my $count = keys %{ $upstream{ $letter } }; $total += $count; $stats{ $letter } = $count; $self->hash_to_cpan_file($letter, $upstream{ $letter }); } $stats{total} = $total; DumpFile($cpan_stats, \%stats); system "cat $cpan_stats"; $self->unlockdata; } sub hash_to_cpan_file { my ($self, $letter, $upstream) = @_; my $data = $self->data; my $cpan_modules_dir = "$data/cpan"; mkdir $cpan_modules_dir unless -d $cpan_modules_dir; my $cpan_modules_file = "$cpan_modules_dir/$letter.tsv"; open my $fh, '>', $cpan_modules_file; for my $dist (sort keys %$upstream) { my $info = $upstream->{ $dist }; my ($version, $url) = @$info; say $fh join "\t", ($dist, $version, $url); } close $fh; } sub from_cpan_file { my ($self, $letter) = @_; my $data = $self->data; my $cpan_modules_dir = "$data/cpan"; mkdir $cpan_modules_dir unless -d $cpan_modules_dir; my $cpan_modules_file = "$cpan_modules_dir/$letter.tsv"; open my $fh, '<', $cpan_modules_file; my %upstream; while (my $line = <$fh>) { chomp $line; my ($dist, $version, $url) = split m/\t/, $line; $upstream{ $dist} = [ $version, $url ]; } close $fh; return \%upstream; } sub fetch_obs_info { my ($self, $letter) = @_; my $data = $self->data; my $apiurl = $self->apiurl; my $project_prefix = $self->project_prefix . $letter; my $cache = $self->fetch_obs_cache($letter); my $obsdir = "$data/obs"; my $letter_xml = "$obsdir/CPAN-$letter.xml"; mkdir $obsdir unless -d $obsdir; my %obs_info; my $old =(not -e $letter_xml or ((stat $letter_xml)[9] + 60 * 60) < time); if (1) { my $cmd = "osc -A $apiurl api '/source/$project_prefix?view=info' >$letter_xml"; debug("CMD $cmd"); system $cmd; } if (not -s $letter_xml) { return \%obs_info; } my $info = XMLin($letter_xml)->{sourceinfo}; return \%obs_info unless $info; $info = [$info] unless ref $info eq 'ARRAY'; mkdir "$data/project-xml" unless -d "$data/project-xml"; for my $pi (@$info) { my $srcmd5 = $pi->{srcmd5}; my $package = $pi->{package}; my $cached = $cache->{ $srcmd5 }; if (not defined $cached) { info("fetch obs xml for $package"); my $cmd = sprintf "osc -A %s api /source/%s/%s >$data/project-xml/$package.xml", $apiurl, $project_prefix, $package; debug("CMD $cmd"); system $cmd; unless (-s "$data/project-xml/$package.xml") { warn __PACKAGE__.':'.__LINE__.": !!!! no osc data for $package\n"; next; } my $pxml = XMLin "$data/project-xml/$package.xml"; my $name = $pxml->{name}; $obs_info{ $name }->{ok} = 1; if ($pxml->{entry}->{'cpanspec.error'}) { $obs_info{ $name }->{ok} = 0; } for my $entry (keys %{ $pxml->{entry} }) { if ($entry =~ m/\.tar/ || $entry =~ m/\.tgz$/ || $entry =~ m/\.zip$/) { $obs_info{ $name }->{archive} = $entry; } } $cache->{ $srcmd5 } = $obs_info{ $package } || {}; $self->store_obs_cache($letter, $cache); } else { if (ref $cached) { $obs_info{ $package } = $cached; } else { $obs_info{ $package } = { archive => $cached, ok => 1 }; } } } return \%obs_info; } sub fetch_obs_cache { my ($self, $letter) = @_; my $data = $self->data; mkdir "$data/obs-cache" unless -d "$data/obs-cache"; my $cachefile = "$data/obs-cache/$letter"; my $cache = {}; if (-f $cachefile) { eval { $cache = retrieve($cachefile); }; if ($@) { warn "Problem loading $cachefile: $@"; } } return $cache; } sub store_obs_cache { my ($self, $letter, $cache) = @_; my $data = $self->data; mkdir "$data/obs-cache" unless -d "$data/obs-cache"; my $cachefile = "$data/obs-cache/$letter"; store $cache, $cachefile; } sub create_package_xml { my ($self, $pkg, $spec) = @_; my $xmlfile = "tmp-package.xml"; my $xml = qq{<package name='$pkg'><title/><description/>}; if (-f $spec) { my $noarch; open my $fh, '<', $spec; while (<$fh>) { $noarch = 1, last if m/^BuildArch.*noarch/; } close $fh; if ($noarch) { $xml .= q{<build><disable arch='i586'/></build>}; } } else { $xml .= q{<build><disable/></build>}; } $xml .= qq{</package>\n}; { open my $fh, '>', $xmlfile; print $fh $xml; close $fh; } return $xmlfile; } sub update_obs { my ($self, $letter, $args) = @_; my $packages = $args->{packages}; $self->lockdata; my $data = $self->data; my $osc = "$data/osc"; my $dir = "$osc/$letter"; # removing previous osc checkouts remove_tree $dir, { verbose => 0, safe => 1 }; my $states = $self->fetch_status($letter); my @keys = sort keys %$states; if ($packages and @$packages) { info("Requested (@$packages)"); @keys = @$packages; } my $max = $args->{max}; my $counter = 0; for my $dist (@keys) { my $dist_status = $states->{ $dist }; unless ($dist_status) { info("No status found for '$dist'"); next; } my ($status, $version, $url) = @$dist_status; unless ($args->{redo}) { next if ($status eq 'done'); } if ($status =~ m/^error/) { info("Skip $dist ($status)"); next; } $counter++; last if $counter > $max; info("($counter) updating $dist (@$dist_status)"); my $answer = 'Y'; if ($args->{ask}) { $answer = prompt("Update $dist? [y/N/q] ") || 'N'; last if $answer eq 'Q'; next if $answer eq 'N'; info("y/n/q") if $answer ne 'Y'; } if ($answer eq 'Y') { eval { $self->osc_update_dist($letter, $dist, $dist_status, $args); }; if (my $err = $@) { debug("ERROR: $dist $err"); $states->{ $dist }->[0] = 'error'; info("updating states ($letter)"); $self->write_status($letter, $states); } } } $self->unlockdata; } sub update_obs_perl { my ($self, $args) = @_; my $packages = $args->{packages}; $self->lockdata; my $project_prefix = $self->project_prefix; my $data = $self->data; my $apiurl = $self->apiurl; my $auto_projects = "$data/auto.xml"; my $cmd = sprintf "osc -A $apiurl api /source/%s > %s", $project_prefix, $auto_projects; debug("CMD $cmd"); system $cmd; my $existing = XMLin($auto_projects)->{entry}; my $osc = "$data/osc"; my $dir = "$osc/perl"; # removing previous osc checkouts remove_tree $dir, { verbose => 0, safe => 1 }; my $states = $self->fetch_status_perl(); my @keys = sort keys %$states; if ($packages and @$packages) { info("Requested (@$packages)"); @keys = @$packages; } my $max = $args->{max}; my $counter = 0; for my $dist (@keys) { my %args = %$args; my $dist_status = $states->{ $dist }; unless ($dist_status) { info("No status found for '$dist'"); next; } my ($status, $version, $url) = @$dist_status; unless ($args{redo}) { next if ($status eq 'done' or $status eq 'older'); } if ($status =~ m/^error/) { info("Skip $dist ($status)"); next; } my $pkg = "perl-$dist"; if ($existing->{ $pkg }) { $args{exists} = 1; my $tar = basename $url; my $pxml = "/tmp/$pkg-autoupdate.xml"; my $cmd = sprintf "osc -A $apiurl api /source/%s/%s >%s", $project_prefix, $pkg, $pxml; system $cmd and die "Error ($cmd): $?"; my $info = XMLin($pxml); unlink $pxml; if ($info->{entry}->{ $tar }) { debug("$tar already exists, skipping"); next; } } $counter++; last if $counter > $max; info("($counter) updating $dist (@$dist_status)"); my $answer = 'Y'; if ($args{ask}) { $answer = prompt("Update $dist? [y/N/q] ") || 'N'; last if $answer eq 'Q'; next if $answer eq 'N'; info("y/n/q") if $answer ne 'Y'; } if ($answer eq 'Y') { eval { $self->osc_update_dist_perl($dist, $dist_status, \%args); }; my $err = $@; if ($err) { debug("ERROR: $dist $err"); $states->{ $dist }->[0] = 'error'; info("updating states (perl)"); $self->write_status(perl => $states); } } } $self->unlockdata; } sub osc_update_dist { my ($self, $letter, $dist, $todo, $args) = @_; my $data = $self->data; my $apiurl = $self->apiurl; my $mirror = $self->cpanmirror; my $cpanspec = $self->cpanspec; my $project_prefix = $self->project_prefix . $letter; my $osc = "$data/osc"; mkdir $osc unless -d $osc; my $dir = "$osc/$letter"; mkdir $dir unless -d $dir; chdir $dir; debug("osc_update_dist($dist)"); my ($status, $version, $url, $obs_tar, $obs_status) = @$todo; my $pkg = "perl-$dist"; my $spec = "$pkg.spec"; my $tar = basename $url; { my $cmd = "wget --tries 5 --timeout 30 --connect-timeout 30 -nc -q $mirror/authors/id/$url -O $tar -o /dev/null"; debug("CMD $cmd"); system $cmd; if ($? or not -f $tar) { info("Error fetching $url, skip ('$cmd': $?)"); return 0; } } my $error = 1; { my $cmd = sprintf "timeout 180 perl $cpanspec -v -f --pkgdetails %s --skip-changes %s > cpanspec.error 2>&1", "$data/02packages.details.txt.gz", $tar; debug("CMD $cmd"); if (system $cmd or not -f $spec) { info("Error executing cpanspec"); system("cat cpanspec.error"); } else { system("cat cpanspec.error"); $error = 0; unlink "cpanspec.error"; } } my $xmlfile = $self->create_package_xml($pkg, $spec); my $project = "$project_prefix/$pkg"; { my $cmd = sprintf "osc -A %s meta pkg %s %s -F %s", $apiurl, $project_prefix, $pkg, $xmlfile; debug("CMD $cmd"); system $cmd and die "Error executing '$cmd': $?"; } my $checkout = "$dir/$project_prefix/$pkg"; if (-e $checkout) { debug("REMOVE $checkout"); remove_tree $checkout, { verbose => 0, safe => 1 }; } { my $cmd = sprintf "osc -A %s co %s/%s", $apiurl, $project_prefix, $pkg; system $cmd and die "Error executing '$cmd': $?"; } chdir $checkout; if ($obs_tar) { my $cmd = "[[ -e $obs_tar ]] && rm $obs_tar || true"; debug("CMD $cmd"); system $cmd and die "Error executig '$cmd': $?"; } { my $cmd = "[[ -e cpanspec.error ]] && rm cpanspec.error || true"; debug("CMD $cmd"); system $cmd and die "Error executig '$cmd': $?"; $cmd = "[[ -e $spec ]] && rm $spec || true"; debug("CMD $cmd"); system $cmd and die "Error executig '$cmd': $?"; } if ($error) { move "../../cpanspec.error", $checkout or die $!; } else { move "../../$spec", $checkout or die $!; } move "../../$tar", $checkout or die $!; { my $cmd = "osc addremove"; debug("CMD $cmd"); system $cmd and die "Error executig '$cmd': $?"; if ($args->{ask_commit}) { my $answer = prompt("Commit? [Y/n]") || 'Y'; if ($answer ne 'Y') { info("$pkg - no commit"); return 1; } } $cmd = "osc ci -mupdate"; debug("CMD $cmd"); system $cmd and die "Error executig '$cmd': $?"; } unlink "$dir/$xmlfile"; } sub osc_update_dist_perl { my ($self, $dist, $todo, $args) = @_; my $exists = $args->{exists}; my $data = $self->data; my $apiurl = $self->apiurl; my $mirror = $self->cpanmirror; my $cpanspec = $self->cpanspec; my $project_prefix = $self->project_prefix; my $letter = 'perl'; my $osc = "$data/osc"; mkdir $osc unless -d $osc; my $dir = "$osc/$letter"; mkdir $dir unless -d $dir; chdir $dir; debug("osc_update_dist($dist)"); my ($status, $version, $url, $obs_tar, $obs_status) = @$todo; my $pkg = "perl-$dist"; my $spec = "$pkg.spec"; my $tar = basename $url; if ($exists) { my $cmd = sprintf "osc -A $apiurl rdelete -mrecreate -f %s %s", $project_prefix, $pkg; debug("CMD $cmd"); system $cmd and die "Error ($cmd): $?"; } { my $cmd = sprintf "osc -A $apiurl branch devel:languages:perl %s %s", $pkg, $project_prefix; debug("CMD $cmd"); system $cmd and die "Error ($cmd): $?"; } { my $cmd = sprintf "wget --tries 5 --timeout 30 --connect-timeout 30 -nc -q %s -O $dir/$tar -o /dev/null", "$mirror/authors/id/$url"; debug("CMD $cmd"); system $cmd; if ($? or not -f "$dir/$tar") { info("Error fetching $url, skip ('$cmd': $?)"); return 0; } } my $checkout = "$dir/$project_prefix/$pkg"; if (-e $checkout) { debug("REMOVE $checkout"); remove_tree $checkout, { verbose => 0, safe => 1 }; } { my $cmd = sprintf "osc -A %s co %s/%s", $apiurl, $project_prefix, $pkg; system $cmd and die "Error executing '$cmd': $?"; } chdir $checkout; my $old_tar = ''; for my $tar (bsd_glob("{$dist-*.tar*,$dist-*.tgz,$dist-*.zip}")) { $old_tar = $tar; unlink $tar; } move "$dir/$tar", $checkout or die $!; my $error = 1; copy("$Bin/../cpanspec.yml", "$checkout/cpanspec.yml") unless -f "cpanspec.yml"; { my $cmd = sprintf "timeout 900 perl $cpanspec -v -f --pkgdetails %s --old-file %s %s > cpanspec.error 2>&1", "$data/02packages.details.txt.gz", ".osc/$old_tar", $tar; debug("CMD $cmd"); if (system $cmd or not -f $spec) { system("cat cpanspec.error"); info("Error executing cpanspec"); } else { system("cat cpanspec.error"); $error = 0; unlink "cpanspec.error"; } } { my $cmd = "osc addremove"; debug("CMD $cmd"); system $cmd and die "Error executig '$cmd': $?"; if ($args->{ask_commit}) { my $answer = prompt("Commit? [Y/n]") || 'Y'; if ($answer ne 'Y') { info("$pkg - no commit"); return 1; } } $cmd = "osc ci -mupdate"; debug("CMD $cmd"); system $cmd and die "Error executig '$cmd': $?"; } } sub update_status { my ($self, $letter) = @_; $self->lockdata; my $states = $self->fetch_status($letter); my $upstream = $self->from_cpan_file($letter); my $obs_info = $self->fetch_obs_info($letter); for my $dist (sort keys %$upstream) { my $pkg = "perl-$dist"; my $info = $upstream->{ $dist }; my $dist_status = $states->{ $dist }; my $obs_ok = $obs_info->{ $pkg }->{ok} // 0; my $obs_tar = $obs_info->{ $pkg }->{archive} || ''; my ($upstream_version, $upstream_url) = @$info; my $upstream_tar = basename $upstream_url; my $status = 'new'; if ($dist_status) { $status = $dist_status->[0]; if ($status =~ m/^error/) { } elsif (not $obs_tar) { $status = 'new'; } elsif ($obs_tar eq $upstream_tar) { $status = 'done'; } else { $status = 'todo'; } } else { if (not $obs_tar) { $status = 'new'; } elsif ($obs_tar eq $upstream_tar) { $status = 'done'; } else { $status = 'todo'; } } $dist_status = [ $status, $upstream_version, $upstream_url, $obs_tar, $obs_ok, ]; $states->{ $dist } = $dist_status; } $self->write_status($letter, $states); $self->unlockdata; } sub update_status_perl { my ($self) = @_; $self->lockdata; my $apiurl = $self->apiurl; my $data = $self->data; my $perl_projects = "$data/perl.xml"; my $cmd = "osc -A $apiurl api /status/project/devel:languages:perl > $perl_projects"; debug("CMD $cmd"); system $cmd; my $existing = XMLin($perl_projects)->{package}; my $states = $self->fetch_status_perl(); my $upstream = {}; for my $letter ('A' .. 'Z') { my $up = $self->from_cpan_file($letter); %$upstream = ( %$upstream, %$up ); } for my $dist (sort keys %$upstream) { my $pkg = "perl-$dist"; next unless defined $existing->{ $pkg }; my $ex = $existing->{ $pkg }; my $ex_version = $ex->{version}; my $ex_version_normal = eval { version->parse($ex_version || 0) }; next unless $ex_version_normal; my $info = $upstream->{ $dist }; my $dist_status = $states->{ $dist }; my ($upstream_version, $upstream_url) = @$info; my $upstream_version_normal = eval { version->parse($upstream_version) }; my $upstream_tar = basename $upstream_url; my $status = 'new'; if ($dist_status) { $status = $dist_status->[0]; } if ($status =~ m/^error/) { } elsif (not $ex_version) { $status = 'new'; } elsif ($ex_version_normal == $upstream_version_normal) { $status = 'done'; } elsif ($ex_version_normal > $upstream_version_normal) { $status = 'older'; } else { $status = 'todo'; } $dist_status = [ $status, $upstream_version, $upstream_url, $ex_version_normal, ]; $states->{ $dist } = $dist_status; } $self->write_status('perl', $states); $self->unlockdata; } sub lockdata { my ($self) = @_; my $data = $self->data; mkdir $data unless -d $data; return 1 if $self->locked; my $lockfile = "$data/lockfile"; my $cmd = "lockfile -1 -r 5 $lockfile"; system $cmd and die "Error using lockfile: $?. Is another process running?"; $self->locked(1); } sub unlockdata { my ($self) = @_; return 1 unless $self->locked; my $data = $self->data; my $lockfile = "$data/lockfile"; unlink $lockfile or die $!; $self->locked(0); } 1; 07070100000021000041ED000000000000000000000002663D00E300000000000000000000000000000000000000000000000F00000000cpanspec/munin07070100000022000081ED000000000000000000000001663D00E300000565000000000000000000000000000000000000002400000000cpanspec/munin/build-status-cron.sh#!/usr/bin/env bash # Writes OBS build statistics to YAML files # crontab example: # 51,11,31 * * * * STAT_DIR=$HOME/munin/obs-cpan BUILD_STATUS=/path/to/cpanspec/bin/build-status /path/to/build-status-cron.sh >>$HOME/munin/build-status-cron.log 2>&1 #STAT_DIR=$HOME/munin/obs-cpan #BUILD_STATUS=$HOME/develop/github/cpanspec/bin/build-status case "$1" in -h|--help) echo "Wrapper for the various build-status scripts" exit ;; esac if [[ -z "$STAT_DIR" ]]; then echo "Set STAT_DIR=~/path/to/stats" >&2 exit 1 fi if [[ -z "$BUILD_STATUS" ]]; then echo "Set BUILD_STATUS=/path/to/cpanspec/bin/build-status" >&2 exit 1 fi [[ ! -d $STAT_DIR ]] && mkdir $STAT_DIR cd $STAT_DIR for LETTER in A B C D E F G H I J K L M N O P Q R S T U V W X Y Z; do $BUILD_STATUS \ --project-prefix devel:languages:perl:CPAN- --yaml "$LETTER" \ >/tmp/build-status.yaml \ && mv /tmp/build-status.yaml $STAT_DIR/build-status-$LETTER.yaml sleep 1 done $BUILD_STATUS \ --project devel:languages:perl --yaml perl \ >/tmp/build-status.yaml \ && mv /tmp/build-status.yaml $STAT_DIR/build-status-perl.yaml $BUILD_STATUS \ --project devel:languages:perl:autoupdate --yaml autoupdate \ --repo standard \ >/tmp/build-status.yaml \ && mv /tmp/build-status.yaml $STAT_DIR/build-status-autoupdate.yaml 07070100000023000081ED000000000000000000000001663D00E3000009CB000000000000000000000000000000000000002200000000cpanspec/munin/munin-build-status#!/usr/bin/env perl # Munin configuration # put the following into /etc/munin/plugin-conf.d/munin-node # # [devel-build-status-*] # env.datadir /home/user/munin/obs-cpan # # Create plugin scripts: # cd /etc/munin/plugins # for i in A B C D E F G H I J K L M N O P Q R S T U V W X Y Z autoupdate perl total; # do # ln -s /path/to/cpanspec/munin/munin-build-status devel-build-status-$i # done # # Then restart munin-node. # # You can manually call a script like this to see what data munin will get: # /etc/munin/plugins/devel-build-status-A # /etc/munin/plugins/devel-build-status-perl # /etc/munin/plugins/devel-build-status-autoupdate use strict; use warnings; use 5.010; use Data::Dumper; use YAML::XS qw/ LoadFile /; my $script = $0; my $path = $ENV{datadir}; if (@ARGV and $ARGV[0] =~ m/^(-h|--help)$/) { say "Script to be called by munin, see munin documentation"; exit; } my @states = qw/ building finished scheduled blocked broken succeeded failed unresolvable disabled excluded /; my $letter; if ($script =~ m/-([A-Z])\z/) { $letter = $1; } elsif ($script =~ m/-total\z/) { $letter = 'total'; } elsif ($script =~ m/-perl\z/) { $letter = 'perl'; } elsif ($script =~ m/-autoupdate\z/) { $letter = 'autoupdate'; } else { die "script '$0' not supported"; } if ( defined $ARGV[0] and $ARGV[0] eq "config" ) { config(); exit; } my $data = {}; if ($letter eq 'total') { for my $letter ('A' .. 'Z') { my $file = "$path/build-status-$letter.yaml"; my $letter_data = LoadFile($file); $letter_data = $letter_data->{ $letter }; for my $state (@states) { $data->{ $state } += $letter_data->{ $state } || 0; } } } else { my $file = "$path/build-status-$letter.yaml"; $data = LoadFile($file); $data = $data->{ $letter }; } { my $total = 0; for my $state (@states) { my $value = $data->{ $state } || 0; $total += $value; print <<"EOM"; $state.value $value EOM } print <<"EOM"; total.value $total EOM } #graph_total total sub config { print <<"EOM"; graph_title CPAN Mirror Build Status $letter graph_args --base 1000 -l 0 graph_category obs-cpan graph_order succeeded unresolvable failed building finished scheduled blocked broken disabled excluded graph_vlabel packages EOM for my $state (@states) { print <<"EOM"; $state.label $state $state.draw LINE EOM } print <<"EOM"; total.label Total total.graph no EOM } 07070100000024000081ED000000000000000000000001663D00E3000028D9000000000000000000000000000000000000001700000000cpanspec/updateallcpan#! /usr/bin/perl # Copyright (C) 2015-2016 Stephan Kulow <coolo@suse.com> # This program is free software; you can redistribute it # and/or modify it under the same terms as Perl itself. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. use strict; use warnings; use XML::Simple; use Data::Dumper; use Compress::Zlib; use version; use List::Util; use File::Basename; use Storable; use Parse::CPAN::Packages; require CPAN::Meta::Requirements; use File::Temp qw/tempdir tempfile/; use File::Copy qw/copy/; my $scriptdir; #my $mirror = 'ftp://cpan.mirror.iphh.net/pub/CPAN'; my $mirror = 'http://cpan.noris.de'; BEGIN { ($scriptdir) = $0 =~ m-(.*)/-; $scriptdir ||= '.'; use Cwd 'abs_path'; $scriptdir = abs_path($scriptdir); } my $modpath = glob("~/.cpan/sources/modules"); my $details = "02packages.details.txt.gz"; chdir($modpath) || die "call cpan once to create $modpath"; system("wget -qm -nd http://www.cpan.org/modules/$details"); my %upstream; my $letter = $ARGV[0]; my %lpkgs; my $cache = {}; eval { $cache = retrieve("$scriptdir/obs_cache"); }; my $xml; my %autoupdate; if ($letter) { open(my $statusfh, "-|", "osc api /status/project/devel:languages:perl"); $xml = XMLin($statusfh)->{entry}; close($statusfh); open($statusfh, "-|", "osc api /source/devel:languages:perl:CPAN-$letter?view=info"); my $info = XMLin($statusfh)->{sourceinfo}; close($statusfh); for my $pi (@$info) { if (!defined $cache->{$pi->{srcmd5}}) { # TODO: cache print "CURL $pi->{package}\n"; open(my $statusfh, "-|", "osc api /source/devel:languages:perl:CPAN-$letter/$pi->{package}"); my $pxml = XMLin($statusfh); close($statusfh); for my $entry (keys %{$pxml->{entry}}) { if ($entry =~ m/\.tar/ || $entry =~ m/\.tgz$/ || $entry =~ m/\.zip$/) { my $name = $pxml->{name}; $lpkgs{$name} = $entry; } } $cache->{$pi->{srcmd5}} = $lpkgs{$pi->{package}} || ''; store $cache, "$scriptdir/obs_cache"; } else { $lpkgs{$pi->{package}} = $cache->{$pi->{srcmd5}}; } } } else { open(my $statusfh, "-|", "osc api /status/project/devel:languages:perl"); $xml = XMLin($statusfh)->{package}; close($statusfh); open($statusfh, "-|", "osc api /source/devel:languages:perl:autoupdate"); my $info = XMLin($statusfh)->{entry}; close($statusfh); for my $p (keys %$info) { $autoupdate{$p} = 1; } } my %letter_todo; my $p = Parse::CPAN::Packages->new("02packages.details.txt.gz"); my %seen_distris; for my $m ($p->packages) { $m = $m->distribution; if (!$m->version) { #printf "Distribution %s has no version defined\n", $m->prefix; next; } next if $seen_distris{$m->prefix} && $seen_distris{$m->prefix}[0] eq $m->version; next unless $m->dist; my $version; eval { $version = version->parse($m->version); }; $version ||= version->declare('0'); $seen_distris{$m->prefix} = [ $m->version, $version ]; my $url = $m->prefix; my $uversion; $uversion = $upstream{$m->dist}[0] if defined $upstream{$m->dist}; $uversion ||= version->declare('0'); #printf "PKG $url $version %s - %s (%s vs %s)\n", $m->dist, $m->filename, $uversion->stringify, $version->stringify; next if $version < $uversion; $upstream{$m->dist} = [$version, $url]; if ($letter) { my $tar = $m->filename; if ($tar && uc(substr($tar, 0, 1)) eq $letter) { if (basename($tar, qw/.tar.gz .tgz .zip/) =~ m!^(\S+)-([^-]*)$!) { my $pkg = $1; while ($pkg =~ m/^(.*)-(v?[0-9][^-]*)$/) { $pkg = $1; } next if defined $xml->{"perl-$pkg"}; my $obs = $lpkgs{"perl-$pkg"} || ''; next if $obs eq 'done'; $letter_todo{$pkg} = [$tar, $obs, $url] if $obs ne $tar; $lpkgs{"perl-$pkg"} = 'done'; } } } } if ($letter) { for my $pkg (sort keys %letter_todo) { my ($tar, $obs, $url) = @{$letter_todo{$pkg}}; my ($fh, $filename) = tempfile(); print $fh "<package name='perl-$pkg'><title/><description/><build><disable/></build></package>\n"; close($fh); system("osc meta pkg devel:languages:perl:CPAN-$letter perl-$pkg -F $filename"); if ($obs) { print "osc api -X DELETE /source/devel:languages:perl:CPAN-$letter/perl-$pkg/$obs\n"; system("osc api -X DELETE /source/devel:languages:perl:CPAN-$letter/perl-$pkg/$obs"); } my $tempdir = tempdir(CLEANUP => 1); chdir($tempdir); system("osc co devel:languages:perl:CPAN-$letter/perl-$pkg") == 0 or goto CHDIR; print "TAR $tar '$obs' '$pkg'\n"; chdir("devel:languages:perl:CPAN-$letter/perl-$pkg") || die "can't chdir"; #print "wget -nc -q $mirror/authors/id/$url\n"; system("wget -nc -q $mirror/authors/id/$url"); my $worked; unlink("perl-$pkg.spec"); if (system("$scriptdir/cpanspec -v -f --skip-changes $tar > cpanspec.error 2>&1") == 0 && -f "perl-$pkg.spec") { unlink("cpanspec.error"); $worked = 1; } system("osc addremove") == 0 or goto CHDIR; my $noarch; if (-f "perl-$pkg.spec") { open(SPEC, "perl-$pkg.spec"); while (<SPEC>) { $noarch = 1 if m/^BuildArch.*noarch/; } close(SPEC); } ($fh, $filename) = tempfile(); if ($worked) { if ($noarch) { print $fh "<package name='perl-$pkg'><title/><description/><build><disable arch='i586'/></build></package>\n"; print "build disable i586\n"; } else { print $fh "<package name='perl-$pkg'><title/><description/></package>\n"; print "build enable all\n"; } close($fh); #print "osc meta pkg devel:languages:perl:CPAN-$letter perl-$pkg -F $filename\n"; system("osc meta pkg devel:languages:perl:CPAN-$letter perl-$pkg -F $filename"); } system("osc ci -mupdate"); CHDIR: # finally emulation chdir("/tmp"); } for my $pkg (sort keys %lpkgs) { next if $lpkgs{$pkg} eq 'done'; print "osc rdelete -mgone devel:languages:perl:CPAN-$letter $pkg\n"; system "osc rdelete -mgone devel:languages:perl:CPAN-$letter $pkg"; } exit(0); } my $tocreate = 3000; my @pkgs = List::Util::shuffle(keys %$xml); @pkgs = sort keys %$xml; my %tobuild; while ((@pkgs && $tocreate) || %tobuild) { if (%tobuild) { my $url = "/build/devel:languages:perl:autoupdate/_result?"; my @tocheck = sort keys %tobuild; $url .= "package=" . shift @tocheck; $url .= "&package=$_" for (@tocheck); print "checking '$url'\n"; open(my $fh, "-|", "osc api '$url'"); my $res = XMLin($fh, forcearray => [qw/status/]); #print Dumper($res); close($fh); if ($res && $res->{result}->{status} && ($res->{result}->{dirty} || '') ne 'true') { #print Dumper($res); for my $status (@{$res->{result}->{status}}) { my $code = $status->{code} || 'unscheduled'; if ($code && $code eq 'finished') { $code = $status->{details} || 'unknown'; } my $built = $status->{package}; print "CODE $built $code\n"; if ($code eq 'succeeded') { system("osc sr -m 'automatic update' devel:languages:perl:autoupdate $built devel:languages:perl --clean < /dev/null"); delete $tobuild{$built}; } if ($code eq 'failed' || $code eq 'unresolvable' || $code eq 'broken') { delete $tobuild{$built}; } } } } if ($tocreate && @pkgs) { my $pkg = shift @pkgs; next unless $pkg =~ m,^perl-,; my $ups = $pkg; $ups =~ s,^perl-,,; if (!defined($upstream{$ups}[0])) { print "echo osc rdelete -mgone devel:languages:perl/$pkg\n"; } next unless defined($upstream{$ups}[0]); my $obs_version; eval { $obs_version = version->parse($xml->{$pkg}->{version}); }; next unless $obs_version; my $older = $obs_version < $upstream{$ups}[0]; if (!$older) { system("osc rdelete -mfresh -f devel:languages:perl:autoupdate $pkg") if $autoupdate{$pkg}; #print "PKG $pkg " . $obs_version . " $ups " . $upstream{$ups}[0] . " NEWER\n"; next; } print "PKG $pkg " . $obs_version . " $ups " . $upstream{$ups}[0] . " " . ($older ? 'OLDER' : 'NEWER') . "\n"; my $ntar = basename($upstream{$ups}[1]); open(my $statusfh, "-|", "osc api /source/devel:languages:perl/$pkg"); my $pxml = XMLin($statusfh); close($statusfh); if ($pxml->{entry}->{$ntar}) { #print "ALREADY THERE: $pkg\n"; system("osc rdelete -mfresh -f devel:languages:perl:autoupdate $pkg") if $autoupdate{$pkg}; next; } my $tempdir = tempdir; # ( CLEANUP => 1 ); chdir($tempdir); if (!$autoupdate{$pkg} && system("osc branch devel:languages:perl $pkg devel:languages:perl:autoupdate") != 0) { print "branch of $pkg failed\n"; next; } my $retries = 0; # we need to keep trying because the service needs to finish first while (system("osc co devel:languages:perl:autoupdate $pkg")) { print "checkout failed: $retries\n"; last if ($retries++ == 3); sleep(3); } next if ($retries > 2); $tocreate--; print "branched\n"; chdir("devel:languages:perl:autoupdate/$pkg") || die "can't chdir"; for my $tar (glob("*.tar*")) { unlink($tar); } system("wget -q $mirror/authors/id/$upstream{$ups}[1]"); copy("$scriptdir/cpanspec.yml", "cpanspec.yml") unless -f "cpanspec.yml"; system("$scriptdir/cpanspec -f --old-file .osc/*.tar* *.tar*") == 0 or next; system("osc addremove") == 0 or next; print "update\n"; system("osc ci -mupdate") == 0 or next; $tobuild{$pkg} = 1; print "SHELL $tempdir\n"; chdir("/tmp"); } else { sleep(5); } #exit(1); #exit(1) if ($requests++ > 5); } 07070100000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000B00000000TRAILER!!!314 blocks
Locations
Projects
Search
Status Monitor
Help
OpenBuildService.org
Documentation
API Documentation
Code of Conduct
Contact
Support
@OBShq
Terms
openSUSE Build Service is sponsored by
The Open Build Service is an
openSUSE project
.
Sign Up
Log In
Places
Places
All Projects
Status Monitor
