Sign Up
Log In
Log In
or
Sign Up
Places
All Projects
Status Monitor
Collapse sidebar
home:crameleon:misc
python-j2lint
_service:obs_scm:j2lint-1.1.0.obscpio
Overview
Repositories
Revisions
Requests
Users
Attributes
Meta
File _service:obs_scm:j2lint-1.1.0.obscpio of Package python-j2lint
07070100000000000041ED00000000000000000000000364AD1D7A00000000000000000000000000000000000000000000001500000000j2lint-1.1.0/.github07070100000001000081A400000000000000000000000164AD1D7A0000061A000000000000000000000000000000000000002800000000j2lint-1.1.0/.github/release_process.md# Notes Notes regarding how to release ## Bumping version In a branch specific for this, use the `bumpver` tool. It is configured to update: * pyproject.toml * j2lint/__init__.py * tests/test_cli.py (where a test verifies the version output) For instance to bump a patch version: ``` bumpver update --patch ``` and for a minor version ``` bumpver update --minor ``` Tip: It is possible to check what the changes would be using `--dry` ``` bumpver update --minor --dry ``` ## Creating release on Github Create the release on Github with the appropriate tag `vx.x.x` ## Release version `x.x.x` TODO - make this a workflow `x.x.x` is the version to be released This is to be executed at the top of the repo 1. Checkout the latest version of devel with the correct tag for the release 2. [Optional] Clean dist if required 3. Build the package locally ``` python -m build ``` 4. Check the package with `twine` (replace with your vesion) ``` twine check dist/j2lint-x.x.x-py3-none-any.whl ``` 5. Upload the package to test.pypi ``` twine upload -r testpypi dist/j2lint-x.x.x.* ``` 6. Verify the package by installing it in a local venv and checking it installs and run correctly (run the tests) ``` # In a brand new venv pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple --no-cache j2lint ``` 7. Upload the package to pypi ``` twine upload dist/j2lint-x.x.x.* ``` 8. Like 5 but for normal pypi ``` # In a brand new venv pip install j2lint ``` 07070100000002000041ED00000000000000000000000264AD1D7A00000000000000000000000000000000000000000000001F00000000j2lint-1.1.0/.github/workflows07070100000003000081A400000000000000000000000164AD1D7A000004AC000000000000000000000000000000000000003B00000000j2lint-1.1.0/.github/workflows/pull-request-management.yml--- name: Code Testing on: pull_request jobs: lint: name: Run pylint runs-on: ubuntu-20.04 steps: - uses: actions/checkout@v3 - name: Setup Python uses: actions/setup-python@v3 with: python-version: "3.10" - name: Install dependencies run: pip install tox tox-gh-actions - name: "Run lint" run: tox -e lint type: name: Run mypy runs-on: ubuntu-20.04 steps: - uses: actions/checkout@v3 - name: Setup Python uses: actions/setup-python@v3 with: python-version: "3.10" - name: Install dependencies run: pip install tox tox-gh-actions - name: "Run mypy" run: tox -e type tox: name: Run pytest for supported Python versions runs-on: ubuntu-20.04 strategy: matrix: python: ["3.8", "3.9", "3.10"] steps: - uses: actions/checkout@v3 - name: Setup Python uses: actions/setup-python@v3 with: python-version: ${{ matrix.python }} - name: Install dependencies run: pip install tox tox-gh-actions - name: "Run tox for ${{ matrix.python }}" run: tox 07070100000004000081A400000000000000000000000164AD1D7A0000006D000000000000000000000000000000000000001800000000j2lint-1.1.0/.gitignore__pycache__ *.pyc *.log build dist *.egg-info **/.python-version .tox/** tests/tmp/** .coverage **/.DS_Store 07070100000005000081A400000000000000000000000164AD1D7A00000643000000000000000000000000000000000000002500000000j2lint-1.1.0/.pre-commit-config.yaml# See https://pre-commit.com for more information # See https://pre-commit.com/hooks.html for more hooks repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v2.4.0 hooks: - id: trailing-whitespace - id: end-of-file-fixer #exclude_types: [jinja, text] - id: check-added-large-files - id: check-merge-conflict - repo: https://github.com/pycqa/flake8 rev: 4.0.1 hooks: - id: flake8 name: Check for PEP8 error on Python files args: - --config=/dev/null - --max-line-length=160 types: [python] - repo: https://github.com/pycqa/pylint rev: v2.17.0 hooks: - id: pylint # Use pylintrc file in repository name: Check for Linting error on Python files description: This hook runs pylint. types: [python] args: # Suppress duplicate code for modules header - -d duplicate-code additional_dependencies: - jinja2 - rich exclude: ^tests/ - repo: https://github.com/pycqa/isort rev: 5.12.0 hooks: - id: isort name: Check for changes when running isort on all python files - repo: https://github.com/psf/black rev: 23.3.0 hooks: - id: black name: Check for changes when running Black on all python files - repo: https://github.com/pre-commit/mirrors-mypy rev: v0.991 hooks: - id: mypy args: - --config-file=pyproject.toml # additional_dependencies: # Do not run on test files: ^(j2lint)/ 07070100000006000081A400000000000000000000000164AD1D7A0000008C000000000000000000000000000000000000002400000000j2lint-1.1.0/.pre-commit-hooks.yaml- id: j2lint name: Check for Linting error on Jinja2 templates entry: j2lint language: python types: [jinja] require_serial: true 07070100000007000081A400000000000000000000000164AD1D7A00000430000000000000000000000000000000000000001500000000j2lint-1.1.0/LICENSEMIT License Copyright (c) 2021 Arista Networks Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 07070100000008000081A400000000000000000000000164AD1D7A000019B8000000000000000000000000000000000000001700000000j2lint-1.1.0/README.md[](https://github.com/aristanetworks/j2lint/blob/devel/LICENSE) [](https://pypi.python.org/pypi/j2lint/) [](https://pypi.python.org/pypi/j2lint/) [](https://pypi.python.org/pypi/j2lint/) [](https://github.com/aristanetworks/j2lint/graphs/commit-activity) # Jinja2-Linter AVD Ecosystem - Jinja2 Linter ## Project Goals Build a Jinja2 linter that will provide the following capabilities: - Validate syntax according to [AVD style guide](https://avd.sh/en/stable/docs/contribution/style-guide.html). - Capability to run as part of a CI pipeline to enforce j2lint rules. - Develop an extension that works with VSCode and potentially other IDEs i.e PyCharm. ## Syntax and code style issues | Code | Short Description | Description | |------|-------------------|-------------| | S0 | `jinja-syntax-error` | Jinja2 syntax should be correct | | S1 | `single-space-decorator` | A single space should be added between Jinja2 curly brackets and a variable's name | | S2 | `operator-enclosed-by-spaces` | When variables are used in combination with an operator, the operator shall be enclosed by space | | S3 | `jinja-statements-indentation` | Nested jinja code block should follow next rules:<br>- All J2 statements must be enclosed by 1 space<br>- All J2 statements must be indented by 4 more spaces within jinja delimiter<br>- To close a control, end tag must have same indentation level | | S4 | `jinja-statements-single-space` | Jinja statement should have at least a single space after '{%' and a single space before '%}' | | S5 | `jinja-statements-no-tabs` | Indentation should not use tabulation but 4 spaces | | S6 | `jinja-statements-delimiter` | Jinja statements should not have {%- or {%+ or -%} as delimiters | | S7 | `single-statement-per-line` | Jinja statements should be on separate lines | | V1 | `jinja-variable-lower-case` | All variables should use lower case | | V2 | `jinja-variable-format` | If variable is multi-words, underscore `_` should be used as a separator | ## Getting Started ### Requirements Python version 3.8+ ### Install with pip To get started, you can use Python pip to install j2lint: **Install the latest stable version:** ```bash pip3 install j2lint ``` **Install the latest development version:** ```bash pip3 install git+https://github.com/aristanetworks/j2lint.git ``` ## Running the linter ```bash j2lint <path-to-directory-of-templates> ``` ### Running the linter on a specific file ```bash j2lint <path-to-directory-of-templates>/template.j2 ``` ### Listing linting rules ```bash j2lint --list ``` ### Running the linter with verbose linter error output ```bash j2lint <path-to-directory-of-templates> --verbose ``` ### Running the linter with logs enabled. Logs saved in jinja2-linter.log in the current directory ```bash j2lint <path-to-directory-of-templates> --log ``` To enable debug logs, use both options: ```bash j2lint <path-to-directory-of-templates> --log --debug ``` ### Running the linter with JSON format for linter error output ```bash j2lint <path-to-directory-of-templates> --json ``` ### Ignoring rules 1. The --ignore option can have one or more of these values: syntax-error, single-space-decorator, filter-enclosed-by-spaces, jinja-statement-single-space, jinja-statements-indentation, no-tabs, single-statement-per-line, jinja-delimiter, jinja-variable-lower-case, jinja-variable-format. 2. If multiple rules are to be ignored, use the --ignore option along with rule descriptions separated by space. ```bash j2lint <path-to-directory-of-templates> --ignore <rule_description1> <rule_desc> ``` > **Note** > This runs the custom linting rules in addition to the default linting rules. > When using the `-i/--ignore` or `-w/--warn` options, the arguments MUST either: > * Be entered at the end of the CLI as in the example above > * Be entered as the last options before the `<path-to-directory-of-templates>` > with `--` separator. e.g. > ```bash > j2lint --ignore <rule_description1> <rule_desc> -- <path-to-directory-of-templates> > ``` 3. If one or more linting rules are to be ignored only for a specific jinja template file, add a Jinja comment at the top of the file. The rule can be disabled using the short description of the rule or the id of the rule. ```jinja2 {# j2lint: disable=S6} # OR {# j2lint: disable=jinja-delimiter #} ``` 4. Disabling multiple rules ```jinja2 {# j2lint: disable=jinja-delimiter j2lint: disable=S1 #} ``` ### Adding custom rules 1. Create a new rules directory under j2lint folder. 2. Add custom rule classes which are similar to classes in j2lint/rules directory: The file name of rules should be in snake_case and the class name should be the PascalCase version of the file name. For example: - File name: `jinja_operator_has_spaces_rule.py` - Class name: `JinjaOperatorHasSpacesRule` 3. Run the jinja2 linter using --rules-dir option ```bash j2lint <path-to-directory-of-templates> --rules-dir <custom-rules-directory> ``` > **Note** > This runs the custom linting rules in addition to the default linting rules. ### Running jinja2 linter help command ```bash j2lint --help ``` ### Running jinja2 linter on STDIN template. This option can be used with VS Code. ```bash j2lint --stdin ``` ### Using j2lint as a pre-commit-hook 1. Add j2lint pre-commit hook inside your repository in .pre-commit-config.yaml. ```bash - repo: https://github.com/aristanetworks/j2lint.git rev: <release_tag/sha> hooks: - id: j2lint ``` 2. Run pre-commit -> `pre-commit run --all-files` > **Note** > When using `-i/--ignore` or `-w/--warn` argument in pre-commit, use the > following syntax > > ```bash > - repo: https://github.com/aristanetworks/j2lint.git > rev: <release_tag/sha> > hooks: > - id: j2lint > # Using -- to separate the end of ignore from the positional arguments > # passed to j2lint > args: [--ignore, S3, jinja-statements-single-space, --] > ``` ## Acknowledgments This project is based on [salt-lint](https://github.com/warpnet/salt-lint) and [jinjalint](https://github.com/motet-a/jinjalint) 07070100000009000041ED00000000000000000000000464AD1D7A00000000000000000000000000000000000000000000001400000000j2lint-1.1.0/j2lint0707010000000A000081A400000000000000000000000164AD1D7A000000D9000000000000000000000000000000000000002000000000j2lint-1.1.0/j2lint/__init__.py"""__init__.py - A command-line utility that checks for best practices in Jinja2. """ NAME = "j2lint" VERSION = "v1.1.0" DESCRIPTION = __doc__ __author__ = "Arista Networks" __license__ = "MIT" __version__ = VERSION 0707010000000B000081A400000000000000000000000164AD1D7A00000145000000000000000000000000000000000000002000000000j2lint-1.1.0/j2lint/__main__.py#!/usr/bin/python """__main__.py - A command-line utility that checks for best practices in Jinja2. """ import sys import traceback from j2lint.cli import run if __name__ == "__main__": try: sys.exit(run()) except Exception: print(traceback.format_exc()) raise SystemExit from BaseException 0707010000000C000081A400000000000000000000000164AD1D7A000023F2000000000000000000000000000000000000001B00000000j2lint-1.1.0/j2lint/cli.py"""cli.py - Command line argument parser. """ from __future__ import annotations import argparse import json import logging import os import sys import tempfile from rich.console import Console from rich.tree import Tree from . import DESCRIPTION, NAME, VERSION from .linter.collection import DEFAULT_RULE_DIR, RulesCollection from .linter.error import LinterError from .linter.runner import Runner from .logger import add_handler, logger from .utils import get_files IGNORE_RULES = WARN_RULES = [ "jinja-syntax-error", "single-space-decorator", "operator-enclosed-by-spaces", "jinja-statements-single-space", "jinja-statements-indentation", "jinja-statements-no-tabs", "single-statement-per-line", "jinja-statements-delimiter", "jinja-variable-lower-case", "jinja-variable-format", "S0", "S1", "S2", "S3", "S4", "S5", "S6", "S7", "V1", "V2", ] CONSOLE = Console() def create_parser() -> argparse.ArgumentParser: """Initializes a new argument parser object Returns: ArgumentParser: Argument parser object """ parser = argparse.ArgumentParser(prog=NAME, description=DESCRIPTION) parser.add_argument( dest="files", metavar="FILE", nargs="*", default=[], help="files or directories to lint", ) parser.add_argument( "-l", "--list", default=False, action="store_true", help="list of lint rules" ) parser.add_argument( "-r", "--rules_dir", dest="rules_dir", action="append", default=[DEFAULT_RULE_DIR], help="rules directory", ) parser.add_argument( "-v", "--verbose", default=False, action="store_true", help="verbose output for lint issues", ) parser.add_argument( "-d", "--debug", default=False, action="store_true", help="enable debug logs" ) parser.add_argument( "-j", "--json", default=False, action="store_true", help="enable JSON output" ) parser.add_argument( "-s", "--stdin", default=False, action="store_true", help="accept template from STDIN", ) parser.add_argument( "--log", default=False, action="store_true", help="enable logging" ) parser.add_argument( "--version", default=False, action="store_true", help="Version of j2lint" ) parser.add_argument( "-o", "--stdout", default=False, action="store_true", help="stdout logging" ) parser.add_argument( "-i", "--ignore", nargs="*", choices=IGNORE_RULES, default=[], help="rules to ignore, use `--` after this option to enter FILES", ) parser.add_argument( "-w", "--warn", nargs="*", choices=WARN_RULES, default=[], help="rules to warn, use `--` after this option to enter FILES", ) return parser def sort_issues(issues: list[LinterError]) -> list[LinterError]: """Sorted list of issues Args: issues (list): list of issue dictionaries Returns: list: list of sorted issue dictionaries """ issues.sort( key=lambda issue: (issue.filename, issue.line_number, issue.rule.rule_id) ) return issues def get_linting_issues( file_or_dir_names: list[str], collection: RulesCollection, checked_files: list[str] ) -> tuple[dict[str, list[LinterError]], dict[str, list[LinterError]]]: """checking errors and warnings""" lint_errors: dict[str, list[LinterError]] = {} lint_warnings: dict[str, list[LinterError]] = {} files = get_files(file_or_dir_names) # Get linting issues for file_name in files: runner = Runner(collection, file_name, checked_files) if file_name not in lint_errors: lint_errors[file_name] = [] if file_name not in lint_warnings: lint_warnings[file_name] = [] j2_errors, j2_warnings = runner.run() lint_errors[file_name].extend(sort_issues(j2_errors)) lint_warnings[file_name].extend(sort_issues(j2_warnings)) return lint_errors, lint_warnings def print_json_output( lint_errors: dict[str, list[LinterError]], lint_warnings: dict[str, list[LinterError]], ) -> tuple[int, int]: """printing json output""" json_output: dict[str, list[str]] = {"ERRORS": [], "WARNINGS": []} for _, errors in lint_errors.items(): for error in errors: json_output["ERRORS"].append(json.loads(str(error.to_json()))) for _, warnings in lint_warnings.items(): for warning in warnings: json_output["WARNINGS"].append(json.loads(str(warning.to_json()))) CONSOLE.print_json(f"\n{json.dumps(json_output)}") return len(json_output["ERRORS"]), len(json_output["WARNINGS"]) def print_string_output( lint_errors: dict[str, list[LinterError]], lint_warnings: dict[str, list[LinterError]], verbose: bool, ) -> tuple[int, int]: """print non-json output""" def print_issues( lint_issues: dict[str, list[LinterError]], issue_type: str ) -> None: CONSOLE.rule(f"[bold red]JINJA2 LINT {issue_type}") for key, issues in lint_issues.items(): if not issues: continue tree = Tree(f"{key}") for j2_issue in issues: tree.add(j2_issue.to_rich(verbose)) CONSOLE.print(tree) total_lint_errors = sum(len(issues) for _, issues in lint_errors.items()) total_lint_warnings = sum(len(issues) for _, issues in lint_warnings.items()) if total_lint_errors: print_issues(lint_errors, "ERRORS") if total_lint_warnings: print_issues(lint_warnings, "WARNINGS") if not total_lint_errors and not total_lint_warnings: if verbose: CONSOLE.print("Linting complete. No problems found!", style="green") else: CONSOLE.print( f"\nJinja2 linting finished with " f"{total_lint_errors} error(s) and {total_lint_warnings} warning(s)" ) return total_lint_errors, total_lint_warnings def remove_temporary_file(stdin_filename: str) -> None: """Remove temporary file""" if stdin_filename: os.unlink(stdin_filename) def print_string_rules(collection: RulesCollection) -> None: """Print active rules as string""" CONSOLE.rule("[bold red]Rules in the Collection") CONSOLE.print(collection.to_rich()) def print_json_rules(collection: RulesCollection) -> None: """Print active rules as json""" CONSOLE.print_json(collection.to_json()) def run(args: list[str] | None = None) -> int: """Runs jinja2 linter Args: args ([string], optional): Command line arguments. Defaults to None. Returns: int: 0 on success """ # pylint: disable=too-many-branches # given the number of input parameters, it is acceptable to keep these many branches. parser = create_parser() options = parser.parse_args(args if args is not None else sys.argv[1:]) # Enable logs if not options.log and not options.stdout: logging.disable(sys.maxsize) else: log_level = logging.DEBUG if options.debug else logging.INFO if options.log: add_handler(logger, False, log_level) if options.stdout: add_handler(logger, True, log_level) logger.debug("Lint options selected %s", options) stdin_filename = None file_or_dir_names: list[str] = list(set(options.files)) checked_files: list[str] = [] if options.stdin and not sys.stdin.isatty(): with tempfile.NamedTemporaryFile( "w", suffix=".j2", delete=False ) as stdin_tmpfile: stdin_tmpfile.write(sys.stdin.read()) stdin_filename = stdin_tmpfile.name file_or_dir_names.append(stdin_filename) # Collect the rules from the configuration collection = RulesCollection(options.verbose) for rules_dir in options.rules_dir: collection.extend( RulesCollection.create_from_directory( rules_dir, options.ignore, options.warn ).rules ) # List lint rules if options.list: if options.json: print_json_rules(collection) else: print_string_rules(collection) return 0 # Version of j2lint if options.version: CONSOLE.print(f"Jinja2-Linter Version [bold red]{VERSION}") return 0 # Print help message if not file_or_dir_names: parser.print_help(file=sys.stderr) return 1 lint_errors, lint_warnings = get_linting_issues( file_or_dir_names, collection, checked_files ) if options.json: logger.debug("JSON output enabled") total_lint_errors, _ = print_json_output(lint_errors, lint_warnings) else: total_lint_errors, _ = print_string_output( lint_errors, lint_warnings, options.verbose ) # Remove temporary file if stdin_filename is not None: remove_temporary_file(stdin_filename) return 2 if total_lint_errors else 0 0707010000000D000041ED00000000000000000000000364AD1D7A00000000000000000000000000000000000000000000001B00000000j2lint-1.1.0/j2lint/linter0707010000000E000081A400000000000000000000000164AD1D7A00000000000000000000000000000000000000000000002700000000j2lint-1.1.0/j2lint/linter/__init__.py0707010000000F000081A400000000000000000000000164AD1D7A00001563000000000000000000000000000000000000002900000000j2lint-1.1.0/j2lint/linter/collection.py"""collection.py - Class to create a collection of linting rules. """ from __future__ import annotations import json import os import pathlib from collections.abc import Iterable from rich.console import Group from rich.tree import Tree from j2lint.logger import logger from j2lint.utils import is_rule_disabled, load_plugins from .error import LinterError from .rule import Rule DEFAULT_RULE_DIR = pathlib.Path(__file__).parent.parent / "rules" class RulesCollection: """RulesCollection class which checks the linting rules against a file.""" def __init__(self, verbose: bool = False) -> None: self.rules: list[Rule] = [] self.verbose = verbose def __iter__(self) -> Iterable[Rule]: return iter(self.rules) def __len__(self) -> int: return len(self.rules) def extend(self, more: list[Rule]) -> None: """Extends list of rules Args: more (list): list of rules classes Note: This does not protect against duplicate rules """ self.rules.extend(more) def run( self, file_dict: dict[str, str] ) -> tuple[list[LinterError], list[LinterError]]: """Runs the linting rules for given file Args: file_dict (dict): file path and file type Returns: tuple(list, list): a tuple containing the list of linting errors and the list of linting warnings found """ text = "" errors: list[LinterError] = [] warnings: list[LinterError] = [] try: with open(file_dict["path"], mode="r", encoding="utf-8") as file: text = file.read() except IOError as err: logger.warning("Could not open %s - %s", file_dict["path"], err.strerror) return errors, warnings for rule in self.rules: if rule.ignore: logger.debug( "Ignoring rule %s:%s for file %s", rule.rule_id, rule.short_description, file_dict["path"], ) continue if is_rule_disabled(text, rule): logger.debug( "Skipping linting rule %s on file %s", rule, file_dict["path"] ) continue logger.debug("Running linting rule %s on file %s", rule, file_dict["path"]) if rule in rule.warn: warnings.extend(rule.checkrule(file_dict, text)) else: errors.extend(rule.checkrule(file_dict, text)) for error in errors: logger.error(error.to_rich()) for warning in warnings: logger.warning(warning.to_rich()) return errors, warnings def __repr__(self) -> str: res = [] current_origin = None for rule in sorted(self.rules, key=lambda x: (x.origin, x.rule_id)): if rule.origin != current_origin: current_origin = rule.origin res.append(f"Origin: {rule.origin}") res.append(repr(rule)) return "\n".join(res) def to_rich(self) -> Group: """ Return a rich Group containing a rich Tree for each different origin for the rules Each Tree contain the rule.to_rich() output Origin: BUILT-IN ├── S0 Jinja syntax should be correct (jinja-syntax-error) ├── S1 <description> (single-space-decorator) └── V2 <description> (jinja-variable-format) """ res = [] current_origin = None tree = None for rule in sorted(self.rules, key=lambda x: (x.origin, x.rule_id)): if rule.origin != current_origin: current_origin = rule.origin tree = Tree(f"Origin: {rule.origin}") res.append(tree) assert tree tree.add(rule.to_rich()) return Group(*res) def to_json(self) -> str: """Return a json representation of the collection as a list of the rules""" return json.dumps( [ json.loads(rule.to_json()) for rule in sorted(self.rules, key=lambda x: (x.origin, x.rule_id)) ] ) @classmethod def create_from_directory( cls, rules_dir: str, ignore_rules: list[str], warn_rules: list[str] ) -> RulesCollection: """Creates a collection from all rule modules Args: rules_dir (string): rules directory ignore_rules (list): list of rule short_descriptions or ids to ignore warn_rules (list): list of rule short_descriptions or ids to consider as warnings rather than errors Returns: list: a collection of rule objects """ result = cls() result.rules = load_plugins(os.path.expanduser(rules_dir)) for rule in result.rules: if rule.short_description in ignore_rules or rule.rule_id in ignore_rules: rule.ignore = True if rule.short_description in warn_rules or rule.rule_id in warn_rules: rule.warn.append(rule) if rules_dir != DEFAULT_RULE_DIR: for rule in result.rules: rule.origin = rules_dir logger.info("Created collection from rules directory %s", rules_dir) return result 07070100000010000081A400000000000000000000000164AD1D7A00000855000000000000000000000000000000000000002400000000j2lint-1.1.0/j2lint/linter/error.py"""error.py - Error classes to format the lint errors. """ from __future__ import annotations import json from typing import TYPE_CHECKING from rich.text import Text from j2lint.logger import logger if TYPE_CHECKING: from .rule import Rule class LinterError: """Class for lint errors.""" def __init__( self, line_number: int, line: str, filename: str, rule: Rule, message: str | None = None, ) -> None: # pylint: disable=too-many-arguments self.line_number = line_number self.line = line self.filename = filename self.rule = rule self.message = message or rule.description def to_rich(self, verbose: bool = False) -> Text: """setting string output format""" text = Text() if not verbose: text.append(self.filename, "green") text.append(":") text.append(str(self.line_number), "red") text.append(f" {self.message}") text.append(f" ({self.rule.short_description})", "blue") else: logger.debug("Verbose mode enabled") text.append("Linting rule: ") text.append(f"{self.rule.rule_id}\n", "blue") text.append("Rule description: ") text.append(f"{self.rule.description}\n", "blue") text.append("Error line: ") text.append(self.filename, "green") text.append(":") text.append(str(self.line_number), "red") text.append(f" {self.line}\n") text.append("Error message: ") text.append(f"{self.message}\n") return text def to_json(self) -> str: """setting json output format""" return json.dumps( { "id": self.rule.rule_id, "message": self.message, "filename": self.filename, "line_number": self.line_number, "line": self.line, "severity": self.rule.severity, } ) class JinjaLinterError(Exception): """Jinja Linter Error""" 07070100000011000041ED00000000000000000000000264AD1D7A00000000000000000000000000000000000000000000002400000000j2lint-1.1.0/j2lint/linter/indenter07070100000012000081A400000000000000000000000164AD1D7A00000000000000000000000000000000000000000000003000000000j2lint-1.1.0/j2lint/linter/indenter/__init__.py07070100000013000081A400000000000000000000000164AD1D7A00002645000000000000000000000000000000000000002C00000000j2lint-1.1.0/j2lint/linter/indenter/node.py"""node.py - Class node for creating a parse tree for jinja statements and checking jinja statement indentation. """ from __future__ import annotations from typing import NoReturn, Tuple from j2lint.linter.error import JinjaLinterError from j2lint.linter.indenter.statement import JINJA_STATEMENT_TAG_NAMES, JinjaStatement from j2lint.logger import logger from j2lint.utils import Statement, delimit_jinja_statement, flatten, get_tuple BEGIN_TAGS = [item[0] for item in JINJA_STATEMENT_TAG_NAMES] END_TAGS = [item[-1] for item in JINJA_STATEMENT_TAG_NAMES] MIDDLE_TAGS = list(flatten([[i[1:-1] for i in JINJA_STATEMENT_TAG_NAMES]])) INDENT_SHIFT = 4 DEFAULT_WHITESPACES = 1 JINJA_START_DELIMITERS = ["{%-", "{%+"] jinja_node_stack: list[Node] = [] jinja_delimiter_stack: list[str] = [] # Using Tuple from typing for 3.8 support NodeIndentationError = Tuple[int, str, str] class Node: """Node class which represents a jinja file as a tree""" # pylint: disable=too-many-instance-attributes # Eight arguments is reasonable in this case def __init__(self) -> None: self.statement: JinjaStatement | None = None self.tag: str | None = None self.parent: Node = self self.node_start: int = 0 self.node_end: int = 0 self.children: list[Node] = [] self.block_start_indent: int = 0 self.expected_indent: int = 0 # pylint: disable=fixme # TODO - This should be called create_child_node def create_node(self, line: Statement, line_no: int, indent_level: int = 0) -> Node: """Initializes a Node class object Args: line (Statement): Parsed line of the template using get_jinja_statement line_no (int): line number indent_level (int, optional): expected indentation level. Defaults to 0. Returns: Node: new Node class object """ node = Node() statement = JinjaStatement(line) node.statement = statement node.tag = statement.words[0] node.node_start = line_no node.node_end = line_no node.expected_indent = indent_level node.parent = self return node @staticmethod def create_indentation_error( node: Node, message: str ) -> NodeIndentationError | None: """Creates indentation error tuple Args: node (Node): Node class object to create error for message (string): error message for the line Returns: tuple: tuple representing the indentation error """ if node.statement is None: return None return ( node.statement.start_line_no, delimit_jinja_statement( node.statement.line, node.statement.start_delimiter, node.statement.end_delimiter, ), message, ) def check_indent_level( self, result: list[NodeIndentationError], node: Node ) -> None: """check if the actual and expected indent level for a line match Args: result (list): list of tuples of indentation errors node (Node): Node object for which to check the level is correct """ if node.statement is None: return actual = node.statement.begin if ( jinja_node_stack and jinja_node_stack[0].statement is not None and jinja_node_stack[0].statement.start_delimiter in JINJA_START_DELIMITERS ): self.block_start_indent = 1 elif ( node.expected_indent == 0 and node.statement.start_delimiter in JINJA_START_DELIMITERS ): self.block_start_indent = 1 else: self.block_start_indent = 0 if node.statement.start_delimiter in JINJA_START_DELIMITERS: expected = node.expected_indent + self.block_start_indent else: expected = ( node.expected_indent + DEFAULT_WHITESPACES + self.block_start_indent ) if actual != expected: message = f"Bad Indentation, expected {expected}, got {actual}" if (error := self.create_indentation_error(node, message)) is not None: result.append(error) logger.debug(error) def _assert_not_none(self, current_line_no: int, new_line_no: int | None) -> int: """ Helper function to verify that the new_line_no is not None TODO: Probably should never return None and instead raise in check_indentation """ if new_line_no is None: raise JinjaLinterError( "Recursive check_indentation returned None for an opening tag " f"line {current_line_no} - missing closing tag", ) return new_line_no # pylint: disable=inconsistent-return-statements,fixme # TODO - newer version of pylint (2.17.0) catches some error here # address in refactoring def check_indentation( self, result: list[NodeIndentationError], lines: list[Statement], line_no: int = 0, indent_level: int = 0, ) -> int | None: """Checks indentation for a list of lines Updates the 'result' list argument with indentation errors Args: result (list): list of indentation error tuples lines (list): lines which are to be checked for indentation line_no (int, optional): the current lines number being evaluated. Defaults to 0. indent_level (int, optional): the expected indent level for the current line. Defaults to 0. Raises: JinjaLinterError: Raises error if the text file has jinja tags which are not supported by this indenter ValueError: Raised when no begin_tag_tuple can be found in a node in the stack Returns: line_no (int) or None """ def _append_error_to_result_and_raise(message: str) -> NoReturn: """ Helper function to append error to result and raise a JinjaLinterError """ if (error := self.create_indentation_error(node, message)) is not None: result.append(error) raise JinjaLinterError(message) def _handle_begin_tag(node: Node, line_no: int) -> int: jinja_node_stack.append(node) self.children.append(node) line_no = self._assert_not_none( line_no, node.check_indentation( result, lines, line_no + 1, indent_level + INDENT_SHIFT ), ) self.check_indent_level(result, node) return line_no def _handle_middle_tag(node: Node, line_no: int) -> int: matchnode = jinja_node_stack[-1] node.node_end = line_no node.expected_indent = matchnode.expected_indent indent_level = node.expected_indent matchnode.parent.children.append(node) node.parent = matchnode.parent line_no = self._assert_not_none( line_no, node.check_indentation( result, lines, line_no + 1, indent_level + INDENT_SHIFT ), ) self.check_indent_level(result, node) return line_no def _handle_end_tag(node: Node, line_no: int) -> int: # type: ignore if f"end{jinja_node_stack[-1].tag}" == node.tag: if jinja_node_stack[-1] != self: return line_no matchnode = jinja_node_stack[-1] matchnode.node_end = line_no node.node_end = line_no node.expected_indent = matchnode.expected_indent self.parent.children.append(node) # type: ignore if matchnode == self: line_no += 1 self.check_indent_level(result, node) jinja_node_stack.pop() return line_no # End Tag not matching the begin tag - raise an error message = f"Line {line_no} - Tag is out of order '{node.tag}'" _append_error_to_result_and_raise(message) # Never reached return line_no while line_no < len(lines): line = lines[line_no] node = self.create_node(line, line_no, indent_level) if node.tag in BEGIN_TAGS: line_no = _handle_begin_tag(node, line_no) continue if node.tag in END_TAGS: return _handle_end_tag(node, line_no) if node.tag in MIDDLE_TAGS: begin_tag_tuple = get_tuple( JINJA_STATEMENT_TAG_NAMES, jinja_node_stack[-1].tag ) if begin_tag_tuple is None: _append_error_to_result_and_raise( f"Node {jinja_node_stack[-1]} should have been a begin_tag" ) if node.tag in begin_tag_tuple: # type: ignore if jinja_node_stack[-1] != self: del node return line_no line_no = _handle_middle_tag(node, line_no) continue message = f"Unsupported tag '{node.tag}' found" _append_error_to_result_and_raise(message) self.children.append(node) line_no = line_no + 1 self.check_indent_level(result, node) return None 07070100000014000081A400000000000000000000000164AD1D7A000003AC000000000000000000000000000000000000003100000000j2lint-1.1.0/j2lint/linter/indenter/statement.py"""statement.py - Class and variables for jinja statements. """ from __future__ import annotations # pylint: disable=too-few-public-methods import re from j2lint.utils import Statement JINJA_STATEMENT_TAG_NAMES = [ ("for", "else", "endfor"), ("if", "elif", "else", "endif"), ("macro", "endmacro"), ] class JinjaStatement: """Class for representing a jinja statement.""" # pylint: disable = fixme # FIXME - this could probably be a method in Node rather than a class # with no method - maybe a dataclass def __init__(self, line: Statement) -> None: whitespaces = re.findall(r"\s*", line[0]) self.begin: int = len(whitespaces[0]) self.line: str = line[0] self.words: list[str] = line[0].split() self.start_line_no: int = line[1] self.end_line_no: int = line[2] self.start_delimiter: str = line[3] self.end_delimiter: str = line[4] 07070100000015000081A400000000000000000000000164AD1D7A000012C6000000000000000000000000000000000000002300000000j2lint-1.1.0/j2lint/linter/rule.py"""rule.py - Base class for all the lint rules with functions for mathching line and text based rule. """ from __future__ import annotations import json from abc import ABC, abstractclassmethod, abstractmethod from typing import Any from rich.text import Text from j2lint.linter.error import JinjaLinterError, LinterError from j2lint.logger import logger from j2lint.utils import is_valid_file_type class Rule(ABC): """Abstract rule class which acts as a base class for rules with regex match functions. """ def __init__( self, ignore: bool = False, warn: list[Any] | None = None, origin: str = "BUILT-IN", ): self.ignore = ignore self.warn = warn if warn is not None else [] self.origin = origin # Mandatory class attributes # ignoring mypy issue as the BDFL said # https://github.com/python/mypy/issues/1362 @property # type: ignore @abstractclassmethod def rule_id(cls) -> str: # sourcery skip: instance-method-first-arg-name """ The rule id like S0 """ @property # type: ignore @abstractclassmethod def description(cls) -> str: # sourcery skip: instance-method-first-arg-name """ The rule description """ @property # type: ignore @abstractclassmethod def short_description(cls) -> str: # sourcery skip: instance-method-first-arg-name """ The rule short_description """ @property # type: ignore @abstractclassmethod def severity(cls) -> str: # sourcery skip: instance-method-first-arg-name """ The rule severity """ def __init_subclass__(cls, *args: Any, **kwargs: Any) -> None: super().__init_subclass__(**kwargs) if cls.severity not in [None, "LOW", "MEDIUM", "HIGH"]: raise JinjaLinterError( f"Rule {cls.rule_id}: severity must be in [None, 'LOW', 'MEDIUM', 'HIGH'], {cls.severity} was provided" ) def __repr__(self) -> str: return f"{self.rule_id}: {self.description}" def to_rich(self) -> Text: """ Return a rich reprsentation of the rule, e.g.: S0 Jinja syntax should be correct (jinja-syntax-error) Where `S0` is in red and `(jinja-syntax-error)` in blue """ res = Text() res.append(f"{self.rule_id} ", "red") res.append(self.description) res.append(f" ({self.short_description})", "blue") return res def to_json(self) -> str: """Return a json representation of the rule""" return json.dumps( { "rule_id": self.rule_id, "short_description": self.short_description, "description": self.description, "severity": self.severity, "origin": self.origin, } ) @abstractmethod def checktext(self, filename: str, text: str) -> list[LinterError]: """This method is expected to be overriden by child classes""" @abstractmethod def checkline(self, filename: str, line: str, line_no: int) -> list[LinterError]: """This method is expected to be overriden by child classes""" def checkrule(self, file: dict[str, Any], text: str) -> list[LinterError]: """ Checks the string text against the current rule by calling either the checkline or checktext method depending on which one is implemented Args: file (string): file path of the file to be checked text (string): file text of the same file Returns: list: list of LinterError from issues in the given file """ errors: list[LinterError] = [] if not is_valid_file_type(file["path"]): logger.debug( "Skipping file %s. Linter does not support linting this file type", file ) return errors try: # First try with checktext results = self.checktext(file["path"], text) errors.extend(results) except NotImplementedError: # checkline it is for index, line in enumerate(text.split("\n")): # pylint: disable = fixme # FIXME - parsing jinja2 templates .. lines starting with `# # should probably still be parsed somewhow as these # are not comments. if line.lstrip().startswith("#"): continue results = self.checkline(file["path"], line, line_no=index + 1) errors.extend(results) # errors.append(LinterError(index + 1, line, file["path"], self)) return errors 07070100000016000081A400000000000000000000000164AD1D7A00000B56000000000000000000000000000000000000002500000000j2lint-1.1.0/j2lint/linter/runner.py"""runner.py - Class to run the rules collection for all the files. """ from __future__ import annotations from j2lint.logger import logger from j2lint.utils import get_file_type from .collection import RulesCollection from .error import LinterError class Runner: """Class to run the rules collection for all the files TODO: refactor - with this code it seems that files will always be a set of 1 file - indeed, a different Runner is created for each file in cli.py """ def __init__( self, collection: RulesCollection, file_name: str, checked_files: list[str], ) -> None: self.collection = collection self.files: set[tuple[str, str]] = set() if (file_type := get_file_type(file_name)) is not None: self.files.add((file_name, file_type)) self.checked_files = checked_files def is_already_checked(self, file_path: str) -> bool: """Returns true if the file is already checked once Args: file_path (string): file path Returns: bool: True if file is already checked once """ return file_path in self.checked_files def run(self) -> tuple[list[LinterError], list[LinterError]]: """Runs the lint rules collection on all the files Returns: tuple(list, list): a tuple containing the list of linting errors and the list of linting warnings found TODO - refactor this - it is quite weird to do the conversion from tuple to dict here maybe simply init with the dict """ file_dicts: list[dict[str, str]] = [] for index, file in enumerate(self.files): logger.debug("Running linting rules for %s", file) file_path = file[0] file_type = file[1] file_dict = {"path": file_path, "type": file_type} # pylint: disable = fixme # FIXME - as of now it seems that both next tests # will never occurs as self.files is always # a single file. # Skip already checked files if self.is_already_checked(file_path): continue # Skip duplicate files if file_dict in file_dicts[:index]: continue file_dicts.append(file_dict) errors: list[LinterError] = [] warnings: list[LinterError] = [] # pylint: disable = fixme # FIXME - if there are multiple files, errors and warnings are overwritten.. for file_dict in file_dicts: errors, warnings = self.collection.run(file_dict) # Update list of checked files self.checked_files.extend([file_dict["path"] for file_dict in file_dicts]) return errors, warnings 07070100000017000081A400000000000000000000000164AD1D7A0000033C000000000000000000000000000000000000001E00000000j2lint-1.1.0/j2lint/logger.py"""logger.py - Creates logger object. """ import logging from logging import handlers from rich.logging import RichHandler JINJA2_LOG_FILE = "jinja2-linter.log" logger = logging.getLogger("") def add_handler(log: logging.Logger, stream_handler: bool, log_level: int) -> None: """defined logging handlers""" log_format = logging.Formatter( "%(asctime)s - %(name)s - %(levelname)s - %(message)s" ) log.setLevel(log_level) if not stream_handler: file_handler = handlers.RotatingFileHandler( JINJA2_LOG_FILE, maxBytes=(1048576 * 5), backupCount=4 ) file_handler.setFormatter(log_format) log.addHandler(file_handler) else: console_handler = RichHandler() console_handler.setFormatter(log_format) log.addHandler(console_handler) 07070100000018000041ED00000000000000000000000264AD1D7A00000000000000000000000000000000000000000000001A00000000j2lint-1.1.0/j2lint/rules07070100000019000081A400000000000000000000000164AD1D7A00000000000000000000000000000000000000000000002600000000j2lint-1.1.0/j2lint/rules/__init__.py0707010000001A000081A400000000000000000000000164AD1D7A00000B2D000000000000000000000000000000000000003C00000000j2lint-1.1.0/j2lint/rules/jinja_operator_has_spaces_rule.py"""jinja_operator_has_spaces_rule.py - Rule class to check if operator has surrounding spaces. """ from __future__ import annotations import re from typing import Any from j2lint.linter.error import LinterError from j2lint.linter.rule import Rule class JinjaOperatorHasSpacesRule(Rule): """Rule class to check if jinja filter has surrounding spaces.""" rule_id = "S2" description = ( "When variables are used in combination with an operator, " "the operator should be enclosed by space: '{{ my_value | to_json }}'" ) short_description = "operator-enclosed-by-spaces" severity = "LOW" # pylint: disable=fixme # TODO make the regex detect the operator position operators = ["|", "+", "=="] regexes = [] for operator in operators: operator = "\\" + operator regex = ( r"({[{|%](.*?)([^ |^}]" + operator + ")(.*?)[}|%]})|({[{|%](.*?)(" + operator + r"[^ |^{])(.*?)[}|%]})|({[{|%](.*?)([^ |^}] \s+" + operator + ")(.*?)[}|%]})|({[{|%](.*?)(" + operator + r" \s+[^ |^{])(.*?)[}|%]})" ) regexes.append(re.compile(regex)) def __init__(self, ignore: bool = False, warn: list[Any] | None = None) -> None: super().__init__() def checktext(self, filename: str, text: str) -> list[LinterError]: raise NotImplementedError def checkline(self, filename: str, line: str, line_no: int) -> list[LinterError]: """Checks if the given line matches the error regex Args: line (string): a single line from the file Returns: list[LinterError]: the list of LinterError generated by this rule """ errors: list[LinterError] = [] # pylint: disable = fixme # TODO - refactor # This code removes any single quoted string # and any double quoted string to avoid # false positive on operators if "'" in line: regx = re.findall("'([^']*)'", line) for match in regx: line = line.replace(("'" + match + "'"), "''") if '"' in line: regx = re.findall('"([^"]*)"', line) for match in regx: line = line.replace(('"' + match + '"'), '""') issues = [ operator for regex, operator in zip(self.regexes, self.operators) if regex.search(line) ] errors.extend( LinterError( line_no, line, filename, self, f"The operator {issue} needs to be enclosed" " by a single space on each side", ) for issue in issues ) return errors 0707010000001B000081A400000000000000000000000164AD1D7A000005DE000000000000000000000000000000000000003C00000000j2lint-1.1.0/j2lint/rules/jinja_statement_delimiter_rule.py"""jinja_statement_delimiter_rule.py - Rule class to check if jinja delimiters are wrong. """ from __future__ import annotations from typing import Any from j2lint.linter.error import LinterError from j2lint.linter.rule import Rule from j2lint.utils import get_jinja_statements class JinjaStatementDelimiterRule(Rule): """Rule class to check if jinja delimiters are wrong.""" rule_id = "S6" description = "Jinja statements should not have {%- or {%+ or -%} as delimiters" short_description = "jinja-statements-delimiter" severity = "LOW" def __init__(self, ignore: bool = False, warn: list[Any] | None = None) -> None: super().__init__() def checktext(self, filename: str, text: str) -> list[LinterError]: raise NotImplementedError def checkline(self, filename: str, line: str, line_no: int) -> list[LinterError]: """Checks if the given line matches the wrong delimiters Args: line (string): a single line from the file Returns: list[LinterError]: the list of LinterError generated by this rule """ # pylint: disable=fixme # TODO think about a better error message that can identify characters statements = get_jinja_statements(line) return [ LinterError(line_no, line, filename, self) for statement in statements if statement[3] in ["{%-", "{%+"] or statement[4] == "-%}" ] 0707010000001C000081A400000000000000000000000164AD1D7A000005C5000000000000000000000000000000000000003D00000000j2lint-1.1.0/j2lint/rules/jinja_statement_has_spaces_rule.py"""jinja_statement_has_spaces_rule.py - Rule class to check if jinja statement has at least a single space surrounding the delimiter. """ from __future__ import annotations import re from typing import Any from j2lint.linter.error import LinterError from j2lint.linter.rule import Rule class JinjaStatementHasSpacesRule(Rule): """Rule class to check if jinja statement has at least a single space surrounding the delimiter. """ rule_id = "S4" description = "Jinja statement should have at least a single space after '{%' and a single space before '%}'" short_description = "jinja-statements-single-space" severity = "LOW" regex = re.compile(r"{%[^ \-\+]|{%[\-\+][^ ]|[^ \-\+]%}|[^ ][\-\+]%}") def __init__(self, ignore: bool = False, warn: list[Any] | None = None) -> None: super().__init__() def checktext(self, filename: str, text: str) -> list[LinterError]: raise NotImplementedError def checkline(self, filename: str, line: str, line_no: int) -> list[LinterError]: """Checks if the given line matches the error regex Args: line (string): a single line from the file Returns: list[LinterError]: the list of LinterError generated by this rule """ matches = self.regex.search(line) return [LinterError(line_no, line, filename, self)] if matches else [] 0707010000001D000081A400000000000000000000000164AD1D7A000008E5000000000000000000000000000000000000003D00000000j2lint-1.1.0/j2lint/rules/jinja_template_indentation_rule.py"""jinja_template_indentation_rule.py - Rule class to check the jinja statement indentation is correct. """ from __future__ import annotations from typing import Any from j2lint.linter.error import JinjaLinterError, LinterError from j2lint.linter.indenter.node import Node, NodeIndentationError from j2lint.linter.rule import Rule from j2lint.logger import logger from j2lint.utils import get_jinja_statements class JinjaTemplateIndentationRule(Rule): """Rule class to check the jinja statement indentation is correct.""" short_description = "jinja-statements-indentation" rule_id = "S3" description = ( "All J2 statements must be indented by 4 more spaces within jinja delimiter. " "To close a control, end tag must have same indentation level." ) severity = "HIGH" def __init__(self, ignore: bool = False, warn: list[Any] | None = None) -> None: super().__init__() def checktext(self, filename: str, text: str) -> list[LinterError]: """Checks if the given text has the error Args: file (string): file path text (string): entire text content of the file Returns: list: Returns list of error objects """ # Collect only Jinja Statements within delimiters {% and %} # and ignore the other statements lines = get_jinja_statements(text, indentation=True) # Build a tree out of Jinja Statements to get the expected # indentation level for each statement root = Node() node_errors: list[NodeIndentationError] = [] try: root.check_indentation(node_errors, lines, 0) # pylint: disable=fixme # TODO need to fix this index error in Node except (JinjaLinterError, IndexError) as exc: logger.error( "Indentation check failed for file %s: Error: %s", filename, str(exc), ) return [ LinterError(line_no, section, filename, self, message) for line_no, section, message in node_errors ] def checkline(self, filename: str, line: str, line_no: int) -> list[LinterError]: raise NotImplementedError 0707010000001E000081A400000000000000000000000164AD1D7A000004EC000000000000000000000000000000000000003900000000j2lint-1.1.0/j2lint/rules/jinja_template_no_tabs_rule.py"""jinja_template_no_tabs_rule.py - Rule class to check the file does not use tabs for indentation. """ from __future__ import annotations import re from typing import Any from j2lint.linter.error import LinterError from j2lint.linter.rule import Rule class JinjaTemplateNoTabsRule(Rule): """Rule class to check the file does not use tabs for indentation.""" rule_id = "S5" description = "Indentation should not use tabulation but 4 spaces" short_description = "jinja-statements-no-tabs" severity = "LOW" regex = re.compile(r"\t+") def __init__(self, ignore: bool = False, warn: list[Any] | None = None) -> None: super().__init__() def checktext(self, filename: str, text: str) -> list[LinterError]: raise NotImplementedError def checkline(self, filename: str, line: str, line_no: int) -> list[LinterError]: """Checks if the given line matches the error regex Args: line (string): a single line from the file Returns: list[LinterError]: the list of LinterError generated by this rule """ matches = self.regex.search(line) return [LinterError(line_no, line, filename, self)] if matches else [] 0707010000001F000081A400000000000000000000000164AD1D7A00000579000000000000000000000000000000000000004200000000j2lint-1.1.0/j2lint/rules/jinja_template_single_statement_rule.py"""jinja_template_single_statement_rule.py - Rule class to check if only a single jinja statement is present on each line. """ from __future__ import annotations from typing import Any from j2lint.linter.error import LinterError from j2lint.linter.rule import Rule from j2lint.utils import get_jinja_statements class JinjaTemplateSingleStatementRule(Rule): """Rule class to check if only a single jinja statement is present on each line. """ rule_id = "S7" description = "Jinja statements should be on separate lines" short_description = "single-statement-per-line" severity = "MEDIUM" def __init__(self, ignore: bool = False, warn: list[Any] | None = None) -> None: super().__init__() def checktext(self, filename: str, text: str) -> list[LinterError]: raise NotImplementedError def checkline(self, filename: str, line: str, line_no: int) -> list[LinterError]: """Checks if the given line matches the error regex Args: line (string): a single line from the file Returns: list[LinterError]: the list of LinterError generated by this rule """ return ( [LinterError(line_no, line, filename, self)] if len(get_jinja_statements(line)) > 1 else [] ) 07070100000020000081A400000000000000000000000164AD1D7A00000660000000000000000000000000000000000000003E00000000j2lint-1.1.0/j2lint/rules/jinja_template_syntax_error_rule.py"""jinja_template_syntax_error_rule.py - Rule class to check that file does not have jinja syntax errors. """ from __future__ import annotations from typing import Any import jinja2 from j2lint.linter.error import LinterError from j2lint.linter.rule import Rule class JinjaTemplateSyntaxErrorRule(Rule): """Rule class to check that file does not have jinja syntax errors.""" rule_id = "S0" description = "Jinja syntax should be correct" short_description = "jinja-syntax-error" severity = "HIGH" def __init__(self, ignore: bool = False, warn: list[Any] | None = None) -> None: super().__init__() def checktext(self, filename: str, text: str) -> list[LinterError]: """Checks if the given text has jinja syntax error Args: file (string): file path text (string): entire text content of the file Returns: list: Returns list of error objects """ result = [] env = jinja2.Environment( extensions=["jinja2.ext.do", "jinja2.ext.loopcontrols"] ) try: env.parse(text) except jinja2.TemplateSyntaxError as error: result.append( LinterError( error.lineno, text.split("\n")[error.lineno - 1], filename, self, error.message, ) ) return result def checkline(self, filename: str, line: str, line_no: int) -> list[LinterError]: raise NotImplementedError 07070100000021000081A400000000000000000000000164AD1D7A00000637000000000000000000000000000000000000003B00000000j2lint-1.1.0/j2lint/rules/jinja_variable_has_space_rule.py"""jinja_variable_has_space_rule.py - Rule class to check if jinja variables have single space between curly brackets and variable name. """ from __future__ import annotations import re from typing import Any from j2lint.linter.error import LinterError from j2lint.linter.rule import Rule class JinjaVariableHasSpaceRule(Rule): """Rule class to check if jinja variables have single space between curly brackets and variable name. """ rule_id = "S1" description = ( "A single space should be added between Jinja2 curly brackets " "and a variable name: {{ ethernet_interface }}" ) short_description = "single-space-decorator" severity = "LOW" regex = re.compile( r"{{[^ \-\+\d][^}]+}}|{{[-\+][^ ][^}]+}}|{{[^}]+[^ \-\+\d]}}|{{[^}]+[^ {][-\+\d]}}|{{ \s+[^ \-\+]}}|{{[^}]+[^ \-\+] \s+}}" ) def __init__(self, ignore: bool = False, warn: list[Any] | None = None) -> None: super().__init__() def checktext(self, filename: str, text: str) -> list[LinterError]: raise NotImplementedError def checkline(self, filename: str, line: str, line_no: int) -> list[LinterError]: """Checks if the given line matches the error regex Args: line (string): a single line from the file Returns: list[LinterError]: the list of LinterError generated by this rule """ matches = self.regex.search(line) return [LinterError(line_no, line, filename, self)] if matches else [] 07070100000022000081A400000000000000000000000164AD1D7A00000729000000000000000000000000000000000000003B00000000j2lint-1.1.0/j2lint/rules/jinja_variable_name_case_rule.py"""jinja_variable_name_case_rule.py - Rule class to check the variables use lower case. """ from __future__ import annotations import re from typing import Any from j2lint.linter.error import LinterError from j2lint.linter.rule import Rule from j2lint.utils import get_jinja_variables # pylint: disable=duplicate-code class JinjaVariableNameCaseRule(Rule): """Rule class to check the variables use lower case.""" rule_id = "V1" description = "All variables should use lower case: '{{ variable }}'" short_description = "jinja-variable-lower-case" severity = "LOW" regex = re.compile(r"([a-zA-Z0-9-_\"']*[A-Z][a-zA-Z0-9-_\"']*)") def __init__(self, ignore: bool = False, warn: list[Any] | None = None) -> None: super().__init__() def checktext(self, filename: str, text: str) -> list[LinterError]: raise NotImplementedError def checkline(self, filename: str, line: str, line_no: int) -> list[LinterError]: """ Checks if the given line matches the error regex, which matches variables with non lower case characters Args: line (string): a single line from the file Returns: list[LinterError]: the list of LinterError generated by this rule """ variables = get_jinja_variables(line) matches = [] for var in variables: matches = re.findall(self.regex, var) matches = [ match for match in matches if (match not in ["False", "True"]) and ("'" not in match) and ('"' not in match) ] return [ LinterError(line_no, line, filename, self, f"{self.description}: {match}") for match in matches ] 07070100000023000081A400000000000000000000000164AD1D7A0000071D000000000000000000000000000000000000003D00000000j2lint-1.1.0/j2lint/rules/jinja_variable_name_format_rule.py""" jinja_variable_name_format_rule.py - Rule class to check that variable names only use underscores. """ from __future__ import annotations import re from typing import Any from j2lint.linter.error import LinterError from j2lint.linter.rule import Rule from j2lint.utils import get_jinja_variables # pylint: disable=duplicate-code class JinjaVariableNameFormatRule(Rule): """Rule class to check that variable names only use underscores.""" rule_id = "V2" description = ( "If variable is multi-words, underscore `_` should be used " "as a separator: '{{ my_variable_name }}'" ) short_description = "jinja-variable-format" severity = "LOW" regex = re.compile(r"[a-zA-Z0-9-_\"']+[-][a-zA-Z0-9-_\"']+") def __init__(self, ignore: bool = False, warn: list[Any] | None = None) -> None: super().__init__() def checktext(self, filename: str, text: str) -> list[LinterError]: raise NotImplementedError def checkline(self, filename: str, line: str, line_no: int) -> list[LinterError]: """ Checks if the given line matches the error regex, which matches variables using `-` in their name Args: line (string): a single line from the file Returns: list[LinterError]: the list of LinterError generated by this rule """ variables = get_jinja_variables(line) matches = [] for var in variables: matches = re.findall(self.regex, var) matches = [ match for match in matches if ("'" not in match) and ('"' not in match) ] return [ LinterError(line_no, line, filename, self, f"{self.description}: {match}") for match in matches ] 07070100000024000081A400000000000000000000000164AD1D7A00002097000000000000000000000000000000000000001D00000000j2lint-1.1.0/j2lint/utils.py"""utils.py - Utility functions for jinja2 linter. """ from __future__ import annotations import glob import importlib.util import os import re from collections.abc import Generator, Iterable from typing import TYPE_CHECKING, Any, Tuple from j2lint.logger import logger if TYPE_CHECKING: from .linter.rule import Rule LANGUAGE_JINJA = "jinja" # Using Tuple from typing for 3.8 support # Statement type is a tuple # (line_without_delimiter, start_line, end_line, start_delimiter, end_delimiter) Statement = Tuple[str, int, int, str, str] def load_plugins(directory: str) -> list[Rule]: """Loads and executes all the Rule modules from the specified directory Args: directory (string): Loads the modules a directory Returns: list: List of rule classes """ result = [] file_handle = None for plugin_file in glob.glob(os.path.join(directory, "[A-Za-z_]*.py")): plugin_name = os.path.basename(plugin_file.replace(".py", "")) try: logger.debug("Loading plugin %s", plugin_name) spec = importlib.util.spec_from_file_location(plugin_name, plugin_file) if plugin_name != "__init__" and spec is not None: class_name = "".join( str(name).capitalize() for name in plugin_name.split("_") ) module = importlib.util.module_from_spec(spec) if spec.loader is not None: spec.loader.exec_module(module) obj = getattr(module, class_name)() result.append(obj) except AttributeError: logger.warning("Failed to load plugin %s", plugin_name) finally: if file_handle: file_handle.close() return result def is_valid_file_type(file_name: str) -> bool: """Checks if the file is a valid Jinja file Args: file_name (string): file path with extension Returns: boolean: True if file type is correct """ extension = os.path.splitext(file_name)[1].lower() return extension in [".jinja", ".jinja2", ".j2"] def get_file_type(file_name: str) -> str | None: """Returns file type as Jinja or None Args: file_name (string): file path with extension Returns: string: jinja or None TODO: this method and the previous one are redundant """ return LANGUAGE_JINJA if is_valid_file_type(file_name) else None def get_files(file_or_dir_names: list[str]) -> list[str]: """Get files from a directory recursively Args: file_or_dir_names (list): list of directories and files Returns: list: list of file paths """ file_paths: list[str] = [] if not isinstance(file_or_dir_names, (list, set)): raise TypeError( f"get_files expects a list or a set and got {file_or_dir_names}" ) for file_or_dir in file_or_dir_names: if os.path.isdir(file_or_dir): for root, _, files in os.walk(file_or_dir): for file in files: file_path = os.path.join(root, file) if get_file_type(file_path) == LANGUAGE_JINJA: file_paths.append(file_path) elif get_file_type(file_or_dir) == LANGUAGE_JINJA: file_paths.append(file_or_dir) logger.debug("Linting directory %s: files %s", file_or_dir_names, file_paths) return file_paths def flatten(nested_list: Iterable[Any]) -> Generator[Any, Any, Any]: """Flattens an iterable Args: nested_list (list): Nested list Returns: a generator that yields the elements of each object in the nested_list """ if not isinstance(nested_list, (list, tuple)): raise TypeError( f"flatten is expecting a list or tuple and received {nested_list}" ) for element in nested_list: if isinstance(element, Iterable) and not isinstance(element, (str, bytes)): yield from flatten(element) else: yield element def get_tuple( list_of_tuples: list[tuple[Any, ...]], item: Any ) -> tuple[Any, ...] | None: """Checks if an item is present in any of the tuples Args: list_of_tuples (list): list of tuples item (object): single object which can be in a tuple Returns: [tuple]: tuple if the item exists in any of the tuples """ return next((entry for entry in list_of_tuples if item in entry), None) def get_jinja_statements(text: str, indentation: bool = False) -> list[Statement]: """Gets jinja statements with {%[-/+] [-]%} delimiters The regex `regex_pattern` will return multiple groups when it matches Note that this is a multiline regex Args: text (string): multiline text to search the jinja statements in indentation (bool): Set to True if parsing for indentation, it will allow to retrieve multiple lines Example: For this given template: {# tcam-profile #} {% if switch.platform_settings.tcam_profile is arista.avd.defined %} tcam_profile: system: {{ switch.platform_settings.tcam_profile }} {% endif %} With indentation=True Found jinja statements [(' if switch.platform_settings.tcam_profile is arista.avd.defined ', 2, 2, '{%', '%}'), (' endif ', 5, 5, '{%', '%}')] With indentation=False Found jinja statements [] Found jinja statements [(' if switch.platform_settings.tcam_profile is arista.avd.defined ', 1, 1, '{%', '%}')] Found jinja statements [] Found jinja statements [] Found jinja statements [(' endif ', 1, 1, '{%', '%}')] Found jinja statements [] Returns: [list]: list of jinja statements # TODO - should probably return a JinjaStatement object.. """ statements: list[Statement] = [] count = 0 regex_pattern = re.compile("(\\{%[-|+]?)((.|\n)*?)([-]?\\%})", re.MULTILINE) newline_pattern = re.compile(r"\n") lines = text.split("\n") for match in regex_pattern.finditer(text): count += 1 start_line = len(newline_pattern.findall(text, 0, match.start(2))) + 1 end_line = len(newline_pattern.findall(text, 0, match.end(2))) + 1 if indentation and lines[start_line - 1].split()[0] not in ["{%", "{%-", "{%+"]: continue statements.append( ( str(match.group(2)), start_line, end_line, str(match.group(1)), str(match.group(4)), ) ) logger.debug("Found jinja statements %s", statements) return statements def delimit_jinja_statement(line: str, start: str = "{%", end: str = "%}") -> str: """Adds end delimiters for a jinja statement Args: line (string): text line Returns: [string]: jinja statement with jinja start and end delimiters """ return start + line + end def get_jinja_comments(text: str) -> list[str]: """Gets jinja comments Args: line (string): text to get jinja comments Returns: [list]: returns list of jinja comments """ regex_pattern = re.compile("(\\{#)((.|\n)*?)(\\#})", re.MULTILINE) return [line.group(2) for line in regex_pattern.finditer(text)] def get_jinja_variables(text: str) -> list[str]: """Gets jinja variables Args: line (string): text to get jinja variables Returns: [list]: returns list of jinja variables """ regex_pattern = regex_pattern = re.compile("(\\{{)((.|\n)*?)(\\}})", re.MULTILINE) return [line.group(2) for line in regex_pattern.finditer(text)] def is_rule_disabled(text: str, rule: Rule) -> bool: """Check if rule is disabled Args: text (string): text to check rule (Rule): Rule object Returns: [boolean]: True if rule is disabled """ comments = get_jinja_comments(text) regex = re.compile(r"j2lint\s*:\s*disable\s*=\s*([\w-]+)") for comment in comments: for line in regex.finditer(comment): if rule.short_description == line.group(1): return True if rule.rule_id == line.group(1): return True return False 07070100000025000081A400000000000000000000000164AD1D7A00000D1E000000000000000000000000000000000000001C00000000j2lint-1.1.0/pyproject.toml# pyproject.toml [build-system] requires = ["setuptools>=61.1.0", "wheel"] build-backend = "setuptools.build_meta" [project] name = "j2lint" version = "v1.1.0" description = "Command-line utility that validates jinja2 syntax according to Arista's AVD style guide." readme = "README.md" authors = [{ name = "Arista Ansible Team", email = "ansible@arista.com" }] license = { file = "LICENSE" } classifiers = [ "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", "Development Status :: 5 - Production/Stable", "Intended Audience :: Developers", "Programming Language :: Python", "Programming Language :: Python :: 3", "Topic :: Software Development :: Libraries :: Python Modules", "Topic :: Software Development :: Testing", ] keywords = ["j2lint", "linter", "jinja", "lint"] dependencies = [ "jinja2>=3.0", "rich>=12.4.4", ] requires-python = ">=3.8" [project.optional-dependencies] dev = [ "pre-commit", "bumpver", ] test = [ "pytest", "pytest-cov", ] lint = [ "black>=23.3.0", "isort[colors]>=5.12.0", "pylint>=2.15.9", "flake8>=4.0.1", ] type = [ "mypy==0.991", ] [project.urls] Homepage = "https://github.com/aristanetworks/j2lint.git" "Bug Tracker" = "https://github.com/aristanetworks/j2lint/issues" [project.scripts] j2lint = "j2lint.cli:run" [tool.bumpver] current_version = "v1.1.0" version_pattern = "vMAJOR.MINOR.PATCH" commit_message = "Chore: Version {old_version} -> {new_version}" commit = true # No Tag tag = false push = false [tool.bumpver.file_patterns] "pyproject.toml" = ['current_version = "{version}"', 'version = "{version}"'] "j2lint/__init__.py" = ["{version}"] "tests/test_cli.py" = ["{version}"] [tool.pylint.'MESSAGES CONTROL'] max-line-length = 160 [tool.tox] legacy_tox_ini = """ [tox] envlist = clean, py38 py39 py310, lint, type, report isolated_build = True [gh-actions] python = 3.8: py38 3.9: py39 3.10: py310, coverage, report [testenv] description = run the test driver with {basepython} extras = test commands = pytest {tty:--color=yes} [testenv:lint] description = check the code style extras = lint test commands = flake8 --max-line-length=160 --config=/dev/null j2lint flake8 --max-line-length=160 --config=/dev/null tests pylint j2lint pylint tests black --check --diff --color . isort --check --diff --color . [testenv:type] description = check the code type extras = type commands = mypy --config-file=pyproject.toml j2lint [testenv:clean] deps = coverage[toml] skip_install = true commands = coverage erase [testenv:report] deps = coverage[toml] commands = coverage report # add the following to make the report fail under some percentage # commands = coverage report --fail-under=80 depends = py310 """ [tool.pytest.ini_options] addopts = "-ra -q -s -vv --capture=tee-sys --cov --cov-append" log_level = "WARNING" log_cli = "True" [tool.coverage.run] source = ['j2lint'] omit = ["j2lint/__main__.py"] [tool.mypy] follow_imports = "skip" ignore_missing_imports = true warn_redundant_casts = true disallow_any_generics = true check_untyped_defs = true no_implicit_reexport = true strict_optional = true # for strict mypy: (this is the tricky one :-)) disallow_untyped_defs = true mypy_path = "j2lint" [tool.isort] profile = "black" 07070100000026000081A400000000000000000000000164AD1D7A00000000000000000000000000000000000000000000001700000000j2lint-1.1.0/setup.cfg07070100000027000041ED00000000000000000000000564AD1D7A00000000000000000000000000000000000000000000001300000000j2lint-1.1.0/tests07070100000028000081A400000000000000000000000164AD1D7A00000000000000000000000000000000000000000000001F00000000j2lint-1.1.0/tests/__init__.py07070100000029000081A400000000000000000000000164AD1D7A00001A4E000000000000000000000000000000000000001F00000000j2lint-1.1.0/tests/conftest.py""" content of conftest.py """ import logging import pathlib from argparse import Namespace from unittest.mock import create_autospec import pytest from j2lint.cli import create_parser from j2lint.linter.collection import DEFAULT_RULE_DIR, RulesCollection from j2lint.linter.error import LinterError from j2lint.linter.rule import Rule from j2lint.linter.runner import Runner CONTENT = "content" # Disabling redefined-outer-name as a solution for pylint # cf https://docs.pytest.org/en/stable/reference/reference.html#pytest-fixture # pylint: disable=fixme, redefined-outer-name # TODO - proper way to compare LinterError following: # https://docs.pytest.org/en/7.1.x/how-to/assert.html#defining-your-own-explanation-for-failed-assertions class TestRule(Rule): """ TestRule class for tests """ rule_id = "TT" description = "test" short_description = "test" severity = "LOW" def checktext(self, filename, text): pass def checkline(self, filename, line, line_no): pass @pytest.fixture def collection(): """ Return the collection with the default rules """ return RulesCollection.create_from_directory(DEFAULT_RULE_DIR, [], []) @pytest.fixture def make_rules(): """ Return a Rule factory that takes one argument `count` and returns count rules following the pattern where `i` is the index rule_id = Ti description = test rule i short_description = test-rule-i severity in [LOW, MEDIUM, HIGH] based on i % 3 The factory can then be invoked as follow: def test_blah(makes_rules): rules = make_rules(5) # do stuff with rules """ def __make_n_rules(count): def get_severity(integer: int): return ( "LOW" if integer % 3 == 0 else ("MEDIUM" if integer % 3 == 1 else "HIGH") ) rules = [] for i in range(count): r_obj = TestRule() r_obj.rule_id = f"T{i}" r_obj.description = f"test rule {i}" r_obj.short_description = f"test-rule-{i}" r_obj.severity = get_severity(i) rules.append(r_obj) return rules return __make_n_rules @pytest.fixture def test_rule(make_rules): """ return a Rule object to use in test from the make_rules - it will have rule_id = T0 description = test rule 0 short_description = test-rule-0 severity = LOW """ yield make_rules(1)[0] @pytest.fixture def test_other_rule(make_rules): """ return the second Rule object to use in test from the make_rules - it will have rule_id = T1 description = test rule 1 short_description = test-rule-1 severity = MEDIUM """ yield make_rules(2)[1] @pytest.fixture def make_issues(make_rules): """ Returns a factory that generates `count` issues and return them as a list The fixture invokes `make_rules` first with count and every LinterError generated is using the rule of same index. The line number is index + 1 so issue 0 will have line number 1 the line content is always "dummy" and the filename "dummy.j2" The factory can then be invoked as follow: def test_blah(makes_issues): issues = make_issues(5) # do stuff with issues """ def __make_n_issues(count): issues = [] rules = make_rules(count) for i in range(count): issues.append(LinterError(i + 1, "dummy", "dummy.j2", rules[i])) return issues return __make_n_issues @pytest.fixture def make_issue_from_rule(): """ Returns a factory that generates an issue based on a Rule object it uses line 42, the line content is "dummy" and the filename is "dummy.j2" """ def __make_issue_from_rule(rule): yield LinterError(42, "dummy", "dummy.j2", rule) return __make_issue_from_rule @pytest.fixture def test_issue(make_issues): """ Get the first issue from the make_issues factory Note: it will use rule T0 as per design """ yield make_issues(1)[0] @pytest.fixture def test_collection(test_rule): """ test_collection using one rule `test_rule` """ collection = RulesCollection() collection.extend([test_rule]) yield collection @pytest.fixture def test_runner(test_collection): """ Fixture to get a test runner using the test_collection """ yield Runner(test_collection, "test.j2", checked_files=[]) @pytest.fixture def j2lint_usage_string(): """ Fixture to get the help generated by argparse """ yield create_parser().format_help() @pytest.fixture() def template_tmp_dir(tmp_path_factory): """ Create a tmp directory with multiple files and hidden files """ tmp_dir = pathlib.Path(__file__).parent / "tmp" # Hacking it # https://stackoverflow.com/questions/40566968/how-to-dynamically-change-pytests-tmpdir-base-directory # + # https://docs.pytest.org/en/7.1.x/_modules/_pytest/tmpdir.html # Using _given_basetemp to trigger creation # pylint: disable=protected-access tmp_path_factory._given_basetemp = tmp_dir rules = tmp_path_factory.mktemp("rules", numbered=False) rules_subdir = rules / "rules_subdir" rules_subdir.mkdir() # no clue if we are suppose to find these rules_hidden_subdir = rules / ".rules_hidden_subdir" rules_hidden_subdir.mkdir() # in each directory add one matching file and one none matching rules_j2 = rules / "rules.j2" rules_j2.write_text(CONTENT) rules_txt = rules / "rules.txt" rules_txt.write_text(CONTENT) rules_subdir_j2 = rules_subdir / "rules.j2" rules_subdir_j2.write_text(CONTENT) rules_subdir_txt = rules_subdir / "rules.txt" rules_subdir_txt.write_text(CONTENT) rules_hidden_subdir_j2 = rules_hidden_subdir / "rules.j2" rules_hidden_subdir_j2.write_text(CONTENT) rules_hidden_subdir_txt = rules_hidden_subdir / "rules.txt" rules_hidden_subdir_txt.write_text(CONTENT) yield [str(rules)] @pytest.fixture def default_namespace(): """ Default ArgPase namespace for j2lint """ return Namespace( files=[], ignore=[], warn=[], list=False, rules_dir=[DEFAULT_RULE_DIR], verbose=False, debug=False, json=False, stdin=False, log=False, version=False, stdout=False, ) @pytest.fixture def logger(): """ Return a MagicMock object with the spec of logging.Logger """ return create_autospec(logging.Logger) 0707010000002A000041ED00000000000000000000000264AD1D7A00000000000000000000000000000000000000000000001800000000j2lint-1.1.0/tests/data0707010000002B000081A400000000000000000000000164AD1D7A00000000000000000000000000000000000000000000002000000000j2lint-1.1.0/tests/data/test.j20707010000002C000081A400000000000000000000000164AD1D7A00002E38000000000000000000000000000000000000001F00000000j2lint-1.1.0/tests/test_cli.py""" Tests for j2lint.cli.py """ import logging import os import re from argparse import Namespace from unittest.mock import patch import pytest from rich.console import ConsoleDimensions from j2lint.cli import ( CONSOLE, create_parser, print_json_output, print_string_output, run, sort_issues, ) from .utils import ( NO_ERROR_NO_WARNING_JSON, ONE_ERROR, ONE_ERROR_JSON, ONE_ERROR_ONE_WARNING_JSON, ONE_ERROR_ONE_WARNING_VERBOSE, ONE_ERROR_REGEX, ONE_WARNING_VERBOSE, does_not_raise, j2lint_default_rules_string, ) # pylint: disable=fixme, too-many-arguments # Fixed size console for tests output CONSOLE.size = ConsoleDimensions(width=80, height=74) @pytest.mark.parametrize( "argv, namespace_modifications", [ (pytest.param([], {}, id="default")), pytest.param( ["--log", "--stdout", "-j", "-d", "-v", "--stdin", "--version"], { "debug": True, "stdout": True, "stdin": True, "version": True, "log": True, "json": True, "verbose": True, }, id="set all debug flags", ), ], ) def test_create_parser(default_namespace, argv, namespace_modifications): """ Test j2lint.cli.create_parser the namespace_modifications is a dictionnary where key is one of the keys in the namespace and value is the value it should be overwritten to. This test only verifies that given a set of arguments on the cli, the parser returns the correct values in the Namespace """ expected_namespace = default_namespace for key, value in namespace_modifications.items(): setattr(expected_namespace, key, value) parser = create_parser() options = parser.parse_args(argv) assert options == expected_namespace @pytest.mark.parametrize( "number_issues, issues_modifications, expected_sorted_issues_ids", [ (0, {}, []), (1, {}, [("dummy.j2", "T0", 1, "test-rule-0")]), pytest.param( 2, {}, [ ("dummy.j2", "T0", 1, "test-rule-0"), ("dummy.j2", "T1", 2, "test-rule-1"), ], id="sort-on-linenumber", ), pytest.param( 2, {2: {"filename": "aaa.j2"}}, [("aaa.j2", "T1", 2, "test-rule-1"), ("dummy.j2", "T0", 1, "test-rule-0")], id="sort-on-filename", ), pytest.param( 2, {2: {"rule": {"rule_id": "AA"}, "line_number": 1}}, [ ("dummy.j2", "AA", 1, "test-rule-1"), ("dummy.j2", "T0", 1, "test-rule-0"), ], id="sort-on-rule-id", ), ], ) def test_sort_issues( make_issues, number_issues, issues_modifications, expected_sorted_issues_ids ): """ Test j2lint.cli.sort_issues the issues_modificartions is a dictionary that has the following structure: { <issue-index>: { <key>: <desired_value } the test will go over these modifications and apply them to the appropriate issues, apply the sort_issues method and verifies the ordering is correct """ issues = make_issues(number_issues) # In the next step we apply modifications on the generated LinterErrors # if required for index, modification in issues_modifications.items(): for key, value in modification.items(): if isinstance(value, dict): nested_obj = getattr(issues[index - 1], key) for n_key, n_value in value.items(): setattr(nested_obj, n_key, n_value) else: setattr(issues[index - 1], key, value) sorted_issues = sort_issues(issues) sorted_issues_ids = [ ( issue.filename, issue.rule.rule_id, issue.line_number, issue.rule.short_description, ) for issue in sorted_issues ] assert sorted_issues_ids == expected_sorted_issues_ids @pytest.mark.parametrize( "options, number_errors, number_warnings, expected_output, expected_stdout", [ pytest.param( Namespace(verbose=False), 0, 0, (0, 0), "", id="No issue - cli", ), pytest.param( Namespace(verbose=True), 0, 0, (0, 0), "Linting complete. No problems found!\n", id="No issue - cli", ), pytest.param( Namespace(verbose=False), 1, 0, (1, 0), ONE_ERROR, id="One error - cli", ), pytest.param( Namespace(verbose=True), 0, 1, (0, 1), ONE_WARNING_VERBOSE, id="One warning - cli", ), ], ) def test_print_string_output( capsys, make_issues, options, number_errors, number_warnings, expected_output, expected_stdout, ): """ Test j2lint.cli.print_string_output """ errors = {"dummy.j2": make_issues(number_errors)} warnings = {"dummy.j2": make_issues(number_warnings)} total_errors, total_warnings = print_string_output( errors, warnings, options.verbose ) assert total_errors == expected_output[0] assert total_warnings == expected_output[1] captured = capsys.readouterr() assert captured.out == expected_stdout @pytest.mark.parametrize( "number_errors, number_warnings, expected_output, expected_stdout", [ pytest.param( 0, 0, (0, 0), NO_ERROR_NO_WARNING_JSON, id="No issue - json", ), pytest.param( 1, 0, (1, 0), ONE_ERROR_JSON, id="one error - json", ), pytest.param( 1, 1, (1, 1), ONE_ERROR_ONE_WARNING_JSON, id="one error and one warning - json", ), ], ) def test_print_json_output( capsys, make_issues, number_errors, number_warnings, expected_output, expected_stdout, ): """ Test j2lint.cli.print_json_output """ errors = {"ERRORS": make_issues(number_errors)} warnings = {"WARNINGS": make_issues(number_warnings)} total_errors, total_warnings = print_json_output(errors, warnings) assert total_errors == expected_output[0] assert total_warnings == expected_output[1] captured = capsys.readouterr() print(captured) assert captured.out == expected_stdout @pytest.mark.parametrize( "argv, expected_stdout, expected_stderr, expected_exit_code, expected_raise, number_errors, number_warnings", [ pytest.param([], "", "HELP", 1, does_not_raise(), 0, 0, id="no input"), pytest.param(["-h"], "HELP", "", 0, pytest.raises(SystemExit), 0, 0, id="help"), pytest.param( ["--version"], "Jinja2-Linter Version v1.1.0\n", "", 0, does_not_raise(), 0, 0, id="version", ), pytest.param( ["--log", "tests/data/test.j2"], "", "", 0, does_not_raise(), 0, 0, id="log level INFO", ), pytest.param( ["-v", "tests/data/test.j2"], ONE_ERROR_ONE_WARNING_VERBOSE, "", 2, does_not_raise(), 1, 1, id="one error and one warning - verbose", ), pytest.param( ["-j", "tests/data/test.j2"], ONE_ERROR_ONE_WARNING_JSON, "", 2, does_not_raise(), 1, 1, id="one error and one warning - json", ), pytest.param( ["-l"], "DEFAULT_RULES", "", 0, does_not_raise(), 0, 0, id="list rules", ), pytest.param( ["--stdout", "--debug", "tests/data/test.j2"], "", "", 0, does_not_raise(), 0, 0, id="log level DEBUG", ), pytest.param( ["--stdout", "tests/data/test.j2"], "", "", 0, does_not_raise(), 0, 0, id="o / stdout", ), ], ) def test_run( capsys, caplog, j2lint_usage_string, make_issues, argv, expected_stdout, expected_stderr, expected_exit_code, expected_raise, number_errors, number_warnings, ): """ Test the j2lint.cli.run method This test is a bit too complex and should probably be splitted out to test various functionalities the call is to test the various options of the main entry point, patching away inner methods when required. The id of the tests explains the intention. """ if "-o" in argv or "--stdout" in argv: caplog.set_level(logging.INFO) if "-d" in argv or "--debug" in argv: caplog.set_level(logging.DEBUG) # TODO this method needs to be split a bit as it has # too many responsibility if expected_stdout == "HELP": expected_stdout = j2lint_usage_string if expected_stdout == "DEFAULT_RULES": expected_stdout = j2lint_default_rules_string() if expected_stderr == "HELP": expected_stderr = j2lint_usage_string with expected_raise: with patch("j2lint.cli.Runner.run") as mocked_runner_run, patch( "logging.disable" ): mocked_runner_run.return_value = ( make_issues(number_errors), make_issues(number_warnings), ) run_return_value = run(argv) captured = capsys.readouterr() if "-o" not in argv and "--stdout" not in argv: assert str(captured.out) == expected_stdout assert captured.out == expected_stdout # Hmm - WHY - need to find why failing with stdout assert captured.err == expected_stderr else: assert expected_stdout in captured.out assert run_return_value == expected_exit_code if ("-o" in argv or "--stdout" in argv) and ( "-d" in argv or "--debug" in argv ): assert "DEBUG" in [record.levelname for record in caplog.records] def test_run_stdin(capsys): """ Test j2lint.cli.run when using stdin Note that the code is checking that this is not run from a tty A solution to run is something like: ``` cat myfile.j2 | j2lint --stdin ``` In this test, the isatty answer is mocked. """ with patch("sys.stdin") as patched_stdin, patch( "os.unlink", side_effect=os.unlink ) as mocked_os_unlink, patch("logging.disable"): patched_stdin.isatty.return_value = False patched_stdin.read.return_value = "{%set test=42 %}" run_return_value = run(["--log", "--stdin"]) patched_stdin.isatty.assert_called_once() captured = capsys.readouterr() normalized_string = " ".join(captured.out.split()) matches = re.search( ONE_ERROR_REGEX, normalized_string, re.MULTILINE, ) assert matches is not None mocked_os_unlink.assert_called_with(matches.groups()[0]) assert os.path.exists(matches.groups()[0]) is False assert run_return_value == 2 0707010000002D000041ED00000000000000000000000464AD1D7A00000000000000000000000000000000000000000000001F00000000j2lint-1.1.0/tests/test_linter0707010000002E000041ED00000000000000000000000264AD1D7A00000000000000000000000000000000000000000000002400000000j2lint-1.1.0/tests/test_linter/data0707010000002F000081A400000000000000000000000164AD1D7A00000027000000000000000000000000000000000000003600000000j2lint-1.1.0/tests/test_linter/data/disable-rule-3.j2{# j2lint: disable=test-rule-3 #} blah 07070100000030000081A400000000000000000000000164AD1D7A00000000000000000000000000000000000000000000002C00000000j2lint-1.1.0/tests/test_linter/data/test.j207070100000031000081A400000000000000000000000164AD1D7A00000000000000000000000000000000000000000000002D00000000j2lint-1.1.0/tests/test_linter/data/test.txt07070100000032000081A400000000000000000000000164AD1D7A0000185D000000000000000000000000000000000000003200000000j2lint-1.1.0/tests/test_linter/test_collection.py""" Tests for j2lint.linter.collection.py """ import logging import pathlib from unittest import mock import pytest from j2lint.linter.collection import RulesCollection from j2lint.rules.jinja_operator_has_spaces_rule import JinjaOperatorHasSpacesRule from j2lint.rules.jinja_statement_delimiter_rule import JinjaStatementDelimiterRule from j2lint.rules.jinja_template_syntax_error_rule import JinjaTemplateSyntaxErrorRule TEST_DATA_DIR = pathlib.Path(__file__).parent / "data" class TestRulesCollection: def test__len__(self, test_collection): """ Test the RuleCollection __len__ method """ assert len(test_collection) == 1 def test__iter__(self): """ Test the RuleCollection __iter__ method """ fake_rules = [1, 2, 3] collection = RulesCollection() assert len(collection) == 0 collection.rules = fake_rules for index, rule in enumerate(collection): assert rule == fake_rules[index] def test_extend(self): """ Test the RuleCollection.extend method """ fake_rules = [1, 2, 3] collection = RulesCollection() assert len(collection) == 0 collection.rules.extend(fake_rules) assert len(collection) == len(fake_rules) collection.rules.extend(fake_rules) assert len(collection) == 2 * len(fake_rules) assert collection.rules == fake_rules + fake_rules @pytest.mark.parametrize( "file_dict, expected_results, verify_logs", [ pytest.param( {"path": "dummy.j2"}, ([], []), False, id="non existing file", ), pytest.param( {"path": f"{TEST_DATA_DIR}/disable-rule-3.j2"}, ( [ ("T0", "test-rule-0"), ("T4", "test-rule-4"), ], [ ("T1", "test-rule-1"), ], ), True, id="Disabled Rule 3", ), ], ) def test_run( self, caplog, test_collection, make_rules, make_issue_from_rule, file_dict, expected_results, verify_logs, ): """ Generate a collection with 5 rules with: * rule number 1 being a warning * rule number 2 being ignored * rule number 3 being disabled in the test file for test number 2 """ caplog.set_level(logging.DEBUG) rules = make_rules(5) # make rule number 1 be a warning rules[1].warn.append(rules[1]) rules[2].ignore = True test_collection.rules = rules def checks_side_effect(self, file_dict, text): return make_issue_from_rule(self) with mock.patch( "j2lint.linter.rule.Rule.checkrule", side_effect=checks_side_effect, autospec=True, ): errors, warnings = test_collection.run(file_dict) error_tuples = [ (error.rule.rule_id, error.rule.short_description) for error in errors ] warning_tuples = [ (warning.rule.rule_id, warning.rule.short_description) for warning in warnings ] assert error_tuples == expected_results[0] assert warning_tuples == expected_results[1] if verify_logs: # True for every test that goes beyond the file do not exist assert any("Ignoring rule T2" in message for message in caplog.messages) # True for the test file assert any( "Skipping linting rule T3" in message for message in caplog.messages ) def test__repr__(self, test_collection, test_other_rule): """ Test the RuleCollection.extend method """ assert str(test_collection) == "Origin: BUILT-IN\nT0: test rule 0" test_other_rule.origin = "DUMMY" test_collection.extend([test_other_rule]) assert ( str(test_collection) == "Origin: BUILT-IN\nT0: test rule 0\nOrigin: DUMMY\nT1: test rule 1" ) @pytest.mark.parametrize( "ignore_rules, warn_rules", [ pytest.param( ["S0", "operator-enclosed-by-spaces"], [], id="no_warn_no_ignore" ), pytest.param( [], ["S0", "operator-enclosed-by-spaces"], id="warn_no_ignore" ), pytest.param(["S0"], ["operator-enclosed-by-spaces"], id="ignore_no_warn"), pytest.param([], ["S42"], id="ignore_absent_rule"), pytest.param(["S42"], [], id="warn_absent_rule"), ], ) def test_create_from_directory(self, ignore_rules, warn_rules): """ Test the RuleCollection.create_from_directory class method In this tests, the return of load_plugins are mocked. consequently, a dummy path can be used for rules_dir """ with mock.patch("j2lint.linter.collection.load_plugins") as mocked_load_plugins: # importing 3 rules for the need of the tests # note that current pylint branch is returning classes # not instances.. mocked_load_plugins.return_value = [ JinjaStatementDelimiterRule(), JinjaOperatorHasSpacesRule(), JinjaTemplateSyntaxErrorRule(), ] collection = RulesCollection.create_from_directory( "dummy", ignore_rules, warn_rules ) mocked_load_plugins.assert_called_once_with("dummy") assert collection.rules == mocked_load_plugins.return_value for rule in collection.rules: if ( rule.rule_id in ignore_rules or rule.short_description in ignore_rules ): assert rule.ignore is True if rule.rule_id in warn_rules or rule.short_description in warn_rules: assert rule in rule.warn 07070100000033000081A400000000000000000000000164AD1D7A00000371000000000000000000000000000000000000002D00000000j2lint-1.1.0/tests/test_linter/test_error.py""" Tests for j2lint.linter.error.py """ import pytest RULE_TEXT_OUTPUT = ( "Linting rule: T0\n" "Rule description: test rule 0\n" "Error line: dummy.j2:1 dummy\n" "Error message: test rule 0\n" ) RULE_JSON_OUTPUT = ( '{"id": "T0", "message": "test rule 0", ' '"filename": "dummy.j2", "line_number": 1, ' '"line": "dummy", "severity": "LOW"}' ) @pytest.mark.parametrize( "verbose, expected", [ (False, "dummy.j2:1 test rule 0 (test-rule-0)"), ( True, RULE_TEXT_OUTPUT, ), ], ) def test_to_rich(test_issue, verbose, expected): """ Test the Rich string formats for LinterError """ assert str(test_issue.to_rich(verbose)) == expected def test_to_json(test_issue): """ Test the json format for LinterError """ assert test_issue.to_json() == RULE_JSON_OUTPUT 07070100000034000041ED00000000000000000000000264AD1D7A00000000000000000000000000000000000000000000002D00000000j2lint-1.1.0/tests/test_linter/test_indenter07070100000035000081A400000000000000000000000164AD1D7A000003B3000000000000000000000000000000000000003A00000000j2lint-1.1.0/tests/test_linter/test_indenter/test_node.py""" Tests for j2lint.linter.node.py """ import pytest from j2lint.linter.indenter.node import Node class TestNode: @pytest.mark.skip("No need to test this") def test_create_node(self): """ """ # TODO - why is it not an __init__ method??? pass def test_create_indentation_error(self): """ Test the Node.create_indentation_error method """ line = ( " if switch.platform_settings.tcam_profile is arista.avd.defined ", 2, 2, "{%", "%}", ) root = Node() node = root.create_node(line, 2) print(node.statement) indentation_error = node.create_indentation_error(node, "test") print(type(indentation_error)) assert indentation_error == ( 2, "{% if switch.platform_settings.tcam_profile is arista.avd.defined %}", "test", ) 07070100000036000081A400000000000000000000000164AD1D7A000006EC000000000000000000000000000000000000003F00000000j2lint-1.1.0/tests/test_linter/test_indenter/test_statement.py""" Tests for j2lint.linter.indenter.statement.py To understand the values given to this class it is paramount to read j2lint.utils.get_jinja_statement as this is the path in the code that will parse a file content and return a list of statements - abusively called lines """ import pytest from j2lint.linter.indenter.statement import JinjaStatement class TestJinjaStatement: @pytest.mark.parametrize( "line, expected_statement", [ pytest.param( ( " if switch.platform_settings.tcam_profile is arista.avd.defined ", 2, 2, "{%", "%}", ), { "begin": 1, "words": [ "if", "switch.platform_settings.tcam_profile", "is", "arista.avd.defined", ], "start_line_no": 2, "end_line_no": 2, "start_delimiter": "{%", "end_delimiter": "%}", }, id="working statement", ), # this next one is failing because we never expect to be called # with an empty list probably should be caught in __init__ # if we even keep the class.. # pytest.param("[]", {}, id="empty_statement"), ], ) def test_jinja_statement(self, line, expected_statement): """ """ # TODO - why is it not an __init__ method??? statement = JinjaStatement(line) for key, value in expected_statement.items(): assert statement.__getattribute__(key) == value 07070100000037000081A400000000000000000000000164AD1D7A00000D89000000000000000000000000000000000000002C00000000j2lint-1.1.0/tests/test_linter/test_rule.py""" Tests for j2lint.linter.rule.py """ import logging import pathlib import pytest TEST_DATA_DIR = pathlib.Path(__file__).parent / "data" class TestRule: def test__repr__(self, test_rule): """ Test the Rule __repr__ format """ assert str(test_rule) == "T0: test rule 0" @pytest.mark.skip("This method will be removed and is tested through other methods") def test_is_valid_language(self, test_rule, file): """ """ @pytest.mark.parametrize( "checktext, checkline, file_path, expected_errors_ids, expected_logs", [ pytest.param( None, None, {"path": f"{TEST_DATA_DIR}/test.txt"}, [], [ ( "root", logging.DEBUG, f"Skipping file {{'path': '{TEST_DATA_DIR}/test.txt'}}. Linter does not support linting this file type", ) ], id="file is wrong type", ), pytest.param( None, 0, {"path": f"{TEST_DATA_DIR}/test.j2"}, [], [], id="no error", ), pytest.param( None, 1, {"path": f"{TEST_DATA_DIR}/test.j2"}, [("T0", 42)], [], id="checkline rule error", ), pytest.param( 2, None, {"path": "tests/test_linter/data/test.j2"}, [("T0", 42), ("T0", 42)], [], id="checktext rule error", ), ], ) def test_checkrule( self, caplog, test_rule, make_issue_from_rule, checktext, checkline, file_path, expected_errors_ids, expected_logs, ): """ Test the Rule.checkrule method """ def raise_NotImplementedError(*args, **kwargs): raise NotImplementedError def return_empty_list(*args, **kwargs): return [] caplog.set_level(logging.DEBUG) # Build checktext and checkline if checktext is None: test_rule.checktext = raise_NotImplementedError elif checktext == 0: test_rule.checktext = return_empty_list else: # checktext > 0 test_rule.checktext = lambda x, y: [ issue for i in range(checktext) for issue in make_issue_from_rule(test_rule) ] if checkline is None: test_rule.checkline = raise_NotImplementedError elif checkline == 0: test_rule.checkline = return_empty_list else: # checkline > 0 test_rule.checkline = lambda x, y, line_no=0: [ issue for i in range(checkline) for issue in make_issue_from_rule(test_rule) ] with open(file_path["path"], "r", encoding="utf-8") as file_d: errors = test_rule.checkrule(file_path, file_d.read()) print(errors) errors_ids = [(error.rule.rule_id, error.line_number) for error in errors] assert errors_ids == expected_errors_ids assert caplog.record_tuples == expected_logs 07070100000038000081A400000000000000000000000164AD1D7A000007A4000000000000000000000000000000000000002E00000000j2lint-1.1.0/tests/test_linter/test_runner.py""" Tests for j2lint.linter.runner.py """ from unittest import mock import pytest from j2lint.utils import get_file_type class TestRunner: @pytest.mark.parametrize( "file_path, checked_files, expected", [ ("test.j2", [], False), ("test.j2", ["other.j2"], False), ("test.j2", ["test.j2"], True), ], ) def test_is_already_checked(self, test_runner, file_path, checked_files, expected): """ test Runner.is_already_checked method """ test_runner.checked_files = checked_files assert test_runner.is_already_checked(file_path) == expected # FIXME: refactor the code and augment the tests.. # today if we give multiple files to a runner # only the errors, warnings of the last one will # be kept.. @pytest.mark.parametrize( "runner_files", [ ([]), (["test.j2"]), (["test.j2", "test2.j2"]), (["test.txt"]), ], ) def test_run(self, test_runner, runner_files): """ test Runner.run method For now only testing with one input file given that this is how the package is working. This test is "bad" for now. """ test_runner.files = {(file, get_file_type(file)) for file in runner_files} # Fake return with mock.patch( "j2lint.linter.collection.RulesCollection.run" ) as patched_collection_run: patched_collection_run.return_value = ([], []) result = test_runner.run() assert len(runner_files) == patched_collection_run.call_count assert result == ([], []) for file in test_runner.files: patched_collection_run.assert_any_call( {"path": file[0], "type": file[1]} ) assert file[0] in test_runner.checked_files 07070100000039000081A400000000000000000000000164AD1D7A000003E7000000000000000000000000000000000000002200000000j2lint-1.1.0/tests/test_logger.py""" Tests for j2lint.logger.py """ import logging import sys from logging import handlers import pytest from rich.logging import RichHandler from j2lint.logger import add_handler @pytest.mark.parametrize( "stream_handler, log_level", [(False, logging.DEBUG), (True, logging.ERROR)] ) def test_add_handler(logger, stream_handler, log_level): """ Test j2lint.logger.add_handler Verify that the correct type of handler is added """ add_handler(logger, stream_handler, log_level) logger.setLevel.assert_called_once_with(log_level) logger.addHandler.assert_called_once() # call_args.args was introduced in Python 3.8 if sys.version_info >= (3, 8): handler_arg = logger.addHandler.call_args.args[0] elif sys.version_info >= (3, 6): handler_arg = logger.addHandler.call_args[0][0] if not stream_handler: assert isinstance(handler_arg, handlers.RotatingFileHandler) else: assert isinstance(handler_arg, RichHandler) 0707010000003A000041ED00000000000000000000000364AD1D7A00000000000000000000000000000000000000000000001E00000000j2lint-1.1.0/tests/test_rules0707010000003B000041ED00000000000000000000000264AD1D7A00000000000000000000000000000000000000000000002300000000j2lint-1.1.0/tests/test_rules/data0707010000003C000081A400000000000000000000000164AD1D7A00000135000000000000000000000000000000000000004500000000j2lint-1.1.0/tests/test_rules/data/jinja_operator_has_spaces_rule.j2{{ test|list }} {{ test| list }} {{ test |list }} {{ test | list }} {# double error in same line #} {{ test+ blah| list }} {# double quotes #} {{ "test" |list }} {# single quotes #} {{ 'test' |list }} {# the two next lines should not trigger #} {{ test | replace('|', '\|') }} {{ test | replace("|", "\|") }} 0707010000003D000081A400000000000000000000000164AD1D7A00000051000000000000000000000000000000000000004500000000j2lint-1.1.0/tests/test_rules/data/jinja_statement_delimiter_rule.j2{%+ if test %} {% endif %} {%- if test %} {% endif %} {% if test -%} {% endif %} 0707010000003E000081A400000000000000000000000164AD1D7A00000032000000000000000000000000000000000000004600000000j2lint-1.1.0/tests/test_rules/data/jinja_statement_has_spaces_rule.j2{%set test=42%} {% set test=42%} {%set test=42 %} 0707010000003F000081A400000000000000000000000164AD1D7A0000002C000000000000000000000000000000000000005100000000j2lint-1.1.0/tests/test_rules/data/jinja_template_indentation_rule.IndexError.j27{% if test %} {% set tets=42%} {% endif %} 07070100000040000081A400000000000000000000000164AD1D7A0000001B000000000000000000000000000000000000005700000000j2lint-1.1.0/tests/test_rules/data/jinja_template_indentation_rule.JinjaLinterError.j2{% if test %} {% endfor %} 07070100000041000081A400000000000000000000000164AD1D7A00000056000000000000000000000000000000000000004600000000j2lint-1.1.0/tests/test_rules/data/jinja_template_indentation_rule.j2{% if test %} {% endif %} {% if test %} {% if inner_test %} {% endif %} {% endif %} 07070100000042000081A400000000000000000000000164AD1D7A00000026000000000000000000000000000000000000005600000000j2lint-1.1.0/tests/test_rules/data/jinja_template_indentation_rule.missing_end_tag.j2{% if test %} {# missing the endif #} 07070100000043000081A400000000000000000000000164AD1D7A00000014000000000000000000000000000000000000004200000000j2lint-1.1.0/tests/test_rules/data/jinja_template_no_tabs_rule.j2 I put a tab before 07070100000044000081A400000000000000000000000164AD1D7A00000023000000000000000000000000000000000000004B00000000j2lint-1.1.0/tests/test_rules/data/jinja_template_single_statement_rule.j2{% set test=42 %}{% set blah=43 %} 07070100000045000081A400000000000000000000000164AD1D7A0000000F000000000000000000000000000000000000004700000000j2lint-1.1.0/tests/test_rules/data/jinja_template_syntax_error_rule.j2{% set test %} 07070100000046000081A400000000000000000000000164AD1D7A0000012A000000000000000000000000000000000000004400000000j2lint-1.1.0/tests/test_rules/data/jinja_variable_has_space_rule.j2{{ethernet}} {{ ethernet}} # Too many spaces {{ ethernet}} {{ ethernet }} {{ethernet }} {{+ethernet }} {{-ethernet }} {{ ethernet-}} # Next ones should not fail {{ ethernet }} # cf issue/62 {% set row.line = row.line ~ "| {t:{t_width}} ".format(t=text, t_width=table.column_width[key]) %} 07070100000047000081A400000000000000000000000164AD1D7A00000022000000000000000000000000000000000000004400000000j2lint-1.1.0/tests/test_rules/data/jinja_variable_name_case_rule.j2{{ TEST }} {{ tEST }} {{ t_A_t }} 07070100000048000081A400000000000000000000000164AD1D7A00000010000000000000000000000000000000000000004600000000j2lint-1.1.0/tests/test_rules/data/jinja_variable_name_format_rule.j2{{ test-test }} 07070100000049000081A400000000000000000000000164AD1D7A00001304000000000000000000000000000000000000002C00000000j2lint-1.1.0/tests/test_rules/test_rules.pyimport logging import pathlib import pytest TEST_DATA_DIR = pathlib.Path(__file__).parent / "data" PARAMS = [ pytest.param( f"{TEST_DATA_DIR}/jinja_template_syntax_error_rule.j2", [("S0", 1)], [], [], id="jinja_template_syntax_error_rule", ), pytest.param( f"{TEST_DATA_DIR}/jinja_variable_has_space_rule.j2", [ ("S1", 1), ("S1", 2), ("S1", 4), ("S1", 5), ("S1", 6), ("S1", 7), ("S2", 7), # the faulty {{+ethernet }} triggers S2 as well ("S1", 8), ("S1", 9), ], [], [], id="jinja_variable_has_space_rule", ), pytest.param( f"{TEST_DATA_DIR}/jinja_operator_has_spaces_rule.j2", [("S2", 1), ("S2", 2), ("S2", 3), ("S2", 6), ("S2", 6), ("S2", 8), ("S2", 10)], [], [], id="jinja_operator_has_space_rule", ), pytest.param( f"{TEST_DATA_DIR}/jinja_template_indentation_rule.j2", [("S3", 2), ("S3", 6), ("S3", 5)], [], [], id="jinja_template_indentation_rule", ), pytest.param( f"{TEST_DATA_DIR}/jinja_template_indentation_rule.JinjaLinterError.j2", [("S0", 2), ("S3", 2)], [], [ ( "root", logging.ERROR, f"Indentation check failed for file {TEST_DATA_DIR}/jinja_template_indentation_rule.JinjaLinterError.j2: " "Error: Line 1 - Tag is out of order 'endfor'", ) ], id="jinja_template_indentation_rule JinjaLinterError", ), pytest.param( f"{TEST_DATA_DIR}/jinja_template_indentation_rule.missing_end_tag.j2", [("S0", 1)], [], [ ( "root", logging.ERROR, f"Indentation check failed for file {TEST_DATA_DIR}/jinja_template_indentation_rule.missing_end_tag.j2: " "Error: Recursive check_indentation returned None for an opening tag line 0 - missing closing tag", ) ], ), pytest.param( f"{TEST_DATA_DIR}/jinja_template_indentation_rule.IndexError.j2", [("S4", 2)], [], [ # somehow this is not picked up when we should expect this log message (which can be seen in CLI) # probably some caplog issue here so commenting for now # FIXME # ( # "root", # logging.ERROR, # f"Indentation check failed for file {TEST_DATA_DIR}/jinja_template_indentation_rule.IndexError.j2: " # "Error: list index out of range", # ) ], ), pytest.param( f"{TEST_DATA_DIR}/jinja_statement_has_spaces_rule.j2", [("S4", 1), ("S4", 2), ("S4", 3)], [], [], ), pytest.param( f"{TEST_DATA_DIR}/jinja_template_no_tabs_rule.j2", [("S5", 1)], [], [], ), pytest.param( f"{TEST_DATA_DIR}/jinja_statement_delimiter_rule.j2", [("S6", 1), ("S6", 3), ("S6", 5)], [], [], ), pytest.param( f"{TEST_DATA_DIR}/jinja_template_single_statement_rule.j2", [("S7", 1)], [], [], ), pytest.param( f"{TEST_DATA_DIR}/jinja_variable_name_case_rule.j2", [("V1", 1), ("V1", 2), ("V1", 3)], [], [], ), pytest.param( f"{TEST_DATA_DIR}/jinja_variable_name_format_rule.j2", [("V2", 1)], [], [], ), ] @pytest.mark.parametrize( "filename, j2_errors_ids, j2_warnings_ids, expected_log", PARAMS, ) def test_rules( caplog, collection, filename, j2_errors_ids, j2_warnings_ids, expected_log ): """ caplog: fixture to capture logs collection: a collection from the j2lint default rules filename: the name of the file to parse j2_errors_ids: the ids of the expected errors (<ID>, <Line Number>) j2_warnings_ids: the ids of the expected warnings (<ID>, <Line Number>) expected_log: a list of expected log tuples as defined per caplog.record_tuples """ with open(filename, "r") as f: print(f.read()) caplog.set_level(logging.INFO) errors, warnings = collection.run({"path": filename, "type": "jinja"}) errors_ids = [(error.rule.rule_id, error.line_number) for error in errors] warnings_ids = [(warning.rule.rule_id, warning.line_number) for warning in warnings] print(caplog.record_tuples) for record_tuple in expected_log: assert record_tuple in caplog.record_tuples assert sorted(warnings_ids) == sorted(j2_warnings_ids) assert sorted(errors_ids) == sorted(j2_errors_ids) 0707010000004A000081A400000000000000000000000164AD1D7A00001725000000000000000000000000000000000000002100000000j2lint-1.1.0/tests/test_utils.py""" Tests for j2lint.utils.py """ import pathlib import pytest from j2lint.utils import ( delimit_jinja_statement, flatten, get_file_type, get_files, get_tuple, is_rule_disabled, is_valid_file_type, ) from .utils import does_not_raise # pylint: disable=fixme @pytest.mark.skip def test_load_plugins(): """ Test the utils.load_plugins function For now this is being tested via other calling methods """ # TODO @pytest.mark.parametrize( "file_name, expected", [ ("test.j2", True), ("test.jinja", True), ("test.jinja2", True), ("test.blah", False), ("test_dir/test.j2", True), ("test", False), ], ) def test_is_valid_file_type(file_name, expected): """ Test the utils.is_valid_file_type function """ assert is_valid_file_type(file_name) == expected @pytest.mark.parametrize( "file_name, expected", [ ("test.j2", "jinja"), ("test.jinja", "jinja"), ("test.jinja2", "jinja"), ("test.blah", None), ("test_dir/test.j2", "jinja"), ("test", None), ], ) def test_get_file_type(file_name, expected): """ Test the utils.get_file_type function """ assert get_file_type(file_name) == expected @pytest.mark.parametrize( "file_or_dir_names, expected_value, expectation", [ (["test.j2"], ["test.j2"], does_not_raise()), (["test.jinja"], ["test.jinja"], does_not_raise()), (["test.jinja2"], ["test.jinja2"], does_not_raise()), (["test.jinja", "test.j2"], ["test.jinja", "test.j2"], does_not_raise()), (["test.blah"], [], does_not_raise()), (["test_dir/test.j2"], ["test_dir/test.j2"], does_not_raise()), (["test"], [], does_not_raise()), pytest.param( "not_a_list", None, pytest.raises(TypeError), id="Invalid input type" ), ], ) def test_get_files(file_or_dir_names, expected_value, expectation): """ Test the utils.get_files function """ with expectation: assert get_files(file_or_dir_names) == expected_value def test_get_files_dir(template_tmp_dir): """ Test the utils.get_files function # sadly cannot use fixture in parametrize... # https://github.com/pytest-dev/pytest/issues/349 """ tmp_dir = pathlib.Path(__file__).parent / "tmp" # as passed to pytest in pytest.ini expected = sorted( [ f"{tmp_dir}/rules/rules.j2", f"{tmp_dir}/rules/rules_subdir/rules.j2", f"{tmp_dir}/rules/.rules_hidden_subdir/rules.j2", ] ) with does_not_raise(): assert sorted(get_files(template_tmp_dir)) == expected @pytest.mark.parametrize( "input_list, expected, raising_context", [ ("not_a_list", [], pytest.raises(TypeError)), ([1, 2, 3], [1, 2, 3], does_not_raise()), ([1, 2, [3, 4]], [1, 2, 3, 4], does_not_raise()), ([[1, 2], [3, 4]], [1, 2, 3, 4], does_not_raise()), ], ) def test_flatten(input_list, expected, raising_context): """ Test the utils.flatten function """ with raising_context: assert list(flatten(input_list)) == expected @pytest.mark.parametrize( "tuple_list, lookup_object, expected_value", [ ( [(1, 2, 3), (1, 2, 4)], 1, (1, 2, 3), ), # test that we get the first tuple that matches ([(1, 2, 3), (1, 2, 4)], 4, (1, 2, 4)), ([], 4, None), ([(1, 2, 3), (1, 2, 4)], 5, None), ], ) def test_get_tuple(tuple_list, lookup_object, expected_value): """ Test the utils.get_tuple function """ assert get_tuple(tuple_list, lookup_object) == expected_value @pytest.mark.skip def test_get_jinja_statements(): """ Test the utils.get_jinja_statements function """ # TODO @pytest.mark.parametrize( "line, kwargs, expected", [ ("foo", {}, "{%foo%}"), ("foo", {"start": "{#"}, "{#foo%}"), ("foo", {"start": "{#", "end": "#}"}, "{#foo#}"), ], ) def test_delimit_jinja_statement(line, kwargs, expected): """ Test the utils.delimit_jinja_statement function """ assert delimit_jinja_statement(line, **kwargs) == expected @pytest.mark.skip def test_get_jinja_comments(): """ Test the utils.get_jinja_comments function """ # TODO @pytest.mark.skip def test_get_jinja_variables(): """ Test the utils.get_jinja_variables function """ # TODO @pytest.mark.parametrize( "comments, expected_value", [ pytest.param( ["{# j2lint: disable=test-rule-0 #}"], True, id="found_short_description" ), pytest.param(["{# j2lint: disable=T0 #}"], True, id="found_id"), pytest.param( ["{# j2lint: disable=test-rule-1 #}"], False, id="not_found_short_description", ), pytest.param(["{# j2lint: disable=T1 #}"], False, id="not_found_id"), pytest.param( ["{# j2lint: disable=dummy-rule, test-rule-0 #}"], False, id="NOT_SUPPORTED_single_comment_list", ), pytest.param( ["{# j2lint: disable=dummy-rule, j2lint: disable=test-rule-0 #}"], True, id="single_comment_repeat_pattern", ), pytest.param( ["{# j2lint: disable=dummy-rule #}", "{# j2lint: disable=test-rule-0 #}"], True, id="found_second_second_syntax", ), ], ) def test_is_rule_disabled(make_rules, comments, expected_value): """ Test the utils.is_rule_disabled function """ # Generate one rule through fixture which is always # T0, test-rule-0 test_rule = make_rules(1)[0] comments_string = "\n".join(comments) print(comments_string) assert is_rule_disabled(comments_string, test_rule) == expected_value 0707010000004B000081A400000000000000000000000164AD1D7A00001465000000000000000000000000000000000000001C00000000j2lint-1.1.0/tests/utils.py""" utils.py - functions to assist with tests """ from contextlib import contextmanager @contextmanager def does_not_raise(): """ Provides a context manager that does not raise anything for pytest tests """ yield def j2lint_default_rules_string(): """ The description of the default rules """ return ( "─────────────────────────── Rules in the Collection ────────────────────────────\n" "Origin: BUILT-IN\n" "├── S0 Jinja syntax should be correct (jinja-syntax-error)\n" "├── S1 A single space should be added between Jinja2 curly brackets and a \n" "│ variable name: {{ ethernet_interface }} (single-space-decorator)\n" "├── S2 When variables are used in combination with an operator, the operator \n" "│ should be enclosed by space: '{{ my_value | to_json }}' \n" "│ (operator-enclosed-by-spaces)\n" "├── S3 All J2 statements must be indented by 4 more spaces within jinja \n" "│ delimiter. To close a control, end tag must have same indentation level. \n" "│ (jinja-statements-indentation)\n" "├── S4 Jinja statement should have at least a single space after '{%' and a \n" "│ single space before '%}' (jinja-statements-single-space)\n" "├── S5 Indentation should not use tabulation but 4 spaces \n" "│ (jinja-statements-no-tabs)\n" "├── S6 Jinja statements should not have {%- or {%+ or -%} as delimiters \n" "│ (jinja-statements-delimiter)\n" "├── S7 Jinja statements should be on separate lines (single-statement-per-line)\n" "├── V1 All variables should use lower case: '{{ variable }}' \n" "│ (jinja-variable-lower-case)\n" "└── V2 If variable is multi-words, underscore `_` should be used as a separator:\n" " '{{ my_variable_name }}' (jinja-variable-format)\n" ) NO_ERROR_NO_WARNING_JSON = """{ "ERRORS": [], "WARNINGS": [] } """ ONE_ERROR_JSON = """{ "ERRORS": [ { "id": "T0", "message": "test rule 0", "filename": "dummy.j2", "line_number": 1, "line": "dummy", "severity": "LOW" } ], "WARNINGS": [] } """ ONE_ERROR_ONE_WARNING_JSON = """{ "ERRORS": [ { "id": "T0", "message": "test rule 0", "filename": "dummy.j2", "line_number": 1, "line": "dummy", "severity": "LOW" } ], "WARNINGS": [ { "id": "T0", "message": "test rule 0", "filename": "dummy.j2", "line_number": 1, "line": "dummy", "severity": "LOW" } ] } """ ONE_ERROR = ( "────────────────────────────── JINJA2 LINT ERRORS ──────────────────────────────\n" "dummy.j2\n" "└── dummy.j2:1 test rule 0 (test-rule-0)\n" "\n" "Jinja2 linting finished with 1 error(s) and 0 warning(s)\n" ) # Cannot use """ because of the trailing whitspaces generated in rich Tree ONE_WARNING_VERBOSE = ( "───────────────────────────── JINJA2 LINT WARNINGS ─────────────────────────────\n" "dummy.j2\n" "└── Linting rule: T0\n" " Rule description: test rule 0\n" " Error line: dummy.j2:1 dummy\n" " Error message: test rule 0\n" " \n" "\n" "Jinja2 linting finished with 0 error(s) and 1 warning(s)\n" ) # Cannot use """ because of the trailing whitspaces generated in rich Tree ONE_ERROR_ONE_WARNING_VERBOSE = ( "────────────────────────────── JINJA2 LINT ERRORS ──────────────────────────────\n" "tests/data/test.j2\n" "└── Linting rule: T0\n" " Rule description: test rule 0\n" " Error line: dummy.j2:1 dummy\n" " Error message: test rule 0\n" " \n" "───────────────────────────── JINJA2 LINT WARNINGS ─────────────────────────────\n" "tests/data/test.j2\n" "└── Linting rule: T0\n" " Rule description: test rule 0\n" " Error line: dummy.j2:1 dummy\n" " Error message: test rule 0\n" " \n" "\n" "Jinja2 linting finished with 1 error(s) and 1 warning(s)\n" ) ONE_ERROR_REGEX = ( r"────────────────────────────── JINJA2 LINT ERRORS ────────────────────────────── " r".*.j2 " r"└── (.*.j2):1 Jinja statement should have at least a single space after '{%' and " r"a single space before '%}' \(jinja-statements-single-space\) " r"Jinja2 linting finished with 1 error\(s\) and 0 warning\(s\)" ) 07070100000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000B00000000TRAILER!!!278 blocks
Locations
Projects
Search
Status Monitor
Help
OpenBuildService.org
Documentation
API Documentation
Code of Conduct
Contact
Support
@OBShq
Terms
openSUSE Build Service is sponsored by
The Open Build Service is an
openSUSE project
.
Sign Up
Log In
Places
Places
All Projects
Status Monitor
