--- version: '3' vars: ANSIBLE_LINT_CONFIG: .config/ansible-lint.yml CSPELL_CONFIG: .config/cspell.json FLAKE8_CONFIG: .config/flake8.toml PRETTIERIGNORE_CONFIG: .config/prettierignore SHELLCHECK_EXCLUDE: SC1091 YAMLLINT_CONFIG: .config/yamllint.yml tasks: all: deps: - '{{.REPOSITORY_TYPE}}' - eslint - :lint:markdown:markdown - :lint:prose:prose - shellcheck - yaml ansible: deps: - :symlink:{{.REPOSITORY_SUBTYPE}} - :install:pipx:ansible-lint desc: Lint Ansible projects using Ansible Lint hide: '{{ne .REPOSITORY_TYPE "ansible"}}' summary: | # Lint an Ansible project using Ansible Lint This task lints the project using Ansible Lint which will scan the project and report design patterns that can possibly be improved. It can be used on both playbooks and roles. The configuration for Ansible Lint is stored in the root of the project in a file titled `.ansible-lint`. This configuration file is shared between all of our Ansible projects so any changes to it need to be made to the upstream Ansible common file repository. Because of this, it probably makes more sense to to disable rules (when absolutely necessary) using the [syntax described in this link](https://ansible-lint.readthedocs.io/en/latest/rules.html#false-positives-skipping-rules). For more information, see [Ansible Lint's GitHub page](https://github.com/ansible-community/ansible-lint). log: error: Ansible Lint has detected possible errors! start: Linting project with Ansible Lint success: Validated project with Ansible Lint cmds: - | PATH="$PATH:$HOME/.local/bin" ansible-lint -c {{.ANSIBLE_LINT_CONFIG}} any: deps: - :install:pipx:blocklint - :install:pipx:pre-commit-hooks desc: Generic linting of files for things like destroyed-symlinks, merge conflicts, etc. log: error: Errors were reported by the global linters for `{{.CLI_ARGS}}` start: Linting `{{.CLI_ARGS}}` with global linters cmds: - | PATH="$PATH:$HOME/.local/bin" blocklint --wordlist blacklist,slave,whitelist {{.CLI_ARGS}} - PATH="$PATH:$HOME/.local/bin" && check-merge-conflict {{.CLI_ARGS}} - PATH="$PATH:$HOME/.local/bin" && check-added-large-files {{.CLI_ARGS}} - PATH="$PATH:$HOME/.local/bin" && check-symlinks {{.CLI_ARGS}} - PATH="$PATH:$HOME/.local/bin" && destroyed-symlinks {{.CLI_ARGS}} codeclimate: deps: - :install:software:codeclimate - :lint:codeclimate:load:custom-engines desc: Run CodeClimate with all available linters summary: | # Run CodeClimate CodeClimate provides a birds-eye view of all the various possible issues that there may be with a project by leveraging various plugins called CodeClimate engines. This task will run all available linters defined in `.codeclimate.yml` after substituting some of the standard linters with custom engines built by Megabyte Labs. **Example:** `task lint:codeclimate` You can optionally output the results in HTML format. To do so, pass in the name of the output HTML file you would like to generate. **Example generating HTML results:** `task lint:codeclimate -- ' -f html > codeclimate.html'` You can also specify a specific path for CodeClimate to scan: **Example specifying specific path to analyze:** `task lint:codeclimate -- ' ./src/component' log: error: '`codeclimate` has detected errors' start: Running `codeclimate` success: Successfully validated the project by running `codeclimate` cmds: - | if [ -z "$CODECLIMATE_INTEGRATION_TEST" ]; then while read PATHH; do cd "$(dirname $PATHH)" CODECLIMATE_DEBUG=1 codeclimate analyze --dev{{if .CLI_ARGS}}{{.CLI_ARGS}}{{end}} done < <(find test -maxdepth 2 -type f -name .codeclimate.yml) else while read PATHH; do cd "$(dirname $PATHH)" && break done < <(find test -maxdepth 2 -type f -name .codeclimate.yml) task lint:codeclimate:taskfiles:add CODECLIMATE_DEBUG=1 codeclimate analyze --dev task lint:codeclimate:taskfiles:remove fi commit: deps: - :install:modules:local - :install:npm:commitlint summary: | # Lint a commit message This task will lint a commit message with `commitlint`. It requires that the commit message be passed as an argument to this task. **Example:** `task lint:commit -- 'My commit message'` log: error: Encountered error while linting commit message ({{.CLI_ARGS}}) cmds: - | if [ ! -z '{{.CLI_ARGS}}' ] && [ '{{.CLI_ARGS}}' != '.git/MERGE_MSG' ] \ && [ "$(test -f .git/MERGE_MSG && head -c12 < .git/MERGE_MSG)" != 'Merge branch' ]; then cp '{{.CLI_ARGS}}' '{{.CLI_ARGS}}.bak' TMP="$(mktemp)" COMMIT_MSG_NOEMOJI="$(sed 's/^..//' < '{{.CLI_ARGS}}' | xargs)" .config/log info 'Linting commit message (with emoji stripped)' echo "$COMMIT_MSG_NOEMOJI" | commitlint || (mv '{{.CLI_ARGS}}.bak' '{{.CLI_ARGS}}' && exit 1) elif [ '{{.CLI_ARGS}}' == '.git/MERGE_MSG' ]; then .config/log info 'Bypassing commitlint since the message is a merge message' fi status: - '[[ -z "{{.CLI_ARGS}}" ]]' preconditions: - sh: '[ ! -z "{{.CLI_ARGS}}" ]' msg: 'A CLI argument must be passed to this task. See `task lint:commit --summary` for details.' docker: deps: - :install:software:docker desc: Lint Dockerfiles using Hadolint (requires Docker) summary: | # Lint Dockerfiles using Hadolint Hadolint is a linter for Dockerfiles. This task uses Hadolint to report warnings and suggestions so that the project is using best practices and is less likely to have errors. The task uses Docker to run an ultra-compact container that includes Hadolint. Installing Hadolint is done at runtime. The task scans for all files that are named either `Dockerfile` or `Dockerfile.j2`. On top of reporting suggestions for adhereing to Docker best-practices, Hadolint also leverages Shellcheck to report possible errors in the shell logic used by the Dockerfile. **If the Dockerfile is named something other than `Dockerfile` or `Dockerfile.j2`, you can manually run Hadolint by running:** `docker run -v "$PWD:/work" -w /work megabytelabs/hadolint:slim CustomDockerfileName` **This task will ignore files in the following directories (and possibly more):** * .cache * .husky * node_modules * .common * .modules * test * .git * .task * venv **Example scanning the whole project:** `task lint:docker` **Example scanning single file:** `task lint:docker -- CustomDockerfile` For more information, see [Hadolint's GitHub page](https://github.com/hadolint/hadolint). vars: PWD: sh: echo "${PWD}/" hide: sh: '[ ! -f Dockerfile ]' log: error: Hadolint reported errors start: Linting {{if .CLI_ARGS}}`{{.CLI_ARGS}}`{{else}}the project{{end}} with Hadolint success: Hadolint reported no errors cmds: - | {{if .CLI_ARGS}} docker run -v "$PWD:/work" -w /work megabytelabs/hadolint:slim -c .config/hadolint.yml {{.CLI_ARGS | replace .PWD ""}} {{else}} find . -type d \( {{.IGNORE_FOLDERS}} \) -prune -o -type f \( -name Dockerfile -o \ -name Dockerfile.j2 \) -print0 | xargs -0 -r -n1 docker run -v "$PWD:/work" -w /work \ megabytelabs/hadolint:slim -c .config/hadolint.yml {{end}} dockerfile: cmds: - task: docker dockerfilelint: deps: - :install:npm:dockerfilelint desc: Lint a Dockerfile with `dockerfilelint` hide: sh: '[ ! -f Dockerfile ]' log: error: '`dockerfilelint` reported errors' start: Linting {{if .CLI_ARGS}}`{{.CLI_ARGS}}`{{else}}the project{{end}} with `dockerfilelint` success: '`dockerfilelint` ran successfully and reported no errors' cmds: - | {{if .CLI_ARGS}} dockerfilelint {{.CLI_ARGS | replace .PWD ""}} {{else}} find . -type d \( {{.IGNORE_FOLDERS}} \) -prune -o -type f \( -name Dockerfile -o \ -name Dockerfile.j2 \) -print0 | xargs -0 -r -n1 dockerfilelint {{end}} eslint: deps: - :install:modules:local - :install:npm:eslint desc: Lint with ESLint summary: | # Lint with `eslint` This task will lint the project or a specific pattern of files with `eslint`. It is capable of linting JSON, YML, JS, TS, and HTML as long as the correct `package.json` dependencies are installed. By default, all projects have the libraries necessary for linting JSON, TOML, and YML. **Example linting entire project:** `task lint:eslint` **Example linting specific file:** `task lint:eslint -- my_file.ts` **Example linting specific pattern of files:** `task lint:eslint -- '**/*.js'` log: error: ESLint has detected errors that need to be addressed start: Linting {{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}the project{{end}} with ESLint success: Successfully passed the ESLint test (please still address warnings) cmds: - > eslint -c package.json --no-eslintrc --format {{.ESLINT_FORMATTER}} --cache --cache-location .cache/eslintcache {{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}.{{end}} eslint:staged: deps: - :install:modules:local - :install:npm:eslint desc: Lint only modified files with ESLint env: ESLINT_STAGED_ONLY: on log: error: Failed to lint modified files with ESLint start: Linting modified files with ESLint success: Successfully linted modified files with ESLint cmds: - > eslint -c package.json --no-eslintrc --format {{.ESLINT_FORMATTER}} --cache --cache-location .cache/eslintcache {{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}.{{end}} eslint:warn-all: deps: - :install:modules:local - :install:npm:eslint desc: Convert ESLint errors into warnings by saving the modified configuration inside `package.json` summary: | # Convert ESLint Errors into Warnings This task will run ESLint against the project and then automatically inline all reported errors as warnings in `package.json`. log: error: Encountered error while converting `eslint` errors to warnings start: Overriding all errors reported from `eslint` as `warnings` in `package.json` success: Successfully overrode `eslint` errors as warnings cmds: - > ESLINT_TMP="$(mktemp)" eslint -c package.json --no-eslintrc --format summary --cache --cache-location .cache/eslintcache {{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}.{{end}} | tee "$ESLINT_TMP" while read LINE; do TMP="$(mktemp)"; ESLINT_RULE="$(echo "$LINE" | grep 'errors ' | sed 's/.*rule:\ //')"; if [ -n "$ESLINT_RULE" ] && [ "$ESLINT_RULE" != 'syntax error' ]; then jq --arg rule "$ESLINT_RULE" '.eslintConfig.rules[$rule] = "warn"' package.json > "$TMP"; mv "$TMP" package.json; fi; done < "$ESLINT_TMP" go: deps: - :install:software:go - :install:software:golangci-lint desc: Lint Go with `golangci-lint` summary: | # Lint Go Files This task will lint Go files with `golangci-lint`. The configuration is stored in `.config/golangci.yml` and uses a handful of linters. If you would like to skip linting the whole project and instead lint an individual file, then you can do so by passing the file path as a CLI parameter like so: **Example linting individual file:** `task lint:go -- path/filename.go` log: error: Detected errors with `golangci-lint` start: Linting with `golangci-lint` success: Successfully completed linting with `golangci-lint` cmds: - golangci-lint run -c .config/golangci.yml{{if .CLI_ARGS}}{{.CLI_ARGS}}{{end}} js: deps: - :install:modules:local - :install:npm:eslint log: error: Errors were reported by ESLint for {{if .CLI_ARGS}}`{{.CLI_ARGS}}`{{else}}JavaScript/TypeScript files{{end}} start: Linting {{if .CLI_ARGS}}`{{.CLI_ARGS}}`{{else}}JavaScript/TypeScript files{{end}} with ESLint success: Successfully passed ESLint checks for {{if .CLI_ARGS}}`{{.CLI_ARGS}}`{{else}}JavaScript/TypeScript files{{end}} cmds: - > eslint -c package.json --no-eslintrc --format pretty --ext .js,.jsx,.ts,.tsx --cache --cache-location .cache/eslintcache {{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}.{{end}} lint-staged: deps: - :install:modules:local - :install:npm:lint-staged log: error: Errors were encountered when validating the staged files start: Linting staged files success: The staged files appear to be good! cmds: - | lint-staged || LINT_EXIT_CODE="$?" if [ -n "$LINT_EXIT_CODE" ]; then .config/log error 'Linting failed!' .config/log info 'If necessary, you can bypass this pre-commit hook by running `git commit --no-verify`. Please try to fix the errors first.' exit 1 fi packer: deps: - :install:software:packer desc: Validate the Packer templates ending with `template.json` hide: '{{ne .REPOSITORY_TYPE "packer"}}' summary: | # Validate Packer templates This task will loop through all the Packer templates ending with `template.json` in the root of this project and report any errors that the templates might have. Alternatively, you can scan a single file (see example below). **Example scanning for all files ending with `template.json` in the root directory:** `task lint:packer` **Example scanning single file:** `task lint:packer -- filename.json` For more information on `packer validate`, see the [Packer website](https://www.packer.io/docs/commands/validate). log: error: Error while running `packer validate` start: Running {{if .CLI_ARGS}}`packer validate {{.CLI_ARGS}}`{{else}}`packer validate` on all files ending with `template.json`{{end}} success: '`packer validate` reported no issues' cmds: - | {{if .CLI_ARGS}} .config/log info 'Validating the {{.CLI_ARGS}} Packer template' packer validate {{.CLI_ARGS}} {{else}} for TEMPLATE in *template.json; do .config/log info "Validating the $TEMPLATE Packer template" packer validate "$TEMPLATE" done {{end}} php: desc: Lint PHP with all available PHP linters summary: | # Lint PHP Files This task will lint PHP files with the following linters: * [go]() If you would like to skip linting the whole project and instead lint an individual file, then you can do so by passing the file path as a CLI parameter like so: **Example linting individual file:** `task lint:php -- path/filename.php` cmds: - task: :donothing prettier: deps: - :install:modules:local - :install:npm:prettier desc: Lint formatting using Prettier summary: | # Report formatting errors with Prettier This task will run Prettier on the project and list the possible fixes without automatically applying the fixes. It will report mistakes like inconsistent indent lengths, trailing spaces, and more. Prettier will use the configuration specified in the `package.json` file under the `prettier` key. If this command is incompatible with a file then you can add the file to the `.prettierignore` file. For more information, see [Prettier's website](https://prettier.io/). log: error: Errors were encountered by Prettier start: Linting {{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}the project{{end}} with Prettier success: Prettier successfully finished cmds: - | prettier --ignore-path {{.PRETTIERIGNORE_CONFIG}} --list-different {{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}.{{end}} || EXIT_CODE=$? if [ -n "$EXIT_CODE" ]; then .config/log warn 'Failed to lint with `prettier` - falling back to `prettier`' prettier --ignore-path {{.PRETTIERIGNORE_CONFIG}} --list-different {{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}.{{end}} fi python: deps: - python:deps:flake8 - python:deps:python label: lint:python desc: Lint Python files using Flake8 hide: '{{ne .REPOSITORY_TYPE "python"}}' summary: | # Lint Python files using Flake8 Flake8 is a Python library that lints Python projects (or projects that may include Python files). It is a combination of several Python linters like PyFlakes and pycodestyle. This task will run `flake8` using the configuration found in the `.flake8` file in the root of this project. The `.flake8` file is a common file shared across many of our repositories so if changes are made to it then the changes need to be made to the appropriate common file repository [here](https://gitlab.com/megabyte-labs/common). Because of this, it might make more sense to add a comment to lines that you wish to be ignored by flake8. For instance, you can ignore rule E234 by adding a comment at the end of the line that looks like, "# noqa: E234". _NOTE: In order to maintain our strict quality standards, disabling Flake8 rules should only be done when absolutely necessary._ **Example scanning all files:** `task lint:python` **Example scanning specific file:** `task lint:python -- myfile.py` For more information, see [Flake8's GitHub page](https://github.com/PyCQA/flake8). vars: CONFIG_COMMAND: sh: | if [ -f {{.FLAKE8_CONFIG}} ]; then echo '--config {{.FLAKE8_CONFIG}} ' fi log: error: flake8 detected some issues start: Linting {{if .CLI_ARGS}}`{{.CLI_ARGS}}`{{else}}the project{{end}} with flake8 success: flake8 validation was successful cmds: - | PATH="$PATH:$HOME/.local/bin" flake8 {{.CONFIG_COMMAND}}{{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}.{{end}} python:deps:flake8: cmds: - task: :install:pipx:flake8 status: - '[ "{{.REPOSITORY_TYPE}}" == "python" ]' python:deps:python: cmds: - task: :install:python:requirements status: - '[ "{{.REPOSITORY_TYPE}}" != "python" ]' shell: cmds: - task: shellcheck - task: prettier shellcheck: deps: - :install:npm:shellcheck desc: Report possible errors in shell scripts summary: | # Report possible errors in shell scripts using Shellcheck Shellcheck is a tool that reports warnings and suggestions for shell (e.g. bash) scripts. This task can scan the project for files ending with `.sh` or `.sh.j2` and runs Shellcheck on them. Files in the following folders are ignored: * .cache * .husky * .git * node_modules * .husky * slim_test **Example scanning all files:** `task lint:scripts` **Example scanning specific file:** `task lint:scripts -- myfile.sh` For more information, see [Shellcheck's GitHub page](https://github.com/koalaman/shellcheck). log: error: Shellcheck reported errors for {{if .CLI_ARGS}}`{{.CLI_ARGS}}`{{else}}the project{{end}} start: Linting {{if .CLI_ARGS}}`{{.CLI_ARGS}}`{{else}}the project{{end}} with Shellcheck success: Linted {{if .CLI_ARGS}}`{{.CLI_ARGS}}`{{else}}the project{{end}} with no errors from Shellcheck cmds: - | .config/log info 'Linting with Shellcheck' {{if .CLI_ARGS}} shellcheck -e {{.SHELLCHECK_EXCLUDE}} {{.CLI_ARGS}} {{else}} find . -type d \( {{.IGNORE_FOLDERS}} \) -prune -o -type f \( -name '*.sh' -o -name '*.sh.j2' \) \ -print0 | xargs -0 -r -n1 shellcheck -e {{.SHELLCHECK_EXCLUDE}} {{end}} spelling: deps: - :install:npm:cspell desc: Checks for spelling errors in staged files summary: | # Check for spelling errors in staged files Use cspell to check for possible spelling errors using the configuration stored in `.config/cspell.json`. This task is utilized by the pre-commit hook. For more information about cspell, see the [cspell NPM page](https://www.npmjs.com/package/cspell). **Example scanning all staged files:** `task lint:spelling` **Example scanning specific file:** `task lint:spelling -- myfile.sh` **Although this task only analyzes staged files, you can manually run cspell, for example, on all JavaScript files by running:** `npx cspell '**/*.js'` cmds: - task: spelling:{{if .CLI_ARGS}}cli{{else}}staged{{end}} spelling:cli: log: start: Running a spell-check with `cspell` cmds: - cmd: | set -e EXIT_CODE=0 cspell --no-progress --show-context --no-must-find-files --config {{.CSPELL_CONFIG}} {{.CLI_ARGS}} || EXIT_CODE=$? if [ "$EXIT_CODE" != '0' ]; then .config/log warn 'Possible spelling errors were detected. Take appropriate action before merging changes.' fi ignore_errors: true spelling:markdown: deps: - :install:go:misspell desc: Run `misspell` (a spell-checker) on all markdown files log: error: Encountered errors while running `misspell` start: Running `misspell` spell-check success: Successfully ran `misspell` with no errors reported cmds: - > .config/log info 'If errors are found, you can auto-fix them by running `task fix:spelling:markdown`' {{if .CLI_ARGS}} misspell {{.CLI_ARGS}} {{else}} find . -type d \( {{.IGNORE_FOLDERS}} \) -prune -o -type f \( -name '*.md' \) -print0 | xargs -0 -r -n1 misspell {{end}} spelling:staged: deps: - :install:npm:cspell - :install:software:git log: start: Detecting possible spelling errors in staged files cmds: - cmd: | STAGED_FILES="$(git diff --cached --name-only)" if [ -n "$STAGED_FILES" ]; then set -e EXIT_CODE=0 .config/log info 'Running a spell-check on staged files with `cspell`' # Must not quote $STAGED_FILES cspell --no-progress --show-context --no-must-find-files --config {{.CSPELL_CONFIG}} $STAGED_FILES || EXIT_CODE=$? if [ "$EXIT_CODE" != '0' ]; then .config/log warn 'Possible spelling errors were detected. Take appropriate action before merging changes.' fi fi ignore_errors: true toml: deps: - :install:pipx:pre-commit-hooks summary: | # Lint TOML files This task verifies the syntax of TOML documents using the `check-toml` module from the Python `pre-commit-hook` library. **Example scanning all '**/*.toml' files:** `task lint:toml` **Example scanning specific file:** `task lint:toml -- myfile.toml` log: error: Possible TOML errors were detected start: Ensuring {{if .CLI_ARGS}}`{{.CLI_ARGS}}` is valid TOML{{else}}the project'\''s TOML files are valid{{end}} success: The TOML appears to be valid! cmds: - | PATH="$PATH:$HOME/.local/bin" check-toml {{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}.{{end}} vagrant: deps: - :install:software:vagrant desc: Validate the Vagrantfile hide: sh: '[ ! -f Vagrantfile ]' summary: | # Validate the Vagrantfile This task is an alias for `vagrant validate`. Vagrant's `validate` command will ensure the Vagrantfile in the root of this repository has no errors and is using valid syntax. For more information on `vagrant validate`, see the [Vagrant website](https://www.vagrantup.com/docs/cli/validate). log: error: '`vagrant validate` detected one or more issues with the Vagrantfile' start: Validating the Vagrantfile success: The Vagrantfile passed `vagrant validate` cmds: - vagrant validate xml: deps: - :install:pipx:pre-commit-hooks summary: | # Lint XML files This task verifies the syntax of XML documents using the `check-xml` module from the Python `pre-commit-hook` library. **Example scanning all '**/*.xml' files:** `task lint:xml` **Example scanning specific file:** `task lint:xml -- myfile.xml` log: error: Possible issues were detected with the XML{{if .CLI_ARGS}}(`{{.CLI_ARGS}}`){{end}} start: Ensuring {{if .CLI_ARGS}}{{.CLI_ARGS}} is valid XML{{else}}the project contains only valid XML{{end}} success: The XML appears to be valid cmds: - | PATH="$PATH:$HOME/.local/bin" check-xml {{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}.{{end}} yaml: deps: - :install:pipx:yamllint desc: Lint `.yml` files using YAML Lint summary: | # Lint YML files using YAML Lint YAML Lint is a general purpose linter tool that reports suggestions for `.yml` files. It checks for syntax validity as well as cosmetic problems like line lengths, trailing spaces, and indentation. The configuration file is in the root of the repository in the file named `.yamllint`. The `.yamllint` file is shared across all of our projects so if you need to make changes to get rid of warnings it will generally make more sense to disable YAML Lint for a single line using the [method described here](https://github.com/adrienverge/yamllint#features). _NOTE: Disabling YAML Lint rules should only be done when absolutely necessary._ **Example scanning all '**/*.yml' files:** `task lint:yaml` **Example scanning specific file:** `task lint:yaml -- myfile.sh` For more information, see the [YAML Lint GitHub page](https://github.com/adrienverge/yamllint). log: error: Possible issues were detected by `yamllint` start: Linting {{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}the project{{end}} with `yamllint` success: No issues were detected by `yamllint` cmds: - | PATH="$PATH:$HOME/.local/bin" yamllint -c {{.YAMLLINT_CONFIG}} -s {{if .CLI_ARGS}}{{.CLI_ARGS}}{{else}}.{{end}}