π Pre-commit Deep Learning Guide
A comprehensive guide to mastering pre-commit hooks for automated code quality checks
π Table of Contentsβ
- What is Pre-commit?
- Learning Roadmap
- Core Concepts
- Configuration Reference
- Built-in Hooks
- Language-Specific Hooks
- Custom Hooks
- Best Practices
- Integration Strategies
- Troubleshooting
- Resources
π― What is Pre-commit?β
Pre-commit is a framework for managing and maintaining multi-language pre-commit hooks.
The Problemβ
Without pre-commit:
# Developer commits bad code
git commit -m "Add feature"
# CI fails 10 minutes later
β Linting failed
β Tests failed
β Formatting check failed
# Developer fixes and commits again
git commit -m "Fix linting"
The Solutionβ
With pre-commit:
# Developer tries to commit
git commit -m "Add feature"
# Pre-commit runs automatically BEFORE commit
β³ Running hooks...
β trailing-whitespace......Failed
β eslint..................Failed
# Commit blocked! Fix issues first.
# After fixing:
β
All hooks passed!
[main abc123] Add feature
Key Benefitsβ
β
Catch issues early: Before they reach CI/CD
β
Save time: Fix problems locally, not in CI
β
Enforce standards: Automatically check code quality
β
Multi-language: Works with Python, JS, Go, Rust, etc.
β
Easy setup: Simple YAML configuration
β
Team consistency: Everyone uses same checks
π§© Core Conceptsβ
1. Git Hooks Overviewβ
Git has built-in hooks for various events:
.git/hooks/
βββ pre-commit # Before commit
βββ prepare-commit-msg # Before commit message editor
βββ commit-msg # After commit message entered
βββ post-commit # After commit completed
βββ pre-push # Before push
βββ pre-rebase # Before rebase
βββ ... (many more)
Problem with raw Git hooks:
- Not version-controlled (in
.git/directory) - Hard to share with team
- Language-specific (shell scripts)
- No easy way to manage dependencies
2. Pre-commit Frameworkβ
Pre-commit solves these problems:
# .pre-commit-config.yaml (version-controlled!)
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
How it works:
1. You run: git commit
β
2. Git triggers: .git/hooks/pre-commit
β
3. Pre-commit reads: .pre-commit-config.yaml
β
4. Pre-commit runs: All configured hooks
β
5. If all pass: Commit succeeds β
If any fail: Commit blocked β
3. Hook Repository Structureβ
A hook repository contains:
hook-repo/
βββ .pre-commit-hooks.yaml # Hook definitions
βββ setup.py or package.json # Dependencies
βββ scripts/
βββ check-something.py
βββ fix-something.sh
.pre-commit-hooks.yaml example:
- id: trailing-whitespace
name: Trim Trailing Whitespace
entry: trailing-whitespace-fixer
language: python
types: [text]
4. Hook Execution Flowβ
File staged for commit
β
Filter by file types/patterns
β
Run hook entry point
β
Check exit code
β
Exit 0: Pass β
| Non-zero: Fail β
β
All hooks pass? β Commit | Any fail? β Block
5. File Filteringβ
Pre-commit automatically filters files:
- id: eslint
types: [javascript] # Only .js files
exclude: '^vendor/' # Skip vendor/
files: '^src/' # Only src/
Filter precedence: types β files β exclude
π Configuration Referenceβ
Basic Structureβ
# Minimum version of pre-commit required
minimum_pre_commit_version: '2.15.0'
# Default settings for all repos
default_install_hook_types: [pre-commit, pre-push]
default_stages: [commit, push]
# Exclude files globally
exclude: '^(vendor/|node_modules/|\.min\.js$)'
# Fail fast (stop on first failure)
fail_fast: false
# Hook repositories
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: trailing-whitespace
Repository Configurationβ
repos:
# Remote repository
- repo: https://github.com/psf/black
rev: 23.11.0
hooks:
- id: black
# Local hooks (in your repo)
- repo: local
hooks:
- id: my-custom-script
name: My Custom Check
entry: ./scripts/check.sh
language: script
# Meta hooks (pre-commit itself)
- repo: meta
hooks:
- id: check-hooks-apply
- id: check-useless-excludes
Hook Configurationβ
hooks:
- id: black
# Override hook name
name: Format Python with Black
# Additional arguments
args: [--line-length=88, --target-version=py311]
# File filtering
types: [python]
types_or: [python, pyi]
exclude: '^tests/fixtures/'
files: '^src/.*\.py$'
# Execution settings
language: python
language_version: python3.11
pass_filenames: true
require_serial: false
# Stages to run in
stages: [commit, push]
# Always run (even if no files match)
always_run: false
# Verbose output
verbose: false
# Additional dependencies
additional_dependencies:
- tokenize-rt==5.2.0
Language Typesβ
language: system # Use system-installed binary
language: python # Create Python virtualenv
language: node # Create Node.js environment
language: ruby # Create Ruby environment
language: golang # Create Go environment
language: rust # Create Rust environment
language: docker # Run in Docker container
language: script # Run as shell script
language: fail # Always fail (for testing)
File Type Filteringβ
types: [file] # Any file
types: [text] # Text files only
types: [python] # Python files
types: [javascript] # JavaScript files
types: [json] # JSON files
types: [yaml] # YAML files
types: [markdown] # Markdown files
types: [shell] # Shell scripts
# Custom types
types_or: [python, pyi, jupyter]
exclude_types: [binary]
Stagesβ
stages:
- commit # Pre-commit (default)
- merge-commit # Pre-merge-commit
- push # Pre-push
- prepare-commit-msg # Prepare commit message
- commit-msg # Commit message hook
- post-checkout # Post-checkout
- post-commit # Post-commit
- post-merge # Post-merge
- post-rewrite # Post-rewrite
- manual # Only when explicitly run
π§ Built-in Hooksβ
Essential Pre-commit Hooksβ
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
# File formatting
- id: trailing-whitespace
name: Trim trailing whitespace
- id: end-of-file-fixer
name: Fix end of files
- id: mixed-line-ending
args: [--fix=lf]
# File checks
- id: check-added-large-files
args: [--maxkb=1000]
- id: check-merge-conflict
- id: check-symlinks
- id: destroyed-symlinks
# Syntax validation
- id: check-yaml
args: [--safe]
- id: check-json
- id: check-toml
- id: check-xml
# Security
- id: detect-private-key
- id: detect-aws-credentials
args: [--allow-missing-credentials]
# Python-specific
- id: check-ast
- id: check-builtin-literals
- id: check-docstring-first
- id: debug-statements
- id: name-tests-test
args: [--pytest-test-first]
# Misc
- id: check-case-conflict
- id: check-executables-have-shebangs
- id: check-shebang-scripts-are-executable
- id: fix-byte-order-marker
Hook Descriptionsβ
| Hook ID | What it Does | Auto-fix? |
|---|---|---|
trailing-whitespace | Remove spaces at end of lines | β |
end-of-file-fixer | Ensure files end with newline | β |
check-yaml | Validate YAML syntax | β |
check-json | Validate JSON syntax | β |
check-merge-conflict | Detect merge conflict markers | β |
check-added-large-files | Prevent large files (default 500KB) | β |
detect-private-key | Find private keys in code | β |
mixed-line-ending | Fix CRLF/LF inconsistencies | β |
check-ast | Verify Python syntax | β |
debug-statements | Find breakpoint(), pdb | β |
π Language-Specific Hooksβ
Pythonβ
repos:
# Black - Code formatter
- repo: https://github.com/psf/black
rev: 23.11.0
hooks:
- id: black
args: [--line-length=88]
# isort - Import sorter
- repo: https://github.com/PyCQA/isort
rev: 5.12.0
hooks:
- id: isort
args: [--profile=black]
# Ruff - Fast linter
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.1.6
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
- id: ruff-format
# mypy - Type checking
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.7.1
hooks:
- id: mypy
additional_dependencies: [types-requests]
# Flake8 - Linter
- repo: https://github.com/PyCQA/flake8
rev: 6.1.0
hooks:
- id: flake8
args: [--max-line-length=88]
# Bandit - Security linter
- repo: https://github.com/PyCQA/bandit
rev: 1.7.5
hooks:
- id: bandit
args: [-c, pyproject.toml]
JavaScript/TypeScriptβ
repos:
# ESLint
- repo: https://github.com/pre-commit/mirrors-eslint
rev: v8.54.0
hooks:
- id: eslint
files: \.[jt]sx?$
types: [file]
additional_dependencies:
- eslint@8.54.0
- eslint-config-standard@17.1.0
- '@typescript-eslint/eslint-plugin@6.12.0'
- '@typescript-eslint/parser@6.12.0'
# Prettier
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.1.0
hooks:
- id: prettier
types_or: [javascript, jsx, ts, tsx, json, yaml, markdown]
# TypeScript compiler
- repo: https://github.com/pre-commit/mirrors-tsc
rev: v5.3.2
hooks:
- id: tsc
Goβ
repos:
# gofmt - Formatter
- repo: https://github.com/dnephin/pre-commit-golang
rev: v0.5.1
hooks:
- id: go-fmt
- id: go-vet
- id: go-imports
- id: go-lint
- id: go-unit-tests
# golangci-lint
- repo: https://github.com/golangci/golangci-lint
rev: v1.55.2
hooks:
- id: golangci-lint
Rustβ
repos:
- repo: https://github.com/doublify/pre-commit-rust
rev: v1.0
hooks:
- id: fmt
- id: cargo-check
- id: clippy
Shell Scriptsβ
repos:
# ShellCheck
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: v0.9.0.6
hooks:
- id: shellcheck
# shfmt - Shell formatter
- repo: https://github.com/scop/pre-commit-shfmt
rev: v3.7.0-1
hooks:
- id: shfmt
args: [-i, '2', -ci]
Dockerβ
repos:
# Hadolint - Dockerfile linter
- repo: https://github.com/hadolint/hadolint
rev: v2.12.0
hooks:
- id: hadolint-docker
Markdownβ
repos:
# markdownlint
- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.37.0
hooks:
- id: markdownlint
args: [--fix]
# markdown-link-check
- repo: https://github.com/tcort/markdown-link-check
rev: v3.11.2
hooks:
- id: markdown-link-check
π οΈ Custom Hooksβ
Local Hooksβ
repos:
- repo: local
hooks:
# Shell script
- id: check-secrets
name: Check for secrets
entry: ./scripts/check-secrets.sh
language: script
# Python script
- id: custom-validator
name: Custom validation
entry: python scripts/validate.py
language: system
types: [python]
# Direct command
- id: no-console-log
name: Block console.log
entry: bash -c 'grep -r "console.log" src/ && exit 1 || exit 0'
language: system
types: [javascript]
pass_filenames: false
# Always run hook
- id: check-api-health
name: Check API is running
entry: curl -f http://localhost:8000/health
language: system
pass_filenames: false
always_run: true
stages: [push]
Creating a Hook Repositoryβ
1. Create repository structure:
my-hooks/
βββ .pre-commit-hooks.yaml
βββ setup.py
βββ my_hooks/
βββ __init__.py
βββ check_something.py
2. Define hooks (.pre-commit-hooks.yaml):
- id: check-api-keys
name: Check for exposed API keys
entry: check-api-keys
language: python
types: [text]
- id: validate-config
name: Validate configuration files
entry: validate-config
language: python
files: \.(json|yaml|toml)$
3. Implement hook (my_hooks/check_something.py):
#!/usr/bin/env python3
import re
import sys
def check_api_keys(filenames):
"""Check files for exposed API keys."""
api_key_pattern = re.compile(r'(api[_-]?key|apikey)\s*=\s*["\'][\w-]{20,}["\']', re.IGNORECASE)
errors = []
for filename in filenames:
with open(filename, 'r') as f:
for line_num, line in enumerate(f, 1):
if api_key_pattern.search(line):
errors.append(f"{filename}:{line_num} - Possible API key found")
if errors:
print("\n".join(errors))
return 1
return 0
def main():
return check_api_keys(sys.argv[1:])
if __name__ == '__main__':
sys.exit(main())
4. Setup packaging (setup.py):
from setuptools import setup, find_packages
setup(
name='my-pre-commit-hooks',
version='1.0.0',
packages=find_packages(),
install_requires=[],
entry_points={
'console_scripts': [
'check-api-keys=my_hooks.check_something:main',
],
},
)
5. Use in projects:
repos:
- repo: https://github.com/yourusername/my-hooks
rev: v1.0.0
hooks:
- id: check-api-keys
β Best Practicesβ
1. Start Simple, Grow Graduallyβ
# β
Good - Start minimal
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
# β Bad - Too many hooks at once
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks: # 20+ hooks listed
Approach: Add hooks incrementally, let team adapt.
2. Auto-fix When Possibleβ
# β
Good - Auto-fixes
- id: trailing-whitespace # Fixes automatically
- id: black # Formats code
- id: isort # Sorts imports
# β οΈ Only check - requires manual fix
- id: flake8 # Reports issues
- id: mypy # Reports type errors
Balance: Mix auto-fix hooks (fast feedback) with check hooks (enforce quality).
3. Order Hooks Logicallyβ
repos:
# 1. Format code first
- repo: https://github.com/psf/black
rev: 23.11.0
hooks:
- id: black
# 2. Then sort imports
- repo: https://github.com/PyCQA/isort
rev: 5.12.0
hooks:
- id: isort
# 3. Finally lint
- repo: https://github.com/PyCQA/flake8
rev: 6.1.0
hooks:
- id: flake8
Rationale: Formatters change code, run them before linters.
4. Set Appropriate Stagesβ
repos:
# Fast checks - run on commit
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: trailing-whitespace
stages: [commit]
# Expensive checks - run on push
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.7.1
hooks:
- id: mypy
stages: [push]
# Critical checks - run on both
- repo: https://github.com/PyCQA/bandit
rev: 1.7.5
hooks:
- id: bandit
stages: [commit, push]
5. Exclude Generated/Vendor Filesβ
# Global exclude
exclude: '^(vendor/|node_modules/|dist/|build/|\.min\.js$)'
# Hook-specific exclude
hooks:
- id: eslint
exclude: '^(vendor/|.*\.generated\..*)$'
6. Pin Versionsβ
# β
Good - Specific version
- repo: https://github.com/psf/black
rev: 23.11.0
# β Bad - Floating version
- repo: https://github.com/psf/black
rev: main
Update regularly:
pre-commit autoupdate
7. Document Your Configurationβ
# .pre-commit-config.yaml
# This configuration runs automatically before commits.
# To run manually: pre-commit run --all-files
# To update hooks: pre-commit autoupdate
# To skip hooks: git commit --no-verify
repos:
# Code formatting - auto-fixes files
- repo: https://github.com/psf/black
rev: 23.11.0
hooks:
- id: black
# Line length matches flake8 config
args: [--line-length=88]
8. Use with CI/CDβ
# .github/workflows/pre-commit.yml
name: Pre-commit
on: [push, pull_request]
jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
- uses: pre-commit/action@v3.0.0
Why: Catch issues even if developers skip hooks locally.
π Integration Strategiesβ
With Existing Projectsβ
Step 1: Install pre-commit
pip install pre-commit
Step 2: Add minimal config
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
Step 3: Run on all files once
pre-commit run --all-files
Step 4: Fix issues or update config
# Fix issues
git add -u
git commit -m "Apply pre-commit fixes"
# Or exclude problematic files
echo "exclude: '^legacy/'" >> .pre-commit-config.yaml
Step 5: Install for team
pre-commit install
With EditorConfigβ
# Use both together
repos:
# EditorConfig validation
- repo: https://github.com/editorconfig-checker/editorconfig-checker.python
rev: 2.7.3
hooks:
- id: editorconfig-checker
# Pre-commit hooks
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
With Prettier + ESLintβ
repos:
# Prettier first (formatting)
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v3.1.0
hooks:
- id: prettier
types_or: [javascript, jsx, ts, tsx, json, yaml]
# ESLint second (linting)
- repo: https://github.com/pre-commit/mirrors-eslint
rev: v8.54.0
hooks:
- id: eslint
files: \.[jt]sx?$
types: [file]
With Husky (Node.js projects)β
Choose one approach:
Option A: Use pre-commit only
{
"devDependencies": {
"pre-commit": "^3.5.0"
}
}
Option B: Use Husky + lint-staged
{
"husky": {
"hooks": {
"pre-commit": "lint-staged"
}
},
"lint-staged": {
"*.js": ["eslint --fix", "prettier --write"]
}
}
Option C: Hybrid (pre-commit calls Husky)
repos:
- repo: local
hooks:
- id: husky
name: Husky hooks
entry: npx husky run pre-commit
language: system
pass_filenames: false
Recommendation: Use pre-commit for multi-language projects, Husky for pure Node.js.