How to Set Up CI Checks for Rule Compliance

CI Checks: The Automated Compliance Gate

Without CI checks: a developer deletes CLAUDE.md accidentally. Nobody notices for 2 weeks. The AI: generates generic code without project conventions. Code reviews: catch the inconsistency but do not trace it to the missing rule file. With CI checks: the next PR after the deletion fails CI: 'CLAUDE.md not found. AI rules are required.' The developer: restores the file. The inconsistency: never reaches the codebase. CI checks: the automated safety net for AI rule compliance.

Three levels of CI checks: Level 1 — Existence check (does the rule file exist and is it not empty?). Level 2 — Version check (is the rule file's version current, matching the latest published ruleset?). Level 3 — Structural validation (does the rule file have the required sections, no vague rules, no conflicts?). Each level: more comprehensive. Most teams: start with Level 1 (simplest, catches the most common issue — missing rules) and add Levels 2-3 as the rule program matures.

Graduated enforcement: advisory mode (the check runs and logs warnings but does not block) → warning mode (the check produces a visible warning in the PR but does not block merge) → blocking mode (the check must pass for the PR to be mergeable). Start with advisory: gives teams time to comply. Move to blocking: after 80%+ of PRs already pass. AI rule: 'Advisory → warning → blocking. Each stage: 2-4 weeks. The team adapts before enforcement tightens.'

Step 1: Existence and Version Checks

Existence check: a CI step that verifies the rule file exists. Script: 'if [ ! -f CLAUDE.md ]; then echo "ERROR: CLAUDE.md not found. AI rules are required."; exit 1; fi'. Also check: the file is not empty ('if [ ! -s CLAUDE.md ]': checks for non-empty). This check: catches accidental deletion, projects that never set up rules, and corrupted files (zero bytes). The check: adds 1 second to CI. The protection: prevents projects from running without AI rules.

Version check: verify the rule file's version matches the expected version. Parse the version header from CLAUDE.md ('# AI Rules v2.5.0'). Compare against the expected version (from a config file, environment variable, or the RuleSync API with rulesync pull --check). If outdated: report which version is current and how to update. The check: catches rule drift (the project has not updated after a ruleset change). For teams using RuleSync: rulesync pull --check handles both existence and version in one command.

Implementation in GitHub Actions: 'steps: - name: Check AI rules, run: | if [ ! -s CLAUDE.md ]; then echo "::error::CLAUDE.md is missing or empty"; exit 1; fi; version=$(head -1 CLAUDE.md | grep -oP "v[\d.]+"); echo "Rule version: $version"'. For version comparison: add a step that compares against the expected version. For RuleSync users: 'run: npx rulesync pull --check' (handles everything). AI rule: 'The simplest check (file exists and is not empty): 2 lines of bash. The most complete check (version + structure): rulesync pull --check. Start simple, add complexity as needed.'

Step 2: Structural Validation (Advanced)

Required sections check: verify the rule file contains expected sections. For an organization that requires security rules: check for '## Security' or '## Critical Rules' heading. Script: 'if ! grep -q "## Security\|## Critical" CLAUDE.md; then echo "::warning::CLAUDE.md is missing the Security rules section"; fi'. The check: ensures every project has the mandatory rule sections, even if the specific rules vary.

Rule quality checks (from the linting tutorial): integrate the rule linter into CI. The linter: checks for vague rules, prohibitions without alternatives, and potential conflicts. In advisory mode: the linter warnings appear in the CI log but do not block. In blocking mode: linter errors (critical issues) block the PR. Linter warnings: remain advisory. This tiered approach: blocks critical quality issues while allowing non-critical improvements.

Custom compliance checks: for organizations with specific requirements. Examples: 'Every CLAUDE.md must include at least 3 security rules' (count rules in the Security section), 'The rule file must reference the organization's base ruleset version' (check for the base version header), 'The rule file must not exceed 5,000 words' (prevent rule file bloat). Each custom check: a small script in the CI pipeline. AI rule: 'Custom checks: specific to your organization's requirements. Start with existence + version. Add structural checks when the organization has specific compliance needs.'

Step 3: PR Integration and Developer Experience

PR status check: the CI check appears as a status check on every PR. Green checkmark: rules are compliant. Red X: rules are missing, outdated, or structurally invalid. The developer: sees the status before merging. If red: they fix the issue (restore the file, update the version, fix the structural issue) and push again. The status check: visible feedback that rules are part of the development workflow.

Helpful error messages: when the check fails, the error message should: identify the specific issue ('CLAUDE.md version v2.3.0 is outdated. Current version: v2.5.0.'), provide the fix ('Run rulesync pull to update, or manually update the version header.'), and link to documentation ('See: docs.rulesync.dev/ci-checks for troubleshooting.'). A helpful error: resolved in 2 minutes. A cryptic error ('Rule check failed'): requires 15 minutes of investigation. AI rule: 'Error messages: specific problem + specific fix + documentation link. The developer should know exactly what is wrong and exactly how to fix it without leaving the CI log.'

Bypass for emergencies: the CI check can be bypassed in genuine emergencies (the production is down and the hotfix PR must merge immediately). But: the bypass should be rare, logged, and followed up. Pattern: add a label ('skip-rule-check') that the CI workflow checks. If the label is present: skip the rule check. The follow-up: after the emergency, remove the label and fix the rule compliance issue. AI rule: 'Emergency bypass: available but logged. If used more than once per quarter: the check is too strict or the team needs better rule maintenance habits.'

CI Rule Checks Summary

Summary of setting up CI checks for AI rule compliance.

• Level 1: existence check (file exists and is not empty). 1 second. Catches accidental deletion
• Level 2: version check (file version matches expected). Catches rule drift
• Level 3: structural validation (required sections, linter checks). Catches quality issues
• Graduated enforcement: advisory (log warnings) → warning (visible in PR) → blocking (must pass to merge)
• Timeline: advisory for 2-4 weeks, then warning, then blocking after 80%+ PRs pass
• Error messages: specific problem + specific fix + documentation link. Resolve in 2 minutes
• RuleSync shortcut: rulesync pull --check handles existence + version in one command
• Emergency bypass: label-based skip, logged, followed up. Rare — once per quarter max