Skill Validation

Overview

Skill validation ensures that skills are properly defined, secure, and ready for production use. FastSkill provides multiple levels of validation to catch issues early in the development process.

Validation happens at multiple stages: definition validation, registration validation, and runtime validation.

Validation Types

1. Structure Validation

Validates that skill definitions follow the correct schema and format:

from fastskill.validation import validate_skill_definition

# Basic skill definition
skill_definition = {
    "id": "text-processor",
    "name": "Text Processor",
    "description": "Process text documents",
    "version": "1.0.0",
    "tags": "text,processing,nlp",
    "capabilities": "text_extraction,text_analysis",
    "skill_file": "text_processor.py",
    "enabled": True
}

try:
    validate_skill_definition(skill_definition)
    print("✅ Skill definition is structurally valid")
except ValidationError as e:
    print(f"❌ Validation error: {e}")
    # Fix the issues before proceeding

use fastskill::validation::validate_skill_definition;

let skill_definition = serde_json::json!({
    "id": "text-processor",
    "name": "Text Processor",
    "description": "Process text documents",
    "version": "1.0.0",
    "tags": "text,processing,nlp",
    "capabilities": "text_extraction,text_analysis",
    "enabled": true
});

match validate_skill_definition(&skill_definition) {
    Ok(_) => println!("✅ Skill definition is valid"),
    Err(e) => println!("❌ Validation error: {}", e),
}

2. Security Validation

Checks for potential security issues in skill definitions:

from fastskill.validation import SecurityValidator

validator = SecurityValidator()

# Security validation checks
security_issues = validator.validate_skill(skill_definition)

if security_issues:
    print("⚠️ Security issues found:")
    for issue in security_issues:
        print(f"   {issue['severity']}: {issue['message']}")
else:
    print("✅ No security issues found")

3. Quality Validation

Ensures skills meet quality standards:

from fastskill.validation import QualityValidator

validator = QualityValidator()

# Quality validation checks
quality_score = validator.validate_quality(skill_definition)

print(f"📊 Quality score: {quality_score['overall']}/100")
print(f"   Completeness: {quality_score['completeness']}")
print(f"   Documentation: {quality_score['documentation']}")
print(f"   Best practices: {quality_score['best_practices']}")

# Get improvement suggestions
suggestions = validator.get_improvement_suggestions(skill_definition)
for suggestion in suggestions:
    print(f"💡 Suggestion: {suggestion}")

Validation Rules

Required Fields

string

required

Must be unique within the service
Alphanumeric characters, dashes, and underscores only
3-50 characters in length
Cannot start or end with dashes/underscores

string

required

Human-readable name
1-100 characters in length
No special formatting requirements

string

required

Detailed description of functionality
10-500 characters in length
Should explain what the skill does

string

required

Semantic version format (e.g., “1.0.0”)
Follows MAJOR.MINOR.PATCH convention
Should be updated for breaking changes

Optional Fields Validation

string

Comma-separated list of tags
Each tag 2-30 characters
Alphanumeric, dashes, and underscores only
Maximum 20 tags

string

Comma-separated list of capabilities
Each capability 3-50 characters
Descriptive of actual functionality
Maximum 20 capabilities

string

Path relative to skill storage directory
Must exist and be readable
Should be a Python file or markdown document

integer

1-300 seconds
Must be reasonable for the skill’s operations

integer

32-2048 MB
Must be sufficient for skill execution

Validation CLI

Use the CLI for validation workflows:

# Validate single skill
fastskill validate ./skills/text-processor/skill.json

# Validate entire directory
fastskill validate ./skills/ --recursive

# Validate with security checks
fastskill validate ./skill.json --security

# Validate with quality report
fastskill validate ./skill.json --quality --report quality-report.json

# Batch validation
fastskill validate ./skills/**/*.json --batch --output validation-results.json

Validation Output

# Example validation output
fastskill validate skill.json --verbose

✅ Skill validation completed
📊 Quality score: 85/100
🔍 Structure: ✅ Valid
🔒 Security: ✅ No issues found
📋 Quality: ⚠️ 3 suggestions for improvement
🛡️  Dependencies: ✅ All dependencies available

📝 Improvement suggestions:
   💡 Add more detailed description (current: 25 chars, recommended: 50+ chars)
   💡 Add execution examples in documentation
   💡 Consider adding performance benchmarks

⏱️  Validation completed in 0.3 seconds

Custom Validation

Create custom validation rules for your organization:

from fastskill.validation import BaseValidator, ValidationResult

class CustomValidator(BaseValidator):
    """Custom validation rules for your organization."""

    def validate(self, skill_definition: dict) -> ValidationResult:
        """Perform custom validation."""
        issues = []

        # Custom business rules
        if not skill_definition.get('author', '').endswith('@company.com'):
            issues.append({
                'field': 'author',
                'message': 'Skills must be authored by company team members',
                'severity': 'error'
            })

        # Check for required company tags
        required_tags = ['company-project', 'reviewed']
        skill_tags = skill_definition.get('tags', '').split(',')
        missing_tags = [tag for tag in required_tags if tag not in skill_tags]
        if missing_tags:
            issues.append({
                'field': 'tags',
                'message': f'Missing required tags: {missing_tags}',
                'severity': 'warning'
            })

        # Validate dependencies are approved
        dependencies = skill_definition.get('dependencies', [])
        approved_deps = ['numpy', 'pandas', 'requests']
        unapproved_deps = [dep for dep in dependencies if dep not in approved_deps]
        if unapproved_deps:
            issues.append({
                'field': 'dependencies',
                'message': f'Unapproved dependencies: {unapproved_deps}',
                'severity': 'error'
            })

        return ValidationResult(
            valid=len([i for i in issues if i['severity'] == 'error']) == 0,
            issues=issues
        )

# Register custom validator
from fastskill.validation import ValidationManager

manager = ValidationManager()
manager.register_validator('custom', CustomValidator())

# Use custom validation
result = manager.validate(skill_definition, validators=['structure', 'security', 'custom'])

Batch Validation

Validate multiple skills efficiently:

async def batch_validation_example():
    service = FastSkillService()
    await service.initialize()

    # Get all skills
    skills = await service.list_skills()

    # Validate all skills
    validation_results = []
    for skill in skills:
        try:
            skill_details = await service.get_skill(skill['id'])
            result = await service.validate_skill(skill_details)
            validation_results.append({
                'skill_id': skill['id'],
                'valid': result['valid'],
                'issues': result.get('issues', [])
            })
        except Exception as e:
            validation_results.append({
                'skill_id': skill['id'],
                'valid': False,
                'error': str(e)
            })

    # Report results
    valid_skills = [r for r in validation_results if r['valid']]
    invalid_skills = [r for r in validation_results if not r['valid']]

    print(f"📊 Validation Results:")
    print(f"   ✅ Valid skills: {len(valid_skills)}")
    print(f"   ❌ Invalid skills: {len(invalid_skills)}")

    # Show issues for invalid skills
    for result in invalid_skills[:5]:
        print(f"\n   {result['skill_id']}:")
        if 'error' in result:
            print(f"     Error: {result['error']}")
        else:
            for issue in result['issues']:
                print(f"     {issue['severity']}: {issue['message']}")

    await service.shutdown()

Validation Webhooks

Set up webhooks for validation events:

# Webhook configuration
webhook_config = {
    "validation_failed": {
        "url": "https://hooks.slack.com/...",
        "events": ["structure_error", "security_issue", "quality_warning"]
    },
    "validation_passed": {
        "url": "https://your-ci-system.com/webhook",
        "events": ["all_checks_passed"]
    }
}

# Configure webhooks
await service.configure_validation_webhooks(webhook_config)

# Validation events will now trigger webhooks

Integration with CI/CD

GitHub Actions

# .github/workflows/validate-skills.yml
name: Validate Skills
on:
  push:
    paths:
      - 'skills/**'
  pull_request:
    paths:
      - 'skills/**'

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2

    - name: Install FastSkill
      run: pip install fastskill

    - name: Validate skills
      run: |
        # Validate all skills
        fastskill validate ./skills/ --recursive --security --quality

        # Check for critical issues
        if fastskill validate ./skills/ --format=json | jq '.has_critical_issues'; then
          echo "❌ Critical validation issues found"
          exit 1
        fi

    - name: Generate validation report
      run: |
        fastskill validate ./skills/ --report validation-report.json
        # Upload report as artifact

Pre-commit Hooks

# .pre-commit-config.yaml
repos:
  - repo: local
    hooks:
      - id: validate-skill
        name: Validate skill definitions
        entry: fastskill validate
        language: system
        files: ^skills/.*\.json$
        args: [--security, --quality]

Best Practices

Validate early and often

Run validation as part of your development workflow, not just before deployment.

Use multiple validation levels

Combine structure, security, and quality validation for comprehensive coverage.

Set up automated validation

Integrate validation into your CI/CD pipeline to catch issues before they reach production.

Monitor validation metrics

Track validation success rates and common issues to improve skill quality over time.

Document validation rules

Maintain clear documentation of your validation rules and requirements for skill contributors.

Troubleshooting

Validation Errors

Missing required fields: Ensure all required fields are present and have valid values.

# Check required fields
required_fields = ['id', 'name', 'description', 'version']
missing_fields = [field for field in required_fields if field not in skill_definition]

Invalid field types: Check that field values match expected types (string, integer, boolean, etc.).

# Validate field types
if not isinstance(skill_definition.get('execution_timeout'), int):
    print("❌ execution_timeout must be an integer")

Security Validation Issues

Common security issues: Review skill dependencies, execution permissions, and network access requirements.

# Check for potentially dangerous dependencies
dangerous_deps = ['subprocess', 'os.system', 'eval', 'exec']
skill_deps = skill_definition.get('dependencies', [])
risky_deps = [dep for dep in dangerous_deps if dep in skill_deps]

Performance Issues

Optimize validation: Use validation caching for repeated checks during development.

# Enable validation caching
config.enable_validation_cache = True

Effective validation catches issues early and ensures skills meet quality standards. Invest time in setting up comprehensive validation for your skill ecosystem.

Getting Started

Configuration

Skill Management

Evals & Quality

Skill Optimization

CLI Reference

Registry

Advanced Topics

Integration

Overview

Validation Types

1. Structure Validation

2. Security Validation

3. Quality Validation

Validation Rules

Required Fields

Optional Fields Validation

Validation CLI

Validation Output

Custom Validation

Batch Validation

Validation Webhooks

Integration with CI/CD

GitHub Actions

Pre-commit Hooks

Best Practices

Troubleshooting

​Overview

​Validation Types

​1. Structure Validation

​2. Security Validation

​3. Quality Validation

​Validation Rules

​Required Fields

​Optional Fields Validation

​Validation CLI

​Validation Output

​Custom Validation

​Batch Validation

​Validation Webhooks

​Integration with CI/CD

​GitHub Actions

​Pre-commit Hooks

​Best Practices

​Troubleshooting

Overview

Validation Types

1. Structure Validation

2. Security Validation

3. Quality Validation

Validation Rules

Required Fields

Optional Fields Validation

Validation CLI

Validation Output

Custom Validation

Batch Validation

Validation Webhooks

Integration with CI/CD

GitHub Actions

Pre-commit Hooks

Best Practices

Troubleshooting