Neural Architecture

AutoML with architecture search, hyperparameter tuning, and deployment optimization

Multi-agent neural architecture search: hardware analysis → architecture generation → hyperparameter optimization → performance estimation. Includes SQLite experiment tracking, structured output, and API deployment.

Setup

# Create environment
python3 -m venv venv && source venv/bin/activate

# Install packages
pip install praisonaiagents praisonai

# Set API key
export OPENAI_API_KEY="your-key"

Create Sample Data

# Create hardware specs
cat > hardware.json << 'EOF'
{
  "gpu": {"name": "NVIDIA A100", "memory_gb": 80, "compute_capability": "8.0", "tensor_cores": true},
  "cpu": {"cores": 64, "memory_gb": 512},
  "constraints": {"max_batch_size": 256, "max_model_params_b": 7, "target_latency_ms": 100}
}
EOF

# Create architecture templates
cat > architectures.json << 'EOF'
{
  "transformer": {"layers": [6, 12, 24], "heads": [8, 12, 16], "hidden": [512, 768, 1024]},
  "cnn": {"blocks": [3, 4, 6], "channels": [64, 128, 256], "kernel_sizes": [3, 5, 7]},
  "mlp": {"layers": [2, 4, 8], "hidden": [256, 512, 1024], "activation": ["relu", "gelu"]}
}
EOF

Run: Python Code

from praisonaiagents import Agent, AgentTeam, Task, tool
from pydantic import BaseModel
from typing import List
import json

# Structured output
class ArchitectureConfig(BaseModel):
    model_type: str
    num_layers: int
    hidden_size: int
    num_params_millions: float
    estimated_flops_b: float
    memory_footprint_gb: float
    recommended_batch_size: int
    hyperparameters: dict
    deployment_config: dict

# Database persistence is configured via memory={} parameter

# Tools
@tool
def get_hardware_specs() -> str:
    """Get available hardware specifications."""
    with open("hardware.json") as f:
        return f.read()

@tool
def get_architecture_templates(model_type: str) -> str:
    """Get architecture templates for a model type."""
    with open("architectures.json") as f:
        archs = json.load(f)
    return json.dumps(archs.get(model_type, {}))

@tool
def estimate_model_size(model_type: str, layers: int, hidden: int) -> str:
    """Estimate model parameters and memory."""
    if model_type == "transformer":
        params = layers * (12 * hidden * hidden + 4 * hidden)  # Simplified
    elif model_type == "cnn":
        params = layers * hidden * hidden * 9  # 3x3 kernels
    else:
        params = layers * hidden * hidden
    
    params_m = params / 1e6
    memory_gb = params * 4 / 1e9  # FP32
    flops_b = params * 2 / 1e9
    
    return json.dumps({
        "params_millions": round(params_m, 2),
        "memory_gb": round(memory_gb, 2),
        "flops_billions": round(flops_b, 2)
    })

@tool
def suggest_hyperparameters(model_type: str, params_millions: float) -> str:
    """Suggest hyperparameters based on model size."""
    if params_millions > 1000:
        lr, batch = 1e-4, 32
    elif params_millions > 100:
        lr, batch = 3e-4, 64
    else:
        lr, batch = 1e-3, 128
    
    return json.dumps({
        "learning_rate": lr,
        "batch_size": batch,
        "optimizer": "adamw",
        "weight_decay": 0.01,
        "warmup_steps": 1000
    })

# Agents
hardware_analyzer = Agent(
    name="HardwareAnalyzer",
    instructions="Analyze hardware constraints. Use get_hardware_specs tool.",
    tools=[get_hardware_specs],
    memory={
        "db": "sqlite:///nas_experiments.db",
        "session_id": "nas-experiment"
    }
)

architect = Agent(
    name="NeuralArchitect",
    instructions="Design neural architectures. Use get_architecture_templates and estimate_model_size tools.",
    tools=[get_architecture_templates, estimate_model_size]
)

optimizer = Agent(
    name="HyperparamOptimizer",
    instructions="Optimize hyperparameters. Use suggest_hyperparameters tool.",
    tools=[suggest_hyperparameters]
)

# Tasks
hardware_task = Task(
    description="Analyze hardware constraints for model training",
    agent=hardware_analyzer,
    expected_output="Hardware specs and constraints"
)

arch_task = Task(
    description="Design optimal architecture for {task_type} task",
    agent=architect,
    expected_output="Architecture configuration"
)

hyperparam_task = Task(
    description="Optimize hyperparameters for the architecture",
    agent=optimizer,
    expected_output="Complete model configuration",
    output_pydantic=ArchitectureConfig
)

# Run
agents = AgentTeam(agents=[hardware_analyzer, architect, optimizer], tasks=[hardware_task, arch_task, hyperparam_task])
result = agents.start(task_type="image_classification")
print(result)

Run: CLI

# Design architecture
praisonai "Design a transformer for text classification" --verbose

# With persistence
praisonai "Optimize neural architecture for 80GB GPU" --memory --user-id ml_team

# Compare architectures
praisonai "Compare CNN vs Transformer for image classification" --telemetry

Run: agents.yaml

Create agents.yaml:

framework: praisonai
topic: "neural architecture search"
roles:
  hardware:
    role: Hardware Analyst
    goal: Analyze compute constraints
    backstory: Expert at ML infrastructure
    tasks:
      analyze:
        description: |
          Analyze hardware:
          - GPU memory and compute
          - Batch size limits
          - Latency requirements
        expected_output: Hardware constraints
        
  architect:
    role: Neural Architect
    goal: Design optimal architectures
    backstory: Expert at deep learning architectures
    tasks:
      design:
        description: |
          Design architecture:
          - Model type selection
          - Layer configuration
          - Parameter estimation
        expected_output: Architecture design
        
  optimizer:
    role: Hyperparameter Optimizer
    goal: Tune training parameters
    backstory: Expert at ML optimization
    tasks:
      tune:
        description: |
          Optimize hyperparameters:
          - Learning rate schedule
          - Batch size
          - Regularization
        expected_output: Training configuration

Run:

praisonai agents.yaml --verbose

Monitor & Verify

# View experiment history
praisonai --history 10 --user-id ml_team

# Check metrics
praisonai --metrics

# Export results
praisonai --save nas_results

Serve API

from praisonaiagents import Agent, tool
import json

@tool
def quick_arch_estimate(model_type: str, target_params_m: int) -> str:
    """Quick architecture size estimation."""
    configs = {
        "transformer": {"layers": target_params_m // 50, "hidden": 768},
        "cnn": {"layers": target_params_m // 10, "channels": 256},
        "mlp": {"layers": target_params_m // 5, "hidden": 512}
    }
    
    config = configs.get(model_type, configs["mlp"])
    memory_gb = target_params_m * 4 / 1000
    
    return json.dumps({
        "model_type": model_type,
        "config": config,
        "params_millions": target_params_m,
        "memory_gb": round(memory_gb, 2),
        "recommended_gpu": "A100-80GB" if memory_gb > 40 else "A100-40GB" if memory_gb > 20 else "V100"
    })

agent = Agent(
    name="NASApi",
    instructions="Estimate neural architecture requirements.",
    tools=[quick_arch_estimate]
)

agent.launch(path="/estimate-arch", port=8000)

Test:

curl -X POST http://localhost:8000/estimate-arch \
  -H "Content-Type: application/json" \
  -d '{"message": "Estimate transformer with 1000M parameters"}'

Cleanup

rm -f nas_experiments.db hardware.json architectures.json
deactivate

Features Demonstrated

Feature	Implementation
Multi-agent	Hardware → Architect → Optimizer
Structured Output	Pydantic ArchitectureConfig
Model Estimation	Params, FLOPS, memory
Hardware Aware	GPU constraints
DB Persistence	SQLite via db()
CLI	--telemetry for tracking
YAML Config	3-agent NAS pipeline
API Endpoint	agent.launch()