Technical Architecture Overview

DSL-Generated CRUD System Architecture Guide

System Architecture

High-Level Design

DSL Schema Definition
         ↓
   Intelligent Converter
         ↓
   JSON Schema + Metadata
         ↓
   Code Generation Engine
         ↓
Flask App + SQLAlchemy Models + JavaScript Components

Component Separation

DSL Module (dsl/): Schema definition and conversion tools
Core System (core_v2/): Flask application with generated models and controllers
Generated Output: Production-ready Python/JavaScript code

DSL Processing Pipeline

Schema Definition

table Manifest {
  id Int [pk, increment]
  bill_of_lading String [unique]
  shipper_id Int [ref: > Client.id]
  consignee_id Int [ref: > Client.id]
  // Relationships automatically detected
}

Intelligent Converter (`dsl/converter/intelligent_converter_v2.py`)

Process Flow:

Parse DSL syntax into structured data
Generate base JSON schema with enhanced metadata
Apply manual overrides from configuration files
Extract relationship metadata for dynamic resolution
Generate final schema with denormalized fields and indexes

Output Files:

schema_base.json: Core schema from DSL
schema_overrides.json: Manual customizations
schema_final.json: Merged schema for generation
relationship_metadata.json: Dynamic relationship data

Code Generation (`core_v2/conversion/generate_controllers_v2.py`)

Generated Components:

SQLAlchemy Models: Complete with relationships, indexes, audit fields
Flask Controllers: CRUD operations extending protected base classes
Database Migrations: Schema evolution and data population
API Endpoints: REST interfaces with relationship traversal

Generated Code Architecture

Model Generation Pattern

class Manifest(db.Model):
    __tablename__ = 'manifest'
    # Primary and foreign keys
    id = db.Column(db.Integer, primary_key=True)
    shipper_id = db.Column(db.Integer, db.ForeignKey("client.id"))
    
    # Denormalized fields for performance
    shipper_name = db.Column(db.String(255), index=True)
    
    # Audit trails
    created_at = db.Column(db.DateTime, default=datetime.utcnow)
    updated_at = db.Column(db.DateTime, onupdate=datetime.utcnow)
    deleted_at = db.Column(db.DateTime)  # Soft delete
    
    # Relationships
    shipper = db.relationship('Client', foreign_keys=[shipper_id])

Controller Generation Pattern

# Generated controllers are minimal (25 lines)
from core.base_controller import BaseCRUDController
from generated.models.shipping_v2 import Manifest

controller = BaseCRUDController(db_session, Manifest)
routes = BaseCRUDRoutes(
    controller=controller,
    blueprint_name='manifest_v2',
    url_prefix='/v2/manifest'
)

Design Principle: Complex logic protected in base classes, generated code stays simple and maintainable.

JavaScript Component System

Memory-Safe Architecture

class ComponentManager {
    constructor() {
        this.activeScopes = new Map();    // scope -> handlers
        this.scopedState = new Map();     // scope -> state data
        this.cleanupTimers = new Map();   // scope -> cleanup timer
    }
    
    registerScope(scope, config) {
        // Clear existing scope if present
        if (this.activeScopes.has(scope)) {
            this.cleanupScope(scope);
        }
        // Register new handlers with cleanup tracking
    }
}

Hierarchical Event Isolation

Scope Format: "containerId::componentId::hierarchyLevel"

Container ID: Unique table or form container
Component ID: Specific component type (form_handler, row_select)
Hierarchy Level: parent, child, grandchild

Isolation Benefits:

Parent table selections don’t affect child tables
Component destruction automatically cleans up event listeners
Memory leaks prevented through explicit cleanup patterns
Debugging simplified with scoped logging

Universal Form System

Architecture: Single form template adapts to multiple display contexts

<!-- Universal form core -->
<div class="universal-form-container" 
     data-container-type="{{ container_type }}"
     data-component-scope="{{ scope }}">
    
    <form class="universal-form {{ container_classes }}">
        <div class="form-fields {{ layout_classes }}">
            <!-- Fields adapt to container -->
        </div>
    </form>
</div>

Display Modes:

Inline: Horizontal layout within table context
Modal: Overlay with backdrop and focus management
Slider: Right-side panel with animation
Wide Slider: Multi-column editing interface

Database Architecture

Generated Schema Features

Performance Optimization:

Automatic indexes on foreign keys and search fields
Denormalized fields (e.g., client_name alongside client_id)
Composite indexes for common query patterns

Audit Compliance:

created_at, updated_at timestamps on all models
user_id tracking for change attribution
Soft delete with deleted_at timestamp
Change history preservation

Relationship Handling:

Proper foreign key constraints with cascading rules
Back-references for bidirectional navigation
Many-to-many relationship tables with metadata

Migration System

Schema Evolution:

Automatic migration generation from DSL changes
Data preservation during schema modifications
Rollback capabilities for deployment safety
Population scripts for test and development data

API Architecture

Generated Endpoints

For each entity, the system generates:

GET    /v2/{entity}           # List with pagination and filtering
POST   /v2/{entity}           # Create new record
GET    /v2/{entity}/{id}      # Retrieve specific record
PUT    /v2/{entity}/{id}      # Update existing record  
DELETE /v2/{entity}/{id}      # Soft delete record
POST   /v2/{entity}/{id}/restore # Restore deleted record

Relationship Traversal

Dynamic Navigation:

Foreign key relationships become navigable API endpoints
Metadata-driven relationship discovery
Contextual data loading based on user interface requirements
Breadcrumb navigation support through relationship chains

Query Optimization

Generated Patterns:

Eager loading to prevent N+1 query problems
Pagination with configurable page sizes
Filtering and search capabilities on indexed fields
Sorting support for common business requirements

Deployment Architecture

Container Strategy

Single Container Deployment:

FROM python:3.9-slim
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . /app
WORKDIR /app
CMD ["python", "run_v2.py"]

Multi-Container Options:

API server container
Static asset server (nginx)
Background task processor
Database container (for development)

Infrastructure Requirements

Minimal Setup:

Python 3.8+ runtime environment
PostgreSQL, MySQL, or SQLite database
Standard web application hosting
File storage for user uploads

Scaling Considerations:

Horizontal scaling through load balancers
Database read replicas for query performance
CDN integration for static assets
Session storage for multi-server deployments

Development Workflow

DSL-Driven Development

Define Business Domain: Create/modify DSL schema files
Generate Application: Run converter and code generation
Customize Logic: Add custom business rules in Python
Test Integration: Validate generated APIs and interfaces
Deploy Changes: Standard container deployment process

Code Organization

dsl/
├── schemas/shipping/current/schema.dsl    # Business domain definition
├── converter/intelligent_converter_v2.py  # DSL processing
└── output/                                # Generated schemas

core_v2/  
├── generated/models/shipping_v2.py        # Generated SQLAlchemy models
├── generated/controllers/                 # Generated Flask controllers
├── core/base_controller.py               # Protected base classes
└── templates/                             # UI templates and components

Extension Points

Custom Business Logic

Model Extensions:

# Extend generated models with custom methods
class Manifest(GeneratedManifest):
    def calculate_shipping_cost(self):
        # Custom business logic
        return self.weight * self.rate.amount

Controller Extensions:

# Add custom endpoints to generated controllers
@manifest_blueprint.route('/manifest/<int:id>/calculate-cost')
def calculate_cost(id):
    manifest = Manifest.query.get_or_404(id)
    return {'cost': manifest.calculate_shipping_cost()}

Integration Patterns

External System Integration:

Standard REST API consumption
Database triggers for real-time synchronization
Message queue integration for async processing
Webhook endpoints for external notifications

Authentication Extensions:

SSO integration through Flask extensions
RBAC implementation using generated models
API key management for external access
Audit logging for compliance requirements

Performance Characteristics

Database Performance

Query Patterns:

Indexed foreign key lookups: Sub-millisecond response
Paginated list queries: 10-50ms for typical result sets
Complex relationship traversal: Optimized through eager loading
Search operations: Full-text search on denormalized fields

Application Performance

Response Times (typical Flask/SQLAlchemy performance):

Simple CRUD operations: 10-50ms
Complex queries with joins: 50-200ms
Form rendering and submission: 100-300ms
Large dataset pagination: 200-500ms

Scaling Limits:

Single server: 100-500 concurrent users
Load balanced: 1000+ concurrent users
Database constraints: Typical PostgreSQL/MySQL limits
Memory usage: Standard Flask application patterns

Security Architecture

Generated Security Features

Data Protection:

SQL injection prevention through SQLAlchemy ORM
XSS protection through template escaping
CSRF protection via Flask-WTF integration
Input validation and sanitization

Access Control Foundation:

User authentication and session management
Permission-based access to generated endpoints
Audit logging for all data modifications
Soft delete for data retention compliance

Extension Requirements

Enterprise Security Needs:

SSO integration requires custom development
Advanced RBAC needs custom authorization logic
Field-level encryption for sensitive data
Advanced audit logging and monitoring

Monitoring and Observability

Built-in Capabilities

Application Monitoring:

Standard Flask logging integration
Database query logging and performance tracking
Error tracking and exception handling
User activity and audit trail logging

Deployment Monitoring:

Container health checks and readiness probes
Database connection pooling and monitoring
Static asset delivery and caching metrics
API endpoint response time tracking

Integration Points

External Monitoring:

APM tools (New Relic, Datadog) via standard Python agents
Log aggregation (ELK Stack, Splunk) through structured logging
Metrics collection (Prometheus) via Flask middleware
Error tracking (Sentry) through Flask error handlers

This architecture provides a solid foundation for rapid application development while maintaining the flexibility to extend and customize based on specific business requirements. The separation of concerns between generated and custom code ensures maintainability as applications evolve.