LLM Prompt Laboratory
Prompt engineer workspace for token optimization, template presets, and multi-model debugging
Transformer Architecture Paradigms & 5 Pipeline Processing Stages
Compare BERT (Encoder-Only), GPT/LLaMA (Decoder-Only), and T5 (Encoder-Decoder) attention masks, pretraining objectives, and 5-stage pipeline processing.Decoder-Only Architecture Profile Analysis
Representative Models: GPT-3/4, LLaMA 1/2/3, Qwen, DeepSeek, Mistral
Attention Type: Causal / Masked Attention (lower triangular mask)
Primary Tasks: Autoregressive Generation (Generative AI), Chat, Code Generation
Pre-training Objective: Causal Language Modeling (CLM: next-token prediction)
Decoder forces each Token to only attend to itself and preceding tokens, powering modern generative AI models.
Attention Mask Matrix Visualization
| Token \ Attends To | LLMs | understand | human | knowledge |
|---|---|---|---|---|
| LLMs | 1 (Visible) | 0 (Masked) | 0 (Masked) | 0 (Masked) |
| understand | 1 (Visible) | 1 (Visible) | 0 (Masked) | 0 (Masked) |
| human | 1 (Visible) | 1 (Visible) | 1 (Visible) | 0 (Masked) |
| knowledge | 1 (Visible) | 1 (Visible) | 1 (Visible) | 1 (Visible) |
Transformer Forward Pass: 5 Pipeline Processing Stages
Stage 1: Embedding & Positional Encoding
Look up Token IDs to get word vectors X_embed and overlay Positional Encoding (Sinusoidal or RoPE) to inject sequence position info.
X_0 = \text{TokenEmbedding}(X) + \text{PositionalEncoding}(pos)