# Multimodal Features: FLUX Image Generation, Strudel Music, and Buffer MCP

This document covers the three multimodal extensions to the LinkedIn SSI Booster: FLUX.1-schnell image generation, Strudel live-coding music agent, and Buffer MCP agent.

---

## 🎨 FLUX.1-schnell Image Generation

### Overview

FLUX.1-schnell is a state-of-the-art image generation model running locally on your GPU. The system generates professional, persona-aligned visuals for LinkedIn posts without cloud costs or privacy concerns.

**Key Features:**

- **Local execution** — No API keys, no cloud costs, full privacy
- **VRAM optimized** — Built for consumer hardware (tested on RTX 3060 12GB)
- **GGUF quantization** — 4-bit quantized model fits in local memory
- **Sequential execution** — Intelligent VRAM management (unloads LLM before image gen)
- **Aesthetic grounding** — Prompts generated by agent to match post theme

### Hardware Requirements

| Component  | Minimum      | Recommended             |
| ---------- | ------------ | ----------------------- |
| GPU        | RTX 3060 8GB | RTX 3060 12GB or better |
| VRAM       | 8GB          | 12GB+                   |
| Disk Space | 10GB free    | 20GB+ free              |
| CUDA       | 12.4.1+      | 12.4.1+                 |

**Note:** 8GB cards may struggle with full profile. Use `core` profile for daily operations and only enable `full` profile when you need post visuals.

### Setup

#### 1. Get Civitai API Key

Sign up at [Civitai](https://civitai.com/) and generate an API key. This is required to download the FLUX GGUF weights.

#### 2. Configure Environment

Add to `.env`:

```bash
CIVITAI_API_KEY=your_civitai_key_here
FLUX_MODEL_PATH=/app/models/flux/flux1-schnell-Q4_K_S.gguf
IMAGE_OUTPUT_DIR=/app/yt-vid-data
```

#### 3. Launch Full Profile

```bash
# Using run.sh (recommended)
bash run.sh --profile full up -d

# Or docker compose directly
docker compose --profile full up -d
```

### How It Works

#### First Start Sequence

1. **`flux-init`** (one-shot container) downloads GGUF weights from Civitai to `./models/flux/`
2. **`flux-app`** waits for `flux-init` to complete
3. **`flux-app`** compiles `llama-cpp-python` with CUDA support (5-10 minutes)
4. **`flux-app`** starts inference service
5. **`app`** can now generate images via FLUX API

> **Note:** `flux-init` exits after downloading weights. This is expected behavior.

#### Image Generation Flow

```mermaid
sequenceDiagram
    participant User
    participant App
    participant Ollama
    participant FLUX

    User->>App: Request post with image
    App->>Ollama: Generate post text
    Ollama-->>App: Post content
    App->>Ollama: Unload LLM (free VRAM)
    App->>Ollama: Generate image prompt
    Ollama-->>App: Aesthetic prompt
    App->>FLUX: Generate image
    FLUX-->>App: Image (PNG)
    App->>User: Post + Image saved
```

### Pre-Download Weights (Optional)

Download weights without starting the full stack:

```bash
docker compose --profile full run --rm flux-init
```

Or manually using the provided script:

```bash
bash scripts/download-flux1-schnell-Q4_K_S.sh
```

### Output

Generated images are saved to:

- **Host:** `./yt-vid-data/` (bind-mounted)
- **Container:** `/app/yt-vid-data/`

**Naming convention:** `flux_<timestamp>_<hash>.png`

### Troubleshooting

**Problem:** `flux-init` fails with "Unauthorized" error

**Solution:** Verify `CIVITAI_API_KEY` in `.env` is valid

**Problem:** `flux-app` crashes with "Out of Memory"

**Solution:**

1. Close other GPU applications
2. Reduce `OLLAMA_NUM_CTX` in `.env` (e.g., from `32768` to `16384`)
3. Use `core` profile for daily operations

**Problem:** Image generation is slow

**Solution:** This is expected. FLUX.1-schnell takes 30-60 seconds per image on RTX 3060 12GB.

---

## 🎵 Strudel Live-Coding Music Agent

### Overview

> **Inspiration:** [Switch Angel](https://www.youtube.com/@Switch-Angel) | [Github: strudel-scripts](https://github.com/switchangel/strudel-scripts) | [Do I have your Attention???](https://youtube.com/shorts/sjsS60OTXSQ?si=s4lk7A1hsDyc5cTM)

Strudel is a live-coding music language for generating algorithmic patterns. The SSI Booster includes an autonomous agent that uses Gemma 4 to generate Strudel.js code and sends it to a WebSocket bridge for real-time audio playback.

**Key Features:**

- **Autonomous agent service** — Runs as standalone Docker service
- **Live-coding generation** — Gemma 4 generates clean, executable Strudel.js code
- **WebSocket bridge** — Real-time evaluation and audio playback
- **Container-native** — All components in Docker with auto-dependency management
- **Future enhancement** — Persona-aligned music tied to LinkedIn post themes/sentiment

### Architecture

```mermaid
graph LR
    A[Strudel Agent] -->|Generate Code| B[Gemma 4 LLM]
    B -->|Strudel.js Pattern| C[Strudel MCP Server]
    C -->|WebSocket| D[Audio Output]
    A -.->|Reads Config| E[.env]
```

### Setup

#### 1. Configure Environment

Strudel agent is enabled by default in `core` and `full` profiles. No additional configuration required.

**Optional overrides** (in `.env`):

```bash
# Ollama server URL (auto-set in Docker)
OLLAMA_HOST=http://ollama:11434

# Strudel MCP WebSocket URL
STRUDEL_WS_URL=ws://strudel-music-server:4321

# Strudel MCP HTTP URL
STRUDEL_MCP_URL=http://strudel-music-server:3000
```

#### 2. Launch Stack

```bash
bash run.sh --profile core up -d
```

### Usage

#### Monitor Agent Activity

```bash
docker compose logs -f strudel-mcp-agent
```

#### Run Agent with Custom Prompt (One-Off)

```bash
docker compose --profile core run --rm strudel-mcp-agent python agents/strudel_mcp_agent.py
```

#### Edit Agent Code

Agent code is bind-mounted from `./agents/` — edit and restart:

```bash
nano agents/strudel_mcp_agent.py
docker compose restart strudel-mcp-agent
```

### How It Works

1. **Agent** prompts Gemma 4 with system prompt optimized for Strudel.js syntax
2. **Gemma 4** generates clean Strudel code (no markdown, no extra text)
3. **Agent** sends code to Strudel MCP server via WebSocket
4. **Strudel MCP** evaluates pattern and outputs audio in real-time

### Example Strudel Pattern

```javascript
// Generated by Gemma 4
note("c3 eb3 g3").s("sawtooth").room(0.5).delay(0.3);
```

### Troubleshooting

**Problem:** Strudel MCP server not starting

**Solution:** Check logs:

```bash
docker compose logs strudel-music-server
```

Verify Node.js environment and Strudel repo clone completed successfully.

**Problem:** Agent generating invalid code

**Solution:** Gemma 4 system prompt is optimized for Strudel syntax. If issues persist, edit `agents/strudel_mcp_agent.py` to refine the prompt.

**Problem:** No audio output

**Solution:** Verify Strudel MCP WebSocket connection:

```bash
docker compose logs strudel-mcp-agent | grep "Connected"
```

---

## 📤 Buffer MCP Agent

### Overview

The Buffer MCP agent provides natural language access to the Buffer API. Uses Gemma 4 to translate plain English commands into properly formatted Buffer MCP requests.

**Key Features:**

- **Autonomous agent service** — Runs as standalone Docker service
- **Natural language interface** — Plain English commands → Buffer API calls
- **Direct MCP integration** — Connects to Buffer's official MCP server at `https://mcp.buffer.com/mcp`
- **Full Buffer API access** — List channels, create posts, manage drafts, schedule content
- **Container-native** — Runs alongside Ollama in Docker with auto-authentication
- **Future enhancement** — Voice-controlled Buffer operations, auto post-performance analytics

### Architecture

```mermaid
graph LR
    A[Buffer Agent] -->|Generate Request| B[Gemma 4 LLM]
    B -->|MCP Request| C[Buffer MCP Server]
    C -->|Buffer API| D[Buffer.com]
    A -.->|Reads API Key| E[.env]
```

### Setup

#### 1. Get Buffer API Key

Sign up at [Buffer](https://buffer.com/) and generate an API key from your [Developer Dashboard](https://buffer.com/developers/api).

#### 2. Configure Environment

Add to `.env`:

```bash
BUFFER_API_KEY=your_buffer_api_key_here
```

#### 3. Launch Stack

```bash
bash run.sh --profile core up -d
```

### Usage

#### Monitor Agent Activity

```bash
docker compose logs -f buffer-mcp-agent
```

#### Run Agent with Custom Prompt (One-Off)

```bash
docker compose --profile core run --rm buffer-mcp-agent python agents/buffer_mcp_agent.py
```

#### Edit Agent Code

Agent code is bind-mounted from `./agents/` — edit and restart:

```bash
nano agents/buffer_mcp_agent.py
docker compose restart buffer-mcp-agent
```

### How It Works

1. **Agent** prompts Gemma 4 with natural language command (e.g., "list my channels")
2. **Gemma 4** generates properly formatted Buffer MCP request JSON
3. **Agent** sends request to Buffer MCP server at `https://mcp.buffer.com/mcp`
4. **Buffer MCP** translates to Buffer API call and returns result
5. **Agent** logs response

### Example Commands

```bash
# List all channels
"List my Buffer channels"

# Create a post
"Create a post on my LinkedIn channel with text 'Hello world' scheduled for tomorrow at 9am"

# Check drafts
"Show me all my draft posts"

# Update a post
"Update post ID 12345 with new text 'Updated content'"
```

### Troubleshooting

**Problem:** Buffer API authentication failed

**Solution:** Verify `BUFFER_API_KEY` in `.env` is valid and not expired:

```bash
# Test API key manually
curl -H "Authorization: Bearer your_key_here" https://api.buffer.com/1/user.json
```

**Problem:** Agent generating invalid MCP requests

**Solution:** Gemma 4 system prompt is optimized for Buffer MCP syntax. If issues persist, edit `agents/buffer_mcp_agent.py` to refine the prompt.

**Problem:** Posts not appearing in Buffer

**Solution:** Check Buffer MCP server response in agent logs:

```bash
docker compose logs buffer-mcp-agent | grep "response"
```

---

## Integration with Main Application

All three multimodal features are **separate autonomous services** that run alongside the main SSI Booster application. They do not interfere with core scheduling, curation, or console workflows.

### When to Use

- **FLUX image generation** — When you need visual content for LinkedIn posts (use `--profile full`)
- **Strudel music agent** — Experimental feature for algorithmic music generation (always enabled)
- **Buffer MCP agent** — Alternative natural language interface to Buffer API (always enabled)

### Resource Management

- **Core profile** (`--profile core`) — Daily operations without image generation
- **Full profile** (`--profile full`) — Includes FLUX image generation (higher VRAM usage)

**Recommendation:** Use `core` profile for daily content generation and only spin up `full` profile when you need post visuals.

---

## Future Enhancements

### FLUX Image Generation

- **Persona-aligned aesthetics** — Train custom LoRA weights on your personal brand colors/style
- **Post-image integration** — Auto-attach generated images to scheduled posts

### Strudel Music Agent

- **Sentiment-aligned music** — Generate patterns that match post tone (upbeat for announcements, calm for reflections)
- **Multi-modal posts** — Combine text + image + music for LinkedIn video content

### Buffer MCP Agent

- **Voice-controlled operations** — Use Wyoming Piper TTS for voice commands
- **Auto post-performance analytics** — Generate weekly reports on which posts performed best
- **Smart scheduling** — Analyze optimal posting times based on historical performance

---

## See Also

- [Docker & Deployment Guide](docker-deployment.md) — Full Docker setup and profile management
- [Environment Variables Reference](environment-variables.md) — All multimodal-related env vars
- [Usage Guide](usage-schedule-curate-console.md) — Core workflows (scheduling, curation, console)
