Kittlib

Kittlib is a comprehensive Rust library for building agent-based LLM (Large Language Model) applications with middleware support, MCP/ACP client capabilities, and advanced tooling.

Features

Core Capabilities

• Agent System: Create and manage AI agents with configurable behavior and tool calling
• LLM Integration: Built-in support for OpenAI-compatible APIs (OpenAI, llama.cpp server, local LLM servers) with full function calling/tool use support for both OpenAI format (with function wrapper) and direct format
• Middleware Pipeline: Process requests and responses through customizable middleware
• Tool Integration: Add custom tools that agents can use during conversations
• Conversation Management: Store and retrieve conversation history and summaries

Advanced Features

• Vector Embeddings: Generate and search vector embeddings for semantic search
• MCP/ACP Support: Connect to MCP servers and ACP proxies for extended capabilities
• Scratchpad: PostgreSQL-backed note storage with vector and text search
• Sequential Thinking: Enable multi-step reasoning for complex tasks
• Branch Thinking: Explore multiple reasoning paths simultaneously
• Subagents: Delegate tasks to specialized agent instances
• Multimodal Support: Vision + text support for VLMs (GPT-4V, Claude 3, LLaVA, etc.)
• Personality: Apply custom personality manifests to agent conversations

Architecture

Core Components

┌─────────────────────────────────────────────────────────────────────┐
│                         Agent                                    │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐          │
│  │ LLM Service │  │   Tools     │  │  Middleware Manager  │          │
│  └─────────────┘  └─────────────┘  └─────────────────────┘          │
└─────────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────────┐
│                      Middleware Pipeline                        │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐          │
│  │ Pre-process │  │ LLM Call    │  │ Post-process        │          │
│  └─────────────┘  └─────────────┘  └─────────────────────┘          │
└─────────────────────────────────────────────────────────────────────┘
                              │
              ┌───────────────┼───────────────┐
              ▼               ▼               ▼
     ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
     │  History    │ │  Summary    │ │  Datetime   │
     │  Storage    │ │  Storage    │ │  Prompt     │
     └─────────────┘ └─────────────┘ └─────────────┘
                              │
              ┌───────────────┼───────────────┐
              ▼               ▼               ▼
     ┌─────────────────────────────────────────────┐
     │           Repositories (Database)            │
     │  PostgreSQL History/Summary with CRUD         │
     │  Session Management and Vector Search        │
     └─────────────────────────────────────────────┘

Modules

• agent: Main agent implementation and runtime with tool execution
• models: Data models for messages, sessions, and conversations (including multimodal support)
• services: LLM, MCP, and ACP client services
  • services::llm: LLM service trait and implementations (OpenAI, Mock, Null)
  • services::mcp: MCP client for connecting to Model Context Protocol servers
  • services::acp: ACP client for Agent Client Protocol integration
• tools: Built-in tools (scratchpad, sequential thinking, branch thinking, subagents)
• prelude: Re-exports commonly used types and traits for convenient single-import usage (use kittlib::prelude::*)
• repositories: Database repositories for session management
  • PostgreSQL history storage with vector embeddings
  • PostgreSQL summary storage with vector embeddings
  • PostgreSQL personality manifest storage (no embeddings)
  • PostgreSQL scratchpad storage with vector and text search
  • New unified [Storage] trait: Common interface for all storage implementations with [EmbeddingStorage] for vector-enabled backends
  • Session CRUD operations (list, delete, check existence)
  • In-memory implementations for testing
  • Traits: [Storage], [EmbeddingStorage], [InMemoryStorage], [PostgresStorage]
• middlewares: Middleware pipeline for request/response processing
  • History middleware for conversation persistence
  • Summary middleware for conversation summarization
  • Datetime prompt middleware for adding temporal context
• tasks: Background tasks (summarization, vectorization)
• error: Error handling and result types
• logging: Logging configuration

Quick Start

Add Kittlib to your Cargo.toml:

[dependencies]
kittlib = "0.7"
tokio = { version = "1", features = ["full"] }

For comprehensive usage documentation, see the API documentation or the doc comments in each module.

Examples

Each feature has a corresponding example and test file:

Feature	Example	Test
Agent System	examples/agent_system.rs	tests/test_agent_system.rs
LLM Integration	examples/llm_integration.rs	tests/test_llm_integration.rs
Middleware Pipeline	examples/middleware_pipeline.rs	tests/test_middleware_pipeline.rs
Tool Integration	examples/tool_integration.rs	tests/test_tool_integration.rs
Conversation Management	examples/conversation_management.rs	tests/test_conversation_management.rs
Vector Embeddings	examples/vector_embeddings.rs	tests/test_vector_embeddings.rs
MCP/ACP Support	examples/mcp_acp_support.rs	tests/test_mcp_acp_support.rs
Scratchpad	examples/scratchpad.rs	tests/test_scratchpad.rs
Sequential Thinking	examples/sequential_thinking.rs	tests/test_sequential_thinking.rs
Branch Thinking	examples/branch_thinking.rs	tests/test_branch_thinking.rs
Subagents	examples/subagents.rs	tests/test_subagents.rs
Multimodal Support	examples/multimodal_support.rs	tests/test_multimodal_support.rs
Personality	examples/personality.rs	tests/test_personality.rs

Run examples with:

cargo run --example agent_system
cargo run --example llm_integration
# etc.

Run tests with:

cargo test --test test_agent_system
cargo test --test test_llm_integration
# etc.

Or run all feature tests:

cargo test --test test_agent_system --test test_llm_integration --test test_middleware_pipeline --test test_tool_integration --test test_conversation_management --test test_vector_embeddings --test test_scratchpad --test test_sequential_thinking --test test_branch_thinking --test test_subagents --test test_mcp_acp_support --test test_multimodal_support --test test_personality

License

This project is licensed under the ISC License - see the LICENSE file for details.

Acknowledgments

• Inspired by the Model Context Protocol
• Built with Rust and Tokio
• Uses pgvector for vector search
• Uses rmcp for MCP support
• Uses agent-client-protocol for ACP support