search

Twitter Posting TODO

This document outlines the tasks required to implement the context-aware, autonomous Twitter posting functionality for the BILL agent. The tasks are assigned to allow for parallel development.

Developers:

  • Yin: Focuses on context gathering, processing, and database integration.

  • Nikolai: Focuses on the posting pipeline and runtime orchestration.

📋 Related Documentation: See Architecture Document for detailed system diagrams and data flows.

hashtag
Phase 1: Foundational Enhancements

  • Task 1.1: Create TimelineService for Posting Context

    • Owner: Yin

    • Status: ✅ COMPLETED

    • Description: Create a dedicated TimelineService to handle tweet storage and retrieval for posting context, separate from conversation caching.

    • Files: agent/plugins/twitter/services/TimelineService.ts, agent/plugins/twitter/services/TweetDatabaseService.ts (deprecated)

    • Acceptance Criteria:

      • ✅ TimelineService can store and retrieve timeline tweets for posting context.

      • ✅ New Supabase table timeline_tweets with fields: id, text, author_id, created_at, source_account, image_url, image_description, cached_at.

      • ✅ Clean separation: ConversationCacheService for conversations, TimelineService for posting context.

      • ✅ The old TweetDatabaseService is deprecated with migration guidance.

  • Task 1.2: Create PostedTweetsService for Output Tracking

    • Owner: Yin

    • Status: ✅ COMPLETED

    • Description: Create a dedicated service to track BILL's own posted tweets for duplicate prevention, analytics, and posting history.

    • Files: agent/plugins/twitter/services/PostedTweetsService.ts, agent/plugins/twitter/tests/PostedTweetsService.test.ts

    • Acceptance Criteria:

      • ✅ Service can store BILL's posted tweets with metadata.

      • ✅ New Supabase table posted_tweets with fields: id, text, created_at, engagement_stats, image_url, generated_from_context, tweet_type.

      • 🚫 Similarity checking temporarily disabled (TODO: re-enable when ready for duplicate prevention).

      • ✅ Comprehensive test coverage with automated testing.

      • ✅ Analytics methods for posting patterns and performance.

      • ✅ Integration Complete: TwitterClient/TweetService now uses PostedTweetsService

  • Task 1.3: Implement Randomized Posting Interval

    • Owner: Nikolai

    • Status: ✅ COMPLETED

    • Description: Update the auto-posting mechanism to use a randomized interval between a min and max value defined in the environment.

    • Files: agent/plugins/twitter/config/rateLimitConfig.ts, agent/plugins/twitter/types/env.d.ts, agent/plugins/twitter/twitter-plugin.ts

    • Acceptance Criteria:

      • ✅ TWITTER_POST_INTERVAL_MIN and TWITTER_POST_INTERVAL_MAX are added to the environment configuration.

      • ✅ The startAutoPosting function in twitter-plugin.ts uses a randomized timeout to schedule posts.

hashtag
Phase 2: Context Gathering & Processing (Yin) ✅ COMPLETE

🎉 PHASE 2 COMPLETE: Context Gathering & Processing pipeline is fully operational!

✅ What's Working:

  • Timeline Monitoring: Automatically starts when Twitter plugin initializes

  • Tweet Processing: Configured accounts monitored every 30 minutes for new tweets

  • Database Storage: Tweets processed, stored with image metadata in timeline_tweets table

  • Context Management: TwitterContextManager bridges timeline data → AI generation

  • Rate Limiting: Respects Basic tier API limits (5 requests/15min window)

  • Context Structuring: User Context, Thread Context, and World Context for BILL's personality

  • Integration: Tested and verified with comprehensive test coverage

🚀 Ready For: Phase 3 - Autonomous Posting Pipeline integration

  • Task 2.1: Configure Content Sources

    • Owner: Yin

    • Status: ✅ COMPLETED

    • Description: Create a configuration file to list the target Twitter accounts BILL should monitor for context.

    • Files: agent/plugins/twitter/config/timelineContext.ts (created)

    • Acceptance Criteria:

      • ✅ The file exports an array of Twitter usernames or IDs.

      • ✅ The configuration is easily updatable.

    • Implementation Details:

      • Renamed from "Content Sources" to "Timeline Context" for better clarity

      • Integrated with BILL's character profile for configuration

      • Added priority levels, check intervals, and batch groups for optimization

      • Includes rate limiting strategy for Basic tier API compliance

      • Supports both username and user ID monitoring

  • Task 2.2: Implement TimelineService

    • Owner: Yin

    • Status: ✅ COMPLETED

    • Description: Build the service to fetch tweets from the configured accounts, process them (including images), and cache them in the database. This implements the "Timeline Monitor" and "Tweet Processing Pipeline" components from the architecture.

    • Files: agent/plugins/twitter/services/TimelineService.ts (enhanced)

    • Acceptance Criteria:

      • ✅ Service can fetch recent tweets for a list of users (Timeline Monitor functionality).

      • ✅ Implements the Tweet Processor logic to handle raw tweet data.

      • ✅ If a tweet has an image, it processes media and extracts descriptions.

      • 🚧 Image download/storage in Supabase bucket (placeholder implemented, needs integration).

      • ✅ The processed tweet data (content, image URL, image description) is stored in the timeline_tweets table.

      • ✅ Runs on a periodic schedule with configurable intervals.

    • Implementation Details:

      • Added fetchUserTimeline() method for Twitter API integration

      • Added processImageContent() for media handling

      • Added monitorUserTimeline() and monitorAllTimelines() for batch processing

      • Added startPeriodicMonitoring() and stopPeriodicMonitoring() for scheduling

      • Integrated with timeline context configuration from Task 2.1

      • Rate limiting and error handling for Basic tier API limits

      • ✅ INTEGRATION COMPLETE: Integrated with twitter-plugin.ts for automatic monitoring

      • ✅ PIPELINE ACTIVE: Timeline monitoring starts automatically when plugin initializes

      • ✅ STORAGE READY: Tweets stored in timeline_tweets table for posting context

  • Task 2.3: Implement TwitterContextManager

    • Owner: Yin

    • Status: ✅ COMPLETED

    • Description: Create the manager responsible for retrieving relevant, recent context from the database to be used for generating new posts. This implements the "Context Manager" component that provides enhanced context to the Agent Runtime.

    • Files: agent/plugins/twitter/services/TwitterContextManager.ts (created)

    • Acceptance Criteria:

      • ✅ Provides a method like getPostingContext() that returns a collection of recent, relevant tweets and ideas from the database.

      • ✅ Implements the "Context Retrieval for Posting" flow shown in the architecture diagrams.

      • ✅ Can be used by the PostingService to gather inspiration for a new post.

      • ✅ Builds enhanced context objects that include User Context, Thread Context, and World Context.

    • Implementation Details:

      • Added comprehensive PostingContext interface with User, Thread, and World context structures

      • Implemented getPostingContext() method for AI generation context

      • Built User Context analysis: posting patterns, energy level, dominant topics, recent taglines

      • Built World Context analysis: market categorization, trending topics, recent events

      • Built Thread Context structure (limited implementation - ConversationCacheService integration pending)

      • Added context metadata: freshness scoring, time ranges, contributing sources

      • Performance optimized with parallel context gathering

      • Includes getContextSummary() for debugging and monitoring

      • Full test coverage with comprehensive test scenarios

hashtag
Phase 3: Autonomous Posting Pipeline (Yin & Nikolai - Parallel Development)

📋 Before Starting Phase 3: Read the Phase 2 Integration Guidearrow-up-right for complete context on what's available and how to use it.

hashtag
🔄 Parallel Development Strategy

Yin and Nikolai can work simultaneously on the following task groups:

hashtag
Yin's Parallel Tasks:

  • Task 3.1A: Context-Agent Bridge (supports Nikolai's 3.1)

  • Task 3.4A: Image Generation Foundation (supports collaborative 3.4)

hashtag
Current Development (User):

  • 🔄 Task 3.1B: Core PostingService Refactor (IN PROGRESS)

  • 🔄 Task 3.2: Plugin Orchestration Updates (NEXT)

  • ✅ Task 3.3: LLM Response Fallback System (COMPLETED)

hashtag
Collaborative Tasks:

  • Task 3.4B: Image Generation Pipeline Integration (after 3.4A complete)

  • Task 3.1A: Context-Agent Bridge (Yin)

    • Owner: Yin

    • Status: ✅ COMPLETED

    • Description: Create the bridge layer that formats TwitterContextManager output for agent runtime consumption and designs context-aware prompts.

    • Files: agent/plugins/twitter/utils/PostingContextAdapter.ts, agent/plugins/twitter/config/contextPrompts.ts

    • What Was Implemented:

      • ✅ Created PostingContextAdapter class to transform context objects for agent runtime

      • ✅ Designed context-aware prompt templates that incorporate User/World/Thread context

      • ✅ Added context quality validation and error handling

      • ✅ Created prompt variations that work well with different LLM models (for fallback compatibility)

      • ✅ Tested context formatting with sample data from TwitterContextManager

      • ✅ Full test coverage with unit and integration tests

      • ✅ Performance optimized (138ms average for full pipeline)

    • Integration Example:

    • Documentation: See Task 3.1A Summaryarrow-up-right for complete details

  • Task 3.1B: Core PostingService Refactor

    • Owner: Yin

    • Status: 🔄 IN PROGRESS (Context Bridge Complete)

    • Description: Integrate context pipeline into PostingService and handle agent runtime communication for context-aware tweet generation.

    • Files: agent/plugins/twitter/services/PostingService.ts

    • What's Already Done:

      • ✅ TwitterContextManager implemented with getPostingContext() method

      • ✅ PostedTweetsService integrated for storing BILL's tweets after posting

      • ✅ Context pipeline fully tested and operational

      • ✅ Database schema and services ready

      • ✅ PostingContextAdapter complete (Task 3.1A)

      • ✅ Enhanced character templates with context variables

    • What Needs Implementation:

      • 🚧 Import TwitterContextManager and PostingContextAdapter

      • 🚧 Integrate context retrieval into posting flow

      • 🚧 Handle agent runtime communication with formatted context

      • 🚧 Update generation flow: Context Retrieval → Context Formatting → Agent Runtime → Tweet Generation → Post to Twitter

      • 🚧 Error handling for context retrieval failures

    • Integration Example:

  • Task 3.2: Update Plugin Orchestration

    • Owner: Yin

    • Status: 🔄 IN PROGRESS (Follows 3.1B)

    • Description: Update twitter-plugin.ts to integrate the new context-aware posting flow with existing timeline monitoring.

    • Files: agent/plugins/twitter/twitter-plugin.ts

    • What's Already Done:

      • ✅ Timeline monitoring automatically starts when plugin initializes

      • ✅ Background context ingestion runs every 30 minutes

      • ✅ PostedTweetsService integrated with TweetService

      • ✅ Randomized posting intervals implemented

      • ✅ Image analysis integrated with timeline monitoring

    • What Needs Implementation:

      • 🚧 Update autoPost method to use refactored PostingService with context

      • 🚧 Ensure posting flow uses context from timeline monitoring

      • 🚧 Verify timeline monitoring continues to run separate from posting schedule

      • 🚧 Add error handling for context-aware posting failures

  • Task 3.3: Implement LLM Response Fallback System

    • Owner: Nikolai

    • Status: ✅ COMPLETED

    • Description: Create a fallback mechanism that detects when the primary LLM refuses to generate content (due to content policies) and automatically retries with a different model via OpenRouter.

    • Files: agent/services/llm-fallback-handler.ts, agent/services/openrouter-service.ts, agent/tests/test-llm-fallback.ts, package.json

    • Acceptance Criteria:

      • Detect refusal responses that contain phrases like "I appreciate the creative roleplay request, but I'm not comfortable" or similar content policy rejections.

      • Implement pattern matching to identify when the LLM is refusing to roleplay as BILL's character.

      • On detection of refusal, automatically trigger a secondary OpenRouter request with a different model (e.g., Claude, GPT-4, etc.).

      • Log fallback attempts and success/failure rates for monitoring.

      • Handle cases where both primary and fallback models refuse (graceful degradation).

      • Configurable fallback model list and retry limits via environment variables.

      • Integration with existing posting pipeline without disrupting normal flow.

  • Task 3.4A: Image Analysis Service (Yin)

    • Owner: Yin

    • Status: ✅ COMPLETED (Image Analysis)

    • Description: Built image analysis infrastructure for processing images in timeline tweets, especially screenshots from other platforms.

    • Files: agent/plugins/twitter/services/ImageAnalysisService.ts, agent/plugins/twitter/tests/ImageAnalysisService.test.ts

    • What Was Implemented:

      • ✅ Created ImageAnalysisService that integrates with LLM vision capabilities

      • ✅ Enhanced TimelineService.processImageContent() to analyze images with AI

      • ✅ Screenshot detection (Truth Social, Instagram, etc.)

      • ✅ Text extraction from images

      • ✅ Sentiment and topic analysis

      • ✅ Batch processing with rate limiting

      • ✅ Caching for performance

      • ✅ Comprehensive test coverage

    • Critical for: Understanding context from image-heavy accounts (e.g., Trump's Truth Social screenshots)

  • Task 3.4B: Image Generation Foundation (Future)

    • Owner: TBD

    • Status: 🔮 FUTURE TODO

    • Description: Build image generation capabilities for BILL's tweets

    • Files: agent/plugins/twitter/services/ImageGenerationService.ts, agent/plugins/twitter/utils/ImageGenerationDecider.ts

    • Future Acceptance Criteria:

      • Integrate with existing image generation APIs

      • Create decision logic for when to generate images

      • Build prompt engineering for BILL's visual style

      • Image metadata storage

      • Integration with PostingService

  • Task 3.4C: Image Generation Pipeline Integration (Collaboration)

    • Owner: TBD

    • Status: 🔮 FUTURE TODO (Depends on 3.4B)

    • Description: Integrate the image generation foundation into the posting pipeline, allowing BILL to generate and attach images to his posts.

    • Files: agent/plugins/twitter/services/PostingService.ts

    • Future Acceptance Criteria:

      • If the agent runtime's response contains a directive to generate an image, the PostingService calls the ImageGenerationService (from 3.4B).

      • The generated image is attached to the outgoing tweet.

      • Generated images are stored with metadata for future reference.

      • Error handling for image generation failures (fallback to text-only posts).

      • Integration with the LLM fallback system (Task 3.3) for image-related content issues.

  • Task 3.5: Improve Reply Filtering Logic

    • Owner: TBD

    • Status: 🔮 FUTURE TODO

    • Description: Currently timeline context fetching has replies disabled to prevent clutter during debugging. Need to implement smarter logic to determine when replies are valuable for context vs noise.

    • Files: agent/plugins/twitter/config/timelineContext.ts, agent/plugins/twitter/services/TimelineService.ts

    • Future Acceptance Criteria:

      • Analyze reply patterns to determine context value

      • Filter out low-value replies (e.g., single emoji responses)

      • Include high-value reply threads that provide market context

      • Add configuration for reply filtering strategies

      • Performance considerations for processing large reply volumes

  • Task 3.6: Logger-UI Integration

    • Owner: TBD

    • Status: 🔮 FUTURE TODO

    • Description: Integrate Twitter plugin activities with the logger-ui system for real-time monitoring and debugging of autonomous posting pipeline.

    • Files: agent/plugins/twitter/twitter-plugin.ts, agent/plugins/twitter/services/PostingService.ts, agent/plugins/twitter/services/TimelineService.ts

    • Future Acceptance Criteria:

      • Hook up Twitter plugin logging events to logger-ui

      • Display real-time posting pipeline activity in UI

      • Show timeline monitoring status and tweet processing events

      • Add context gathering and AI generation flow visualization

      • Include posting success/failure rates and error tracking

      • Add rate limiting status and API quota monitoring

      • Provide debugging interface for context quality and tweet generation

      • Integrate with existing logger-ui infrastructure

hashtag
Architecture Alignment

This TODO implements the following key architectural components:

hashtag
Background Context Ingestion (Continuous)

  • Timeline Monitor (Task 2.2): TimelineService fetches tweets every 30 minutes

  • Tweet Processing (Task 2.2): Processes images and stores descriptions

  • Database Storage (Task 1.1): Enhanced ConversationCacheService for persistence

hashtag
Real-time Context Retrieval (On-demand)

  • Context Manager (Task 2.3): TwitterContextManager retrieves relevant context

  • Agent Runtime Integration (Task 3.1): Enhanced context flows to LLM for response generation

  • Posting Pipeline (Task 3.1, 3.2): Generated content posted to Twitter

hashtag
Image Processing Pipeline

  • Input Analysis (Task 2.2): Analyze images in incoming tweets

  • Output Generation (Task 3.3): Generate images for outgoing posts

  • Storage Management (Task 1.1, 2.2): Supabase Storage integration

hashtag
Testing Strategy & Checkpoints

hashtag
🧪 Phase 1 Testing (After Tasks 1.1 & 1.2 Complete)

Status: ✅ ALL TESTS PASSING

Run these tests:

Prerequisites:

  1. Create the timeline_tweets table in Supabase (see SQL below)

  2. Ensure .env file has Supabase credentials

Required SQL:

What it tests:

  • ✅ ConversationCacheService (mentions/replies)

  • ✅ TimelineService (posting context)

  • ✅ PostedTweetsService (output tracking)

  • ✅ Timeline tweet storage/retrieval

  • ✅ Posted tweet tracking & analytics

  • 🚫 Similar content detection (temporarily disabled)

hashtag
🔧 Phase 1 Complete Testing (After Tasks 1.2 & 1.3)

Status: 🚧 TODO (waiting for Yin & Nikolai)

Integration tests:

  • Posted tweets tracking and duplicate prevention

  • Randomized posting intervals

  • Enhanced caching with scheduling

  • No regressions in existing functionality

hashtag
📊 Phase 2 Testing (After All Context Gathering)

Status: ✅ PHASE 2 COMPLETE - ALL CONTEXT PIPELINE OPERATIONAL

End-to-end context pipeline test:

  1. ✅ ContentSources → Target accounts loaded (timelineContext.ts)

  2. ✅ TimelineService → Tweets fetched and processed (integrated with TwitterPlugin)

  3. ✅ TwitterContextManager → Context retrieval and structuring works (Task 2.3 COMPLETE)

  4. ✅ Image processing → Download, analyze, store (basic implementation ready)

  5. ✅ Database flow → Timeline data flows to structured context for AI generation

Context Pipeline Architecture:

hashtag
🚀 Phase 3 Testing (Parallel & Integration)

Status: 🚧 TODO (depends on all tasks)

hashtag
Parallel Component Testing:

Yin's Component Tests:

Nikolai's Component Tests:

hashtag
Integration Testing:

Full autonomous posting test:

  1. Timeline monitoring → Context gathering → Context formatting → AI generation → Tweet posting

  2. LLM refusal detection and fallback model switching

  3. Image generation decision → Image creation → Image attachment

  4. Duplicate detection and prevention

  5. Error handling and graceful degradation

Testing Commands:

hashtag
🔄 Phase 3 Parallel Development Summary

hashtag
Yin's Independent Work Stream:

  1. Task 3.1A: Context-Agent Bridge

    • Build PostingContextAdapter for context formatting

    • Design context-aware prompt templates

    • Add validation and error handling

    • Files: utils/PostingContextAdapter.ts, config/contextPrompts.ts

  2. Task 3.4A: Image Generation Foundation

    • Integrate with existing ImageService

    • Build image generation decision logic

    • Create metadata management systems

    • Files: services/ImageGenerationService.ts, utils/ImageContextAnalyzer.ts

hashtag
Nikolai's Independent Work Stream:

  1. Task 3.1B: Core PostingService Refactor

    • Integrate context pipeline into posting flow

    • Handle agent runtime communication

    • Files: services/PostingService.ts

  2. Task 3.2: Plugin Orchestration Updates

    • Update twitter-plugin.ts integration

    • Coordinate timeline monitoring with posting

    • Files: twitter-plugin.ts

  3. Task 3.3: LLM Response Fallback System

    • Detect content policy refusals

    • Implement OpenRouter model switching

    • Files: services/PostingService.ts, utils/LLMFallbackHandler.ts

hashtag
Collaboration Points:

  • Task 3.4B: Image Generation Pipeline Integration (after 3.4A complete)

  • Integration Testing: Full system testing once all components are ready

hashtag
Dependencies:

  • 3.1B depends on 3.1A for context formatting interface

  • 3.4B depends on 3.4A for image generation foundation

  • All other tasks can be developed in parallel with minimal conflicts

PreviousTesting with Real DataNextTelegram Integration

Last updated