Integrating AutoGen and Semantic Kernel with Gemini Flash

This tutorial demonstrates how to integrate AutoGen and Semantic Kernel with Google's Gemini Flash model to build advanced multi-agent AI workflows. The integration begins by setting up the GeminiWrapper and SemanticKernelGeminiPlugin classes, which connect Gemini's generative capabilities with AutoGen's multi-agent orchestration framework.

Configuring Specialist Agents

Specialist agents such as code reviewers, creative analysts, data specialists, and assistants are configured using AutoGen's ConversableAgent API. These agents leverage Semantic Kernel's decorated functions for tasks including text analysis, summarization, code review, and creative problem-solving.

Core Dependencies and Setup

The required Python libraries include pyautogen, semantic-kernel, google-generativeai, and python-dotenv. Essential Python modules like os, asyncio, and typing are imported along with AutoGen for agent orchestration, Gemini API access via google.generativeai, and Semantic Kernel classes and decorators.

Gemini API Integration

The GeminiWrapper class encapsulates interactions with the Gemini API. It initializes a GenerativeModel and exposes a generate_response method that sends prompts to Gemini's generate_content endpoint, returning generated text or error messages.

Semantic Kernel Plugin Implementation

The SemanticKernelGeminiPlugin class uses the @kernel_function decorator to define AI functions:

• analyze_text: analyzes sentiment, key themes, insights, recommendations, and tone.
• generate_summary: produces executive summaries, key points, details, and conclusions.
• code_analysis: evaluates code quality, performance, security, maintainability, and suggestions.
• creative_solution: generates conventional, innovative, and hybrid solutions along with implementation strategies and challenges.

Each function constructs a detailed prompt and delegates processing to Gemini Flash.

Advanced Gemini Agent and Multi-Agent Collaboration

The AdvancedGeminiAgent class initializes the Semantic Kernel plugin, Gemini wrapper, and multiple AutoGen agents:

• Assistant
• Code Reviewer
• Creative Analyst
• Data Specialist
• User Proxy

It provides methods to bridge AutoGen with Semantic Kernel functions, orchestrate multi-agent collaboration on tasks, and perform direct Gemini calls for comprehensive analysis.

Running the Demo

The main() function initializes the advanced agent and runs demo queries showcasing various AI capabilities. Results from semantic kernel analyses, multi-agent collaboration, and direct Gemini responses are printed, illustrating the power of this multi-agent system.

Summary

This project highlights the synergy between AutoGen and Semantic Kernel to create a flexible, multi-agent AI system powered by Gemini Flash. AutoGen simplifies expert agent orchestration, while Semantic Kernel offers a declarative framework for AI function definition, enabling rapid prototyping and experimentation of complex AI workflows.

Installation Example

!pip install pyautogen semantic-kernel google-generativeai python-dotenv

Sample GeminiWrapper Class

class GeminiWrapper:
   """Wrapper for Gemini API to work with AutoGen"""
  
   def __init__(self, model_name="gemini-1.5-flash"):
       self.model = genai.GenerativeModel(model_name)
  
   def generate_response(self, prompt: str, temperature: float = 0.7) -> str:
       """Generate response using Gemini"""
       try:
           response = self.model.generate_content(
               prompt,
               generation_config=genai.types.GenerationConfig(
                   temperature=temperature,
                   max_output_tokens=2048,
               )
           )
           return response.text
       except Exception as e:
           return f"Gemini API Error: {str(e)}"

Sample SemanticKernelGeminiPlugin Function

@kernel_function(name="analyze_text", description="Analyze text for sentiment and key insights")
def analyze_text(self, text: str) -> str:
   """Analyze text using Gemini Flash"""
   prompt = f"""
   Analyze the following text comprehensively:
  
   Text: {text}
  
   Provide analysis in this format:
   - Sentiment: [positive/negative/neutral with confidence]
   - Key Themes: [main topics and concepts]
   - Insights: [important observations and patterns]
   - Recommendations: [actionable next steps]
   - Tone: [formal/informal/technical/emotional]
   """
  
   return self.gemini.generate_response(prompt, temperature=0.3)

This approach enables the development of an AI assistant capable of tackling diverse tasks with structured and actionable insights.