Building a Collaborative AI Workflow: Multi-Agent Summarization with CrewAI, crewai-tools, and Hugging Face Transformers

Binance
Building a Collaborative AI Workflow: Multi-Agent Summarization with CrewAI, crewai-tools, and Hugging Face Transformers
fiverr


CrewAI is an open-source framework for orchestrating autonomous AI agents in a team. It allows you to create an AI “crew” where each agent has a specific role and goal and works together to accomplish complex tasks. In a CrewAI system, multiple agents can collaborate, share information, and coordinate their actions toward a common objective. This makes it possible to break down a problem into sub-tasks and have specialized agents tackle each part, much like a team of humans with different expertise.

In this tutorial, we’ll demonstrate a use case of multiple AI agents working together using CrewAI. Our example scenario will involve summarizing an article using three agents with distinct roles:

Research Assistant Agent – Reads the article and extracts the key points or facts.

Summarizer Agent – Takes the key points and concisely summarizes the article.

bybit

Writer Agent – Reviews the summary and formats it into a structured final output (for example, adding a title or conclusion).

This collaborative workflow mimics how a team might work: one member gathers information, another condenses it, and a third polishes the presentation. We will implement this workflow with CrewAI’s abstractions (Agents, Tasks, and Crew).

Code Implementation

!pip install crewai crewai-tools transformers

First we install all the necessary packages for the project in one go. The crewai and crewai-tools packages provide the framework and additional utilities for orchestrating AI agents, while the transformers package from Hugging Face supplies pre-trained models for text processing tasks like summarization.

from crewai import Agent, Task, Crew, Process

Here we import the core classes from the CrewAI framework. The Agent class lets you define an AI agent with a specific role and behavior, Task represents a unit of work assigned to an agent, Crew orchestrates the collaboration among these agents, and Process sets the execution workflow (like sequential or parallel).

# Define the agents’ roles, goals, and backstories to satisfy the model requirements.
research_agent = Agent(
role=”Research Assistant”,
goal=”Extract the main points and important facts from the article.”,
backstory=”You are a meticulous research assistant who carefully reads texts and extracts key points.”
)
summarizer_agent = Agent(
role=”Summarizer”,
goal=”Summarize the key points into a concise paragraph.”,
backstory=”You are an expert summarizer who can condense information into a clear and brief summary.”
)
writer_agent = Agent(
role=”Writer”,
goal=”Organize the summary into a final report with a title and conclusion.”,
backstory=”You are a creative writer with an eye for structure and clarity, ensuring the final output is polished.”
)

We define three specialized AI agents using the CrewAI framework through the above code. Each agent is configured with a specific role, goal, and backstory, which instructs them on how to contribute to the overall task: the Research Assistant extracts key points from an article, the Summarizer condenses those points into a concise paragraph, and the Writer formats the final output into a polished report.

# Example: Create tasks for each agent
research_task = Task(
description=”Read the article and identify the main points and important facts.”,
expected_output=”A list of bullet points summarizing the key information from the article.”,
agent=research_agent
)
summarization_task = Task(
description=”Take the above bullet points and summarize them into a concise paragraph that captures the article’s essence.”,
expected_output=”A brief paragraph summarizing the article.”,
agent=summarizer_agent
)
writing_task = Task(
description=”Review the summary paragraph and format it with a clear title and a concluding sentence.”,
expected_output=”A structured summary of the article with a title and conclusion.”,
agent=writer_agent
)

The above three task definitions assign specific responsibilities to the respective agents. The research_task instructs the Research Assistant to extract key points from the article, the summarization_task directs the Summarizer to turn those points into a concise paragraph, and the writing_task asks the Writer to format the summary into a structured final report with a title and conclusion.

# Create a crew with a sequential process
crew = Crew(
agents=[research_agent, summarizer_agent, writer_agent],
tasks=[research_task, summarization_task, writing_task],
process=Process.sequential,
verbose=True
)

print(“Agents defined successfully!”)

Now, we create a Crew object that orchestrates the collaboration between the three defined agents and their corresponding tasks in a sequential workflow. With Process.sequential, each task is executed one after the other, and setting verbose=True enables detailed logging of the process.

# Sample article text (as a multi-paragraph string)
article_text = “””Artificial intelligence (AI) has made significant inroads in various sectors, transforming how we work and live.
One of the most notable impacts of AI is seen in healthcare, where machine learning algorithms assist doctors in diagnosing diseases faster and more accurately.

In the automotive industry, AI powers self-driving cars, analyzing traffic patterns in real-time to ensure passenger safety.
This technology also plays a crucial role in finance, with AI-driven algorithms detecting fraudulent transactions and enabling personalized banking services.

Education is another field being revolutionized by AI. Intelligent tutoring systems and personalized learning platforms adapt to individual student needs, making education more accessible and effective.

Despite these advancements, AI adoption also raises important questions.
Concerns about job displacement, ethical use of AI, and ensuring data privacy are at the forefront of public discourse as AI continues to evolve.

Overall, AI’s growing influence across industries showcases its potential to tackle complex problems, but it also underscores the need for thoughtful integration to address the challenges that accompany these innovations.
“””
print(“Article length (characters):”, len(article_text))

In this part, we define a multi-paragraph string named article_text that simulates an article discussing AI’s impact across various industries, such as healthcare, automotive, finance, and education. It then prints the character length of the article, helping to verify that the text has been loaded correctly for further processing.

import re

def extract_key_points(text):
paragraphs = [p.strip() for p in text.split(“nn”) if p.strip()]
key_points = []
for para in paragraphs:
sentences = re.split(r'(?<=.)s+’, para.strip())
if not sentences:
continue
main_sentence = max(sentences, key=len)
point = main_sentence.strip()
if point and point[-1] not in “.!?”:
point += “.”
key_points.append(point)
return key_points

# Use the function on the article_text
key_points = extract_key_points(article_text)
print(“Research Assistant Agent – Key Points:n”)
for i, point in enumerate(key_points, 1):
print(f”{i}. {point}”)

Here we define a function extract_key_points that processes an article’s text by splitting it into paragraphs and further into sentences. For each paragraph, it selects the longest sentence as the key point (as a heuristic) and ensures it ends with proper punctuation. Finally, it prints each key point as a numbered list, simulating the Research Assistant agent’s output.

from transformers import pipeline

# Initialize a summarization pipeline
summarizer_pipeline = pipeline(“summarization”, model=”sshleifer/distilbart-cnn-12-6″)

# Prepare the input for summarization by joining the key points
input_text = ” “.join(key_points)
summary_output = summarizer_pipeline(input_text, max_length=100, min_length=30, do_sample=False)
summary_text = summary_output[0][‘summary_text’]
print(“Summarizer Agent – Summary:n”)
print(summary_text)

We initialize a Hugging Face summarization pipeline using the “sshleifer/distilbart-cnn-12-6” model. It then joins the extracted key points into a single string and feeds it to the pipeline, which generates a concise summary. Finally, it prints the resulting summary, simulating the output from the Summarizer Agent.

# Writer agent formatting
title = “AI’s Impact Across Industries: A Summary”
conclusion = “In conclusion, while AI offers tremendous benefits across sectors, addressing its challenges is crucial for its responsible adoption.”

# Combine title, summary, and conclusion
final_report = f”# {title}nn{summary_text}nn{conclusion}”
print(“Writer Agent – Final Output:n”)
print(final_report)

Finally, we simulate the Writer Agent’s work by formatting the final output. It defines a title and a conclusion and then combines these with the previously generated summary (stored in summary_text) using an f-string. The final report is formatted in Markdown, with the title as a header, followed by the summary and concluding sentence. It is then printed to display the complete, structured summary.

In conclusion, this tutorial showed how to create a team of AI agents using CrewAI and have them work together on a task. We structured a problem (summarizing an article) into sub-tasks handled by specialized agents and demonstrated the end-to-end flow with example outputs. CrewAI provides a flexible framework to manage these multi-agent collaborations, while we, as users, define the roles and processes that guide the agents’ teamwork.

Here is the Colab Notebook for the above project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. Don’t Forget to join our 80k+ ML SubReddit.

🚨 Recommended Read- LG AI Research Releases NEXUS: An Advanced System Integrating Agent AI System and Data Compliance Standards to Address Legal Concerns in AI Datasets

Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.

🚨 Recommended Open-Source AI Platform: ‘IntellAgent is a An Open-Source Multi-Agent Framework to Evaluate Complex Conversational AI System’ (Promoted)



Source link

Bybit

Be the first to comment

Leave a Reply

Your email address will not be published.


*