Orchestrating

Orchestration¶

Interaction and Orchestration Frameworks and SDKs¶

Handling the inputs/outputs to GenAI in a consistent and reliable manner has spurred the creation of software libraries that can work with GenAI that is called as a service, or hosted locally.

LangChain¶

Langchain is an open source SDK that allows for creation and management of chat and RAG based interactions. It has a Large user community emphasizing extensions to multiple types of models and documents. It has entrprise offerings with Langsmith for observability, Langserve for serving. It alos can enable multi-agent interactions with lang-graph.

Langchain Is a thorough python and javascript orchestration language for adaptable, memory and tooling-equipped calls that can enable agentic AI.

LangServe will provide a hosted version of LangServe for one-click deployments of LangChain applications.

OpenGPTs Provides an open-source effort to integrate multiple LLMs, and builds upon Langchain, LangServe, and LangSmith

Their Stack

Lang Smith for more Low-code solutions for agentic needs.

Langgraph

Langchain service deployment

Awesome Langchain

Langflow

Toolkit Generates LangChain plugins for javascript. May be deprecated.

Tutorials:

GPT and PDFS

Langchain prompt tmeplates

Deep learn langchain

¶

Other SDKs¶

Semantic Kernel

Github

EmbedChain is a framework to easily create LLM powered bots over any dataset.

Example:

    import os

    from embedchain import Llama2App

    os.environ['REPLICATE_API_TOKEN'] = "REPLICATE API TOKEN"

    zuck_bot = Llama2App()

    # Embed your data
    zuck_bot.add("youtube_video", "https://www.youtube.com/watch?v=Ff4fRgnuFgQ")
    zuck_bot.add("web_page", "https://en.wikipedia.org/wiki/Mark_Zuckerberg")

    # Nice, your bot is ready now. Start asking questions to your bot.
    zuck_bot.query("Who is Mark Zuckerberg?")
    # Answer: Mark Zuckerberg is an American internet entrepreneur and business magnate. He is the co-founder and CEO of Facebook.

txtai 'is an all-in-one embeddings database for semantic search, LLM orchestration and language model workflows.

Flowise
Chain Forge A data flow prompt engineering environment for evaluating ana analyzing LLM responses
llm-chain ChatGPT and Alpaca support. Agentic with bash commands.n
Agent Flow
Auto Chain
Chatall To interact with multiple chatbots at the same time.
LocalAI drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing.

Open Agent IN DEVELOPMENT Microservices approach to AGI. Modular components for AI apps or AGI agents

DSPY is a framework for solving advanced tasks with language models and retrieval models

Useful for exploring automatic prompt opteimization.

Language-like interfaces¶

LMQL is a query language that enables simplified representations of chats and agents with minimal code.

"Greet LMQL:[GREETINGS]\n" where stops_at(GREETINGS, ".") and not "\n" in GREETINGS

if "Hi there" in GREETINGS:
    "Can you reformulate your greeting in the speech of \
     victorian-era English: [VIC_GREETINGS]\n" where stops_at(VIC_GREETINGS, ".")

"Analyse what part of this response makes it typically victorian:\n"

for i in range(4):
    "-[THOUGHT]\n" where stops_at(THOUGHT, ".")

"To summarize:[SUMMARY]"

Control libraries¶

Guidance
RELM
Outlines

Retrieval Augmentation focus¶

is a framework that helps you evaluate your Retrieval Augmented Generation (RAG) RAGAS is a framework that helps you evaluate your Retrieval Augmented Generation (RAG) pipelines

Llama Index¶

Create Llama The easiest way to get started with LlamaIndex is by using create-llama.

This CLI tool enables you to quickly start building a new LlamaIndex application, with everything set up for you.

Llamaindex Provides an orchestration framework for with multiple connectors

Llama Lab enables flexible tools to use and indesx various tools

??? abstract " LLaMA2-Accessory: An Open-source Toolkit for LLM Development 🚀 🚀LLaMA2-Accessory is an open-source toolkit for pretraining, finetuning and deployment of Large Language Models (LLMs) and multimodal LLMs. This repo is mainly inherited from LLaMA-Adapter with more advanced features.🧠

LlamaGPT A self-hosted, offline, ChatGPT-like chatbot, powered by Llama 2. 100% private, with no data leaving your device.
Lit-Llama
MedAlpaca
Llama-2 on a CPU and Github
GPT LLM Training Generates and trains fine-tuned LLAMA-2 LLMs for specific tasks.
llama index at Github for integrating data ingestion and models.
LlamaHub (community library of data loaders)
LlamaLab (cutting-edge AGI projects using LlamaIndex)

Haystack¶

Haystack is an e2e llm orchestration framework that allows a number of versatile interactions.

Open source by DeepSet Designed for scaleable search and retrieval Evaluation pipelines for system eval Deployable as REST API

Griptape¶

Griptape an enterprise alternative to Langchain

Open source / managemed Commercial Support Optimized for scalability and cloud Encryption, access control, security

Others¶

AGent Smith AI makes it easy to instantiate AI agents that can safely and easily call APIs and locally defined functions to interact with the world.

Monarch Assistant Uses AGent Smith for RAG purposes

Curage GPT

Langfuse is an open Source LLM Engineering platform with Traces, Evals, Prompt management, metrics and a playground

Agents¶