farcasclaudiu/Flowise
1
0
Fork 0
mirror of https://github.com/farcasclaudiu/Flowise.git synced 2026-06-28 11:00:55 +03:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into main

Browse Source
This commit is contained in:
Jakub Drobník
2023-07-31 13:15:05 +02:00
committed by GitHub
parent 2c0a8723f9 7c45961964
commit f3d74248dd
303 changed files with 20038 additions and 8006 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+2 -1
View File
@@ -8,6 +8,7 @@
**/yarn.lock
## logs
**/logs
**/*.log
## build
@@ -42,4 +43,4 @@
**/uploads
## compressed
**/*.tgz
**/*.tgz
CONTRIBUTING.md
+34 -1
View File
@@ -114,9 +114,42 @@ Flowise has 3 different modules in a single mono repository.
11. Commit code and submit Pull Request from forked branch pointing to [Flowise master](https://github.com/FlowiseAI/Flowise/tree/master).
## 🌱 Env Variables
Flowise support different environment variables to configure your instance. You can specify the following variables in the `.env` file inside `packages/server` folder. Read [more](https://docs.flowiseai.com/environment-variables)
| Variable | Description | Type | Default |
| -------------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------ | ----------------------------------- |
| PORT | The HTTP port Flowise runs on | Number | 3000 |
| FLOWISE_USERNAME | Username to login | String | |
| FLOWISE_PASSWORD | Password to login | String | |
| DEBUG | Print logs from components | Boolean | |
| LOG_PATH | Location where log files are stored | String | `your-path/Flowise/logs` |
| LOG_LEVEL | Different levels of logs | Enum String: `error`, `info`, `verbose`, `debug` | `info` |
| APIKEY_PATH | Location where api keys are saved | String | `your-path/Flowise/packages/server` |
| EXECUTION_MODE | Whether predictions run in their own process or the main process | Enum String: `child`, `main` | `main` |
| TOOL_FUNCTION_BUILTIN_DEP | NodeJS built-in modules to be used for Tool Function | String | |
| TOOL_FUNCTION_EXTERNAL_DEP | External modules to be used for Tool Function | String | |
| OVERRIDE_DATABASE | Override current database with default | Enum String: `true`, `false` | `true` |
| DATABASE_TYPE | Type of database to store the flowise data | Enum String: `sqlite`, `mysql`, `postgres` | `sqlite` |
| DATABASE_PATH | Location where database is saved (When DATABASE_TYPE is sqlite) | String | `your-home-dir/.flowise` |
| DATABASE_HOST | Host URL or IP address (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_PORT | Database port (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_USER | Database username (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_PASSWORD | Database password (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_NAME | Database name (When DATABASE_TYPE is not sqlite) | String | |
| PASSPHRASE | Passphrase used to create encryption key | String | `MYPASSPHRASE` |
| SECRETKEY_PATH | Location where encryption key (used to encrypt/decrypt credentials) is saved | String | `your-path/Flowise/packages/server` |
You can also specify the env variables when using `npx`. For example:
```
npx flowise start --PORT=3000 --DEBUG=true
```
## 📖 Contribute to Docs
In-Progress
[Flowise Docs](https://github.com/FlowiseAI/FlowiseDocs)
## 🏷️ Pull Request process
Dockerfile
+5
View File
@@ -8,7 +8,12 @@ FROM node:18-alpine
RUN apk add --update libc6-compat python3 make g++
# needed for pdfjs-dist
RUN apk add --no-cache build-base cairo-dev pango-dev
# Install Chromium
RUN apk add --no-cache chromium
ENV PUPPETEER_SKIP_DOWNLOAD=true
ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser
WORKDIR /usr/src/packages
README.md
+22 -5
View File
@@ -1,12 +1,19 @@
<!-- markdownlint-disable MD030 -->
# Flowise - LangchainJS UI
<img width="100%" src="https://github.com/FlowiseAI/Flowise/blob/main/images/flowise.png?raw=true"></a>
# Flowise - Build LLM Apps Easily
[![Release Notes](https://img.shields.io/github/release/FlowiseAI/Flowise)](https://github.com/FlowiseAI/Flowise/releases)
[![Discord](https://img.shields.io/discord/1087698854775881778?label=Discord&logo=discord)](https://discord.gg/jbaHfsRVBW)
[![Twitter Follow](https://img.shields.io/twitter/follow/FlowiseAI?style=social)](https://twitter.com/FlowiseAI)
[![GitHub star chart](https://img.shields.io/github/stars/FlowiseAI/Flowise?style=social)](https://star-history.com/#FlowiseAI/Flowise)
[![GitHub fork](https://img.shields.io/github/forks/FlowiseAI/Flowise?style=social)](https://github.com/FlowiseAI/Flowise/fork)
<h3>Drag & drop UI to build your customized LLM flow</h3>
<a href="https://github.com/FlowiseAI/Flowise">
<img width="100%" src="https://github.com/FlowiseAI/Flowise/blob/main/images/flowise.gif?raw=true"></a>
Drag & drop UI to build your customized LLM flow using [LangchainJS](https://github.com/hwchase17/langchainjs)
## ⚡Quick Start
Download and Install [NodeJS](https://nodejs.org/en/download) >= 18.15.0
@@ -34,7 +41,7 @@ Download and Install [NodeJS](https://nodejs.org/en/download) >= 18.15.0
### Docker Compose
1. Go to `docker` folder at the root of the project
2. Create `.env` file and specify the `PORT` (refer to `.env.example`)
2. Copy `.env.example` file, paste it into the same location, and rename to `.env`
3. `docker-compose up -d`
4. Open [http://localhost:3000](http://localhost:3000)
5. You can bring the containers down by `docker-compose stop`
@@ -126,6 +133,10 @@ FLOWISE_USERNAME=user
FLOWISE_PASSWORD=1234
```
## 🌱 Env Variables
Flowise support different environment variables to configure your instance. You can specify the following variables in the `.env` file inside `packages/server` folder. Read [more](https://github.com/FlowiseAI/Flowise/blob/main/CONTRIBUTING.md#-env-variables)
## 📖 Documentation
[Flowise Docs](https://docs.flowiseai.com/)
@@ -134,7 +145,7 @@ FLOWISE_PASSWORD=1234
### [Railway](https://docs.flowiseai.com/deployment/railway)
[![Deploy on Railway](https://railway.app/button.svg)](https://railway.app/template/YK7J0v)
[![Deploy on Railway](https://railway.app/button.svg)](https://railway.app/template/pn4G8S?referralCode=WVNPD9)
### [Render](https://docs.flowiseai.com/deployment/render)
@@ -158,6 +169,12 @@ Feel free to ask any questions, raise problems, and request new features in [dis
## 🙌 Contributing
Thanks go to these awesome contributors
<a href="https://github.com/FlowiseAI/Flowise/graphs/contributors">
<img src="https://contrib.rocks/image?repo=FlowiseAI/Flowise" />
</a>
See [contributing guide](CONTRIBUTING.md). Reach out to us at [Discord](https://discord.gg/jbaHfsRVBW) if you have any questions or issues.
[![Star History Chart](https://api.star-history.com/svg?repos=FlowiseAI/Flowise&type=Timeline)](https://star-history.com/#FlowiseAI/Flowise&Date)
artillery-load-test.yml
+36
View File
@@ -0,0 +1,36 @@
# npm install -g artillery@latest
# artillery run artillery-load-test.yml
# Refer https://www.artillery.io/docs
config:
target: http://128.128.128.128:3000 # replace with your url
phases:
- duration: 1
arrivalRate: 1
rampTo: 2
name: Warm up phase
- duration: 1
arrivalRate: 2
rampTo: 3
name: Ramp up load
- duration: 1
arrivalRate: 3
name: Sustained peak load
scenarios:
- flow:
- loop:
- post:
url: '/api/v1/prediction/chatflow-id' # replace with your chatflowid
json:
question: 'hello' # replace with your question
count: 1 # how many request each user make
# User __
# 3 /
# 2 /
# 1 _/
# 1 2 3
# Seconds
# Total Users = 2 + 3 + 3 = 8
# Each making 1 HTTP call
# Over a duration of 3 seconds
docker/.env.example
+19 -2
View File
@@ -1,5 +1,22 @@
PORT=3000
PASSPHRASE=MYPASSPHRASE # Passphrase used to create encryption key
DATABASE_PATH=/root/.flowise
APIKEY_PATH=/root/.flowise
SECRETKEY_PATH=/root/.flowise
LOG_PATH=/root/.flowise/logs
# DATABASE_TYPE=postgres
# DATABASE_PORT=""
# DATABASE_HOST=""
# DATABASE_NAME="flowise"
# DATABASE_USER=""
# DATABASE_PASSWORD=""
# OVERRIDE_DATABASE=true
# FLOWISE_USERNAME=user
# FLOWISE_PASSWORD=1234
# DATABASE_PATH=/your_database_path/.flowise
# EXECUTION_MODE=child or main
# DEBUG=true
# LOG_LEVEL=debug (error | warn | info | verbose | debug)
# EXECUTION_MODE=main (child | main)
# TOOL_FUNCTION_BUILTIN_DEP=crypto,fs
# TOOL_FUNCTION_EXTERNAL_DEP=moment,lodash
docker/Dockerfile
+5
View File
@@ -6,7 +6,12 @@ RUN apk add --no-cache git
RUN apk add --no-cache python3 py3-pip make g++
# needed for pdfjs-dist
RUN apk add --no-cache build-base cairo-dev pango-dev
# Install Chromium
RUN apk add --no-cache chromium
ENV PUPPETEER_SKIP_DOWNLOAD=true
ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser
# You can install a specific version like: flowise@1.0.0
RUN npm install -g flowise
docker/README.md
+12 -1
View File
@@ -9,7 +9,7 @@ Starts Flowise from [DockerHub Image](https://hub.docker.com/repository/docker/f
3. Open [http://localhost:3000](http://localhost:3000)
4. You can bring the containers down by `docker-compose stop`
## With Authrorization
## 🔒 Authentication
1. Create `.env` file and specify the `PORT`, `FLOWISE_USERNAME`, and `FLOWISE_PASSWORD` (refer to `.env.example`)
2. Pass `FLOWISE_USERNAME` and `FLOWISE_PASSWORD` to the `docker-compose.yml` file:
@@ -22,3 +22,14 @@ Starts Flowise from [DockerHub Image](https://hub.docker.com/repository/docker/f
3. `docker-compose up -d`
4. Open [http://localhost:3000](http://localhost:3000)
5. You can bring the containers down by `docker-compose stop`
## 🌱 Env Variables
If you like to persist your data (flows, logs, apikeys, credentials), set these variables in the `.env` file inside `docker` folder:
- DATABASE_PATH=/root/.flowise
- APIKEY_PATH=/root/.flowise
- LOG_PATH=/root/.flowise/logs
- SECRETKEY_PATH=/root/.flowise
Flowise also support different environment variables to configure your instance. Read [more](https://docs.flowiseai.com/environment-variables)
docker/docker-compose.yml
+6
View File
@@ -6,9 +6,15 @@ services:
restart: always
environment:
- PORT=${PORT}
- PASSPHRASE=${PASSPHRASE}
- FLOWISE_USERNAME=${FLOWISE_USERNAME}
- FLOWISE_PASSWORD=${FLOWISE_PASSWORD}
- DEBUG=${DEBUG}
- DATABASE_PATH=${DATABASE_PATH}
- APIKEY_PATH=${APIKEY_PATH}
- SECRETKEY_PATH=${SECRETKEY_PATH}
- LOG_LEVEL=${LOG_LEVEL}
- LOG_PATH=${LOG_PATH}
- EXECUTION_MODE=${EXECUTION_MODE}
ports:
- '${PORT}:${PORT}'
images/flowise.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 524 KiB

package.json
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "flowise",
"version": "1.2.13",
"version": "1.3.2",
"private": true,
"homepage": "https://flowiseai.com",
"workspaces": [
@@ -28,7 +28,6 @@
"*.{js,jsx,ts,tsx,json,md}": "eslint --fix"
},
"devDependencies": {
"turbo": "1.7.4",
"@babel/preset-env": "^7.19.4",
"@babel/preset-typescript": "7.18.6",
"@types/express": "^4.17.13",
@@ -48,6 +47,7 @@
"pretty-quick": "^3.1.3",
"rimraf": "^3.0.2",
"run-script-os": "^1.1.6",
"turbo": "1.7.4",
"typescript": "^4.8.4"
},
"engines": {
packages/components/.env.example
-1
View File
@@ -1 +0,0 @@
DEBUG=true
packages/components/README.md
-8
View File
@@ -12,14 +12,6 @@ Install:
npm i flowise-components
```
## Debug
To view all the logs, create an `.env` file and add:
```
DEBUG=true
```
## License
Source code in this repository is made available under the [MIT License](https://github.com/FlowiseAI/Flowise/blob/master/LICENSE.md).
packages/components/credentials/AirtableApi.credential.ts
+27
View File
@@ -0,0 +1,27 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class AirtableApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Airtable API'
this.name = 'airtableApi'
this.version = 1.0
this.description =
'Refer to <a target="_blank" href="https://support.airtable.com/docs/creating-and-using-api-keys-and-access-tokens">official guide</a> on how to get accessToken on Airtable'
this.inputs = [
{
label: 'Access Token',
name: 'accessToken',
type: 'password',
placeholder: '<AIRTABLE_ACCESS_TOKEN>'
}
]
}
}
module.exports = { credClass: AirtableApi }
packages/components/credentials/AnthropicApi.credential.ts
+23
View File
@@ -0,0 +1,23 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class AnthropicApi implements INodeCredential {
label: string
name: string
version: number
inputs: INodeParams[]
constructor() {
this.label = 'Anthropic API'
this.name = 'anthropicApi'
this.version = 1.0
this.inputs = [
{
label: 'Anthropic Api Key',
name: 'anthropicApiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: AnthropicApi }
packages/components/credentials/AzureOpenAIApi.credential.ts
+47
View File
@@ -0,0 +1,47 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class AzureOpenAIApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Azure OpenAI API'
this.name = 'azureOpenAIApi'
this.version = 1.0
this.description =
'Refer to <a target="_blank" href="https://azure.microsoft.com/en-us/products/cognitive-services/openai-service">official guide</a> of how to use Azure OpenAI service'
this.inputs = [
{
label: 'Azure OpenAI Api Key',
name: 'azureOpenAIApiKey',
type: 'password',
description: `Refer to <a target="_blank" href="https://learn.microsoft.com/en-us/azure/cognitive-services/openai/quickstart?tabs=command-line&pivots=rest-api#set-up">official guide</a> on how to create API key on Azure OpenAI`
},
{
label: 'Azure OpenAI Api Instance Name',
name: 'azureOpenAIApiInstanceName',
type: 'string',
placeholder: 'YOUR-INSTANCE-NAME'
},
{
label: 'Azure OpenAI Api Deployment Name',
name: 'azureOpenAIApiDeploymentName',
type: 'string',
placeholder: 'YOUR-DEPLOYMENT-NAME'
},
{
label: 'Azure OpenAI Api Version',
name: 'azureOpenAIApiVersion',
type: 'string',
placeholder: '2023-06-01-preview',
description:
'Description of Supported API Versions. Please refer <a target="_blank" href="https://learn.microsoft.com/en-us/azure/cognitive-services/openai/reference#chat-completions">examples</a>'
}
]
}
}
module.exports = { credClass: AzureOpenAIApi }
packages/components/credentials/BraveSearchApi.credential.ts
+24
View File
@@ -0,0 +1,24 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class BraveSearchApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Brave Search API'
this.name = 'braveSearchApi'
this.version = 1.0
this.inputs = [
{
label: 'BraveSearch Api Key',
name: 'braveApiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: BraveSearchApi }
packages/components/credentials/CohereApi.credential.ts
+23
View File
@@ -0,0 +1,23 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class CohereApi implements INodeCredential {
label: string
name: string
version: number
inputs: INodeParams[]
constructor() {
this.label = 'Cohere API'
this.name = 'cohereApi'
this.version = 1.0
this.inputs = [
{
label: 'Cohere Api Key',
name: 'cohereApiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: CohereApi }
packages/components/credentials/ConfluenceApi.credential.ts
+33
View File
@@ -0,0 +1,33 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class ConfluenceApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Confluence API'
this.name = 'confluenceApi'
this.version = 1.0
this.description =
'Refer to <a target="_blank" href="https://support.atlassian.com/confluence-cloud/docs/manage-oauth-access-tokens/">official guide</a> on how to get accessToken on Confluence'
this.inputs = [
{
label: 'Access Token',
name: 'accessToken',
type: 'password',
placeholder: '<CONFLUENCE_ACCESS_TOKEN>'
},
{
label: 'Username',
name: 'username',
type: 'string',
placeholder: '<CONFLUENCE_USERNAME>'
}
]
}
}
module.exports = { credClass: ConfluenceApi }
packages/components/credentials/DynamodbMemoryApi.credential.ts
+29
View File
@@ -0,0 +1,29 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class DynamodbMemoryApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'DynamodbMemory API'
this.name = 'dynamodbMemoryApi'
this.version = 1.0
this.inputs = [
{
label: 'Access Key',
name: 'accessKey',
type: 'password'
},
{
label: 'Secret Access Key',
name: 'secretAccessKey',
type: 'password'
}
]
}
}
module.exports = { credClass: DynamodbMemoryApi }
packages/components/credentials/FigmaApi.credential.ts
+27
View File
@@ -0,0 +1,27 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class FigmaApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Figma API'
this.name = 'figmaApi'
this.version = 1.0
this.description =
'Refer to <a target="_blank" href="https://www.figma.com/developers/api#access-tokens">official guide</a> on how to get accessToken on Figma'
this.inputs = [
{
label: 'Access Token',
name: 'accessToken',
type: 'password',
placeholder: '<FIGMA_ACCESS_TOKEN>'
}
]
}
}
module.exports = { credClass: FigmaApi }
packages/components/credentials/GithubApi.credential.ts
+27
View File
@@ -0,0 +1,27 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class GithubApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Github API'
this.name = 'githubApi'
this.version = 1.0
this.description =
'Refer to <a target="_blank" href="https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens">official guide</a> on how to get accessToken on Github'
this.inputs = [
{
label: 'Access Token',
name: 'accessToken',
type: 'password',
placeholder: '<GITHUB_ACCESS_TOKEN>'
}
]
}
}
module.exports = { credClass: GithubApi }
packages/components/credentials/GoogleSearchApi.credential.ts
+31
View File
@@ -0,0 +1,31 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class GoogleSearchApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Google Custom Search API'
this.name = 'googleCustomSearchApi'
this.version = 1.0
this.description =
'Please refer to the <a target="_blank" href="https://console.cloud.google.com/apis/credentials">Google Cloud Console</a> for instructions on how to create an API key, and visit the <a target="_blank" href="https://programmablesearchengine.google.com/controlpanel/create">Search Engine Creation page</a> to learn how to generate your Search Engine ID.'
this.inputs = [
{
label: 'Google Custom Search Api Key',
name: 'googleCustomSearchApiKey',
type: 'password'
},
{
label: 'Programmable Search Engine ID',
name: 'googleCustomSearchApiId',
type: 'string'
}
]
}
}
module.exports = { credClass: GoogleSearchApi }
packages/components/credentials/HuggingFaceApi.credential.ts
+23
View File
@@ -0,0 +1,23 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class HuggingFaceApi implements INodeCredential {
label: string
name: string
version: number
inputs: INodeParams[]
constructor() {
this.label = 'HuggingFace API'
this.name = 'huggingFaceApi'
this.version = 1.0
this.inputs = [
{
label: 'HuggingFace Api Key',
name: 'huggingFaceApiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: HuggingFaceApi }
packages/components/credentials/MotorheadMemoryApi.credential.ts
+31
View File
@@ -0,0 +1,31 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class MotorheadMemoryApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Motorhead Memory API'
this.name = 'motorheadMemoryApi'
this.version = 1.0
this.description =
'Refer to <a target="_blank" href="https://docs.getmetal.io/misc-get-keys">official guide</a> on how to create API key and Client ID on Motorhead Memory'
this.inputs = [
{
label: 'Client ID',
name: 'clientId',
type: 'string'
},
{
label: 'API Key',
name: 'apiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: MotorheadMemoryApi }
packages/components/credentials/NotionApi.credential.ts
+26
View File
@@ -0,0 +1,26 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class NotionApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Notion API'
this.name = 'notionApi'
this.version = 1.0
this.description =
'You can find integration token <a target="_blank" href="https://developers.notion.com/docs/create-a-notion-integration#step-1-create-an-integration">here</a>'
this.inputs = [
{
label: 'Notion Integration Token',
name: 'notionIntegrationToken',
type: 'password'
}
]
}
}
module.exports = { credClass: NotionApi }
packages/components/credentials/OpenAIApi.credential.ts
+23
View File
@@ -0,0 +1,23 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class OpenAIApi implements INodeCredential {
label: string
name: string
version: number
inputs: INodeParams[]
constructor() {
this.label = 'OpenAI API'
this.name = 'openAIApi'
this.version = 1.0
this.inputs = [
{
label: 'OpenAI Api Key',
name: 'openAIApiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: OpenAIApi }
packages/components/credentials/OpenAPIAuth.credential.ts
+25
View File
@@ -0,0 +1,25 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class OpenAPIAuth implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'OpenAPI Auth Token'
this.name = 'openAPIAuth'
this.version = 1.0
this.inputs = [
{
label: 'OpenAPI Token',
name: 'openAPIToken',
type: 'password',
description: 'Auth Token. For example: Bearer <TOKEN>'
}
]
}
}
module.exports = { credClass: OpenAPIAuth }
packages/components/credentials/PineconeApi.credential.ts
+29
View File
@@ -0,0 +1,29 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class PineconeApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Pinecone API'
this.name = 'pineconeApi'
this.version = 1.0
this.inputs = [
{
label: 'Pinecone Api Key',
name: 'pineconeApiKey',
type: 'password'
},
{
label: 'Pinecone Environment',
name: 'pineconeEnv',
type: 'string'
}
]
}
}
module.exports = { credClass: PineconeApi }
packages/components/credentials/QdrantApi.credential.ts
+24
View File
@@ -0,0 +1,24 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class QdrantApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Qdrant API'
this.name = 'qdrantApi'
this.version = 1.0
this.inputs = [
{
label: 'Qdrant API Key',
name: 'qdrantApiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: QdrantApi }
packages/components/credentials/ReplicateApi.credential.ts
+23
View File
@@ -0,0 +1,23 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class ReplicateApi implements INodeCredential {
label: string
name: string
version: number
inputs: INodeParams[]
constructor() {
this.label = 'Replicate API'
this.name = 'replicateApi'
this.version = 1.0
this.inputs = [
{
label: 'Replicate Api Key',
name: 'replicateApiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: ReplicateApi }
packages/components/credentials/SerpApi.credential.ts
+24
View File
@@ -0,0 +1,24 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class SerpApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Serp API'
this.name = 'serpApi'
this.version = 1.0
this.inputs = [
{
label: 'Serp Api Key',
name: 'serpApiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: SerpApi }
packages/components/credentials/SerperApi.credential.ts
+24
View File
@@ -0,0 +1,24 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class SerperApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Serper API'
this.name = 'serperApi'
this.version = 1.0
this.inputs = [
{
label: 'Serper Api Key',
name: 'serperApiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: SerperApi }
packages/components/credentials/SingleStoreApi.credential.ts
+31
View File
@@ -0,0 +1,31 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class SingleStoreApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'SingleStore API'
this.name = 'singleStoreApi'
this.version = 1.0
this.inputs = [
{
label: 'User',
name: 'user',
type: 'string',
placeholder: '<SINGLESTORE_USERNAME>'
},
{
label: 'Password',
name: 'password',
type: 'password',
placeholder: '<SINGLESTORE_PASSWORD>'
}
]
}
}
module.exports = { credClass: SingleStoreApi }
packages/components/credentials/SupabaseApi.credential.ts
+24
View File
@@ -0,0 +1,24 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class SupabaseApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Supabase API'
this.name = 'supabaseApi'
this.version = 1.0
this.inputs = [
{
label: 'Supabase API Key',
name: 'supabaseApiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: SupabaseApi }
packages/components/credentials/VectaraApi.credential.ts
+34
View File
@@ -0,0 +1,34 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class VectaraAPI implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Vectara API'
this.name = 'vectaraApi'
this.version = 1.0
this.inputs = [
{
label: 'Vectara Customer ID',
name: 'customerID',
type: 'string'
},
{
label: 'Vectara Corpus ID',
name: 'corpusID',
type: 'string'
},
{
label: 'Vectara API Key',
name: 'apiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: VectaraAPI }
packages/components/credentials/WeaviateApi.credential.ts
+24
View File
@@ -0,0 +1,24 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class WeaviateApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Weaviate API'
this.name = 'weaviateApi'
this.version = 1.0
this.inputs = [
{
label: 'Weaviate API Key',
name: 'weaviateApiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: WeaviateApi }
packages/components/credentials/ZapierNLAApi.credential.ts
+24
View File
@@ -0,0 +1,24 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class ZapierNLAApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Zapier NLA API'
this.name = 'zapierNLAApi'
this.version = 1.0
this.inputs = [
{
label: 'Zapier NLA Api Key',
name: 'zapierNLAApiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: ZapierNLAApi }
packages/components/credentials/ZepMemoryApi.credential.ts
+26
View File
@@ -0,0 +1,26 @@
import { INodeParams, INodeCredential } from '../src/Interface'
class ZepMemoryApi implements INodeCredential {
label: string
name: string
version: number
description: string
inputs: INodeParams[]
constructor() {
this.label = 'Zep Memory API'
this.name = 'zepMemoryApi'
this.version = 1.0
this.description =
'Refer to <a target="_blank" href="https://docs.getzep.com/deployment/auth/">official guide</a> on how to create API key on Zep'
this.inputs = [
{
label: 'API Key',
name: 'apiKey',
type: 'password'
}
]
}
}
module.exports = { credClass: ZepMemoryApi }
packages/components/nodes/agents/AirtableAgent/AirtableAgent.ts
+232
View File
@@ -0,0 +1,232 @@
import { ICommonObject, INode, INodeData, INodeParams, PromptTemplate } from '../../../src/Interface'
import { AgentExecutor } from 'langchain/agents'
import { getBaseClasses, getCredentialData, getCredentialParam } from '../../../src/utils'
import { LoadPyodide, finalSystemPrompt, systemPrompt } from './core'
import { LLMChain } from 'langchain/chains'
import { BaseLanguageModel } from 'langchain/base_language'
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
import axios from 'axios'
class Airtable_Agents implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
category: string
baseClasses: string[]
credential: INodeParams
inputs: INodeParams[]
constructor() {
this.label = 'Airtable Agent'
this.name = 'airtableAgent'
this.version = 1.0
this.type = 'AgentExecutor'
this.category = 'Agents'
this.icon = 'airtable.svg'
this.description = 'Agent used to to answer queries on Airtable table'
this.baseClasses = [this.type, ...getBaseClasses(AgentExecutor)]
this.credential = {
label: 'Connect Credential',
name: 'credential',
type: 'credential',
credentialNames: ['airtableApi']
}
this.inputs = [
{
label: 'Language Model',
name: 'model',
type: 'BaseLanguageModel'
},
{
label: 'Base Id',
name: 'baseId',
type: 'string',
placeholder: 'app11RobdGoX0YNsC',
description:
'If your table URL looks like: https://airtable.com/app11RobdGoX0YNsC/tblJdmvbrgizbYICO/viw9UrP77Id0CE4ee, app11RovdGoX0YNsC is the base id'
},
{
label: 'Table Id',
name: 'tableId',
type: 'string',
placeholder: 'tblJdmvbrgizbYICO',
description:
'If your table URL looks like: https://airtable.com/app11RobdGoX0YNsC/tblJdmvbrgizbYICO/viw9UrP77Id0CE4ee, tblJdmvbrgizbYICO is the table id'
},
{
label: 'Return All',
name: 'returnAll',
type: 'boolean',
default: true,
additionalParams: true,
description: 'If all results should be returned or only up to a given limit'
},
{
label: 'Limit',
name: 'limit',
type: 'number',
default: 100,
additionalParams: true,
description: 'Number of results to return'
}
]
}
async init(): Promise<any> {
// Not used
return undefined
}
async run(nodeData: INodeData, input: string, options: ICommonObject): Promise<string> {
const model = nodeData.inputs?.model as BaseLanguageModel
const baseId = nodeData.inputs?.baseId as string
const tableId = nodeData.inputs?.tableId as string
const returnAll = nodeData.inputs?.returnAll as boolean
const limit = nodeData.inputs?.limit as string
const credentialData = await getCredentialData(nodeData.credential ?? '', options)
const accessToken = getCredentialParam('accessToken', credentialData, nodeData)
let airtableData: ICommonObject[] = []
if (returnAll) {
airtableData = await loadAll(baseId, tableId, accessToken)
} else {
airtableData = await loadLimit(limit ? parseInt(limit, 10) : 100, baseId, tableId, accessToken)
}
let base64String = Buffer.from(JSON.stringify(airtableData)).toString('base64')
const loggerHandler = new ConsoleCallbackHandler(options.logger)
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId)
const pyodide = await LoadPyodide()
// First load the csv file and get the dataframe dictionary of column types
// For example using titanic.csv: {'PassengerId': 'int64', 'Survived': 'int64', 'Pclass': 'int64', 'Name': 'object', 'Sex': 'object', 'Age': 'float64', 'SibSp': 'int64', 'Parch': 'int64', 'Ticket': 'object', 'Fare': 'float64', 'Cabin': 'object', 'Embarked': 'object'}
let dataframeColDict = ''
try {
const code = `import pandas as pd
import base64
import json
base64_string = "${base64String}"
decoded_data = base64.b64decode(base64_string)
json_data = json.loads(decoded_data)
df = pd.DataFrame(json_data)
my_dict = df.dtypes.astype(str).to_dict()
print(my_dict)
json.dumps(my_dict)`
dataframeColDict = await pyodide.runPythonAsync(code)
} catch (error) {
throw new Error(error)
}
// Then tell GPT to come out with ONLY python code
// For example: len(df), df[df['SibSp'] > 3]['PassengerId'].count()
let pythonCode = ''
if (dataframeColDict) {
const chain = new LLMChain({
llm: model,
prompt: PromptTemplate.fromTemplate(systemPrompt),
verbose: process.env.DEBUG === 'true' ? true : false
})
const inputs = {
dict: dataframeColDict,
question: input
}
const res = await chain.call(inputs, [loggerHandler])
pythonCode = res?.text
}
// Then run the code using Pyodide
let finalResult = ''
if (pythonCode) {
try {
const code = `import pandas as pd\n${pythonCode}`
finalResult = await pyodide.runPythonAsync(code)
} catch (error) {
throw new Error(`Sorry, I'm unable to find answer for question: "${input}" using follwoing code: "${pythonCode}"`)
}
}
// Finally, return a complete answer
if (finalResult) {
const chain = new LLMChain({
llm: model,
prompt: PromptTemplate.fromTemplate(finalSystemPrompt),
verbose: process.env.DEBUG === 'true' ? true : false
})
const inputs = {
question: input,
answer: finalResult
}
if (options.socketIO && options.socketIOClientId) {
const result = await chain.call(inputs, [loggerHandler, handler])
return result?.text
} else {
const result = await chain.call(inputs, [loggerHandler])
return result?.text
}
}
return pythonCode
}
}
interface AirtableLoaderResponse {
records: AirtableLoaderPage[]
offset?: string
}
interface AirtableLoaderPage {
id: string
createdTime: string
fields: ICommonObject
}
const fetchAirtableData = async (url: string, params: ICommonObject, accessToken: string): Promise<AirtableLoaderResponse> => {
try {
const headers = {
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
Accept: 'application/json'
}
const response = await axios.get(url, { params, headers })
return response.data
} catch (error) {
throw new Error(`Failed to fetch ${url} from Airtable: ${error}`)
}
}
const loadAll = async (baseId: string, tableId: string, accessToken: string): Promise<ICommonObject[]> => {
const params: ICommonObject = { pageSize: 100 }
let data: AirtableLoaderResponse
let returnPages: AirtableLoaderPage[] = []
do {
data = await fetchAirtableData(`https://api.airtable.com/v0/${baseId}/${tableId}`, params, accessToken)
returnPages.push.apply(returnPages, data.records)
params.offset = data.offset
} while (data.offset !== undefined)
return data.records.map((page) => page.fields)
}
const loadLimit = async (limit: number, baseId: string, tableId: string, accessToken: string): Promise<ICommonObject[]> => {
const params = { maxRecords: limit }
const data = await fetchAirtableData(`https://api.airtable.com/v0/${baseId}/${tableId}`, params, accessToken)
if (data.records.length === 0) {
return []
}
return data.records.map((page) => page.fields)
}
module.exports = { nodeClass: Airtable_Agents }
packages/components/nodes/agents/AirtableAgent/airtable.svg
+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg width="256px" height="215px" viewBox="0 0 256 215" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" preserveAspectRatio="xMidYMid">
<g>
<path d="M114.25873,2.70101695 L18.8604023,42.1756384 C13.5552723,44.3711638 13.6102328,51.9065311 18.9486282,54.0225085 L114.746142,92.0117514 C123.163769,95.3498757 132.537419,95.3498757 140.9536,92.0117514 L236.75256,54.0225085 C242.08951,51.9065311 242.145916,44.3711638 236.83934,42.1756384 L141.442459,2.70101695 C132.738459,-0.900338983 122.961284,-0.900338983 114.25873,2.70101695" fill="#FFBF00"></path>
<path d="M136.349071,112.756863 L136.349071,207.659101 C136.349071,212.173089 140.900664,215.263892 145.096461,213.600615 L251.844122,172.166219 C254.281184,171.200072 255.879376,168.845451 255.879376,166.224705 L255.879376,71.3224678 C255.879376,66.8084791 251.327783,63.7176768 247.131986,65.3809537 L140.384325,106.815349 C137.94871,107.781496 136.349071,110.136118 136.349071,112.756863" fill="#26B5F8"></path>
<path d="M111.422771,117.65355 L79.742409,132.949912 L76.5257763,134.504714 L9.65047684,166.548104 C5.4112904,168.593211 0.000578531073,165.503855 0.000578531073,160.794612 L0.000578531073,71.7210757 C0.000578531073,70.0173017 0.874160452,68.5463864 2.04568588,67.4384994 C2.53454463,66.9481944 3.08848814,66.5446689 3.66412655,66.2250305 C5.26231864,65.2661153 7.54173107,65.0101153 9.47981017,65.7766689 L110.890522,105.957098 C116.045234,108.002206 116.450206,115.225166 111.422771,117.65355" fill="#ED3049"></path>
<path d="M111.422771,117.65355 L79.742409,132.949912 L2.04568588,67.4384994 C2.53454463,66.9481944 3.08848814,66.5446689 3.66412655,66.2250305 C5.26231864,65.2661153 7.54173107,65.0101153 9.47981017,65.7766689 L110.890522,105.957098 C116.045234,108.002206 116.450206,115.225166 111.422771,117.65355" fill-opacity="0.25" fill="#000000"></path>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.9 KiB

packages/components/nodes/agents/AirtableAgent/core.ts
+29
View File
@@ -0,0 +1,29 @@
import type { PyodideInterface } from 'pyodide'
import * as path from 'path'
import { getUserHome } from '../../../src/utils'
let pyodideInstance: PyodideInterface | undefined
export async function LoadPyodide(): Promise<PyodideInterface> {
if (pyodideInstance === undefined) {
const { loadPyodide } = await import('pyodide')
const obj: any = { packageCacheDir: path.join(getUserHome(), '.flowise', 'pyodideCacheDir') }
pyodideInstance = await loadPyodide(obj)
await pyodideInstance.loadPackage(['pandas', 'numpy'])
}
return pyodideInstance
}
export const systemPrompt = `You are working with a pandas dataframe in Python. The name of the dataframe is df.
The columns and data types of a dataframe are given below as a Python dictionary with keys showing column names and values showing the data types.
{dict}
I will ask question, and you will output the Python code using pandas dataframe to answer my question. Do not provide any explanations. Do not respond with anything except the output of the code.
Question: {question}
Output Code:`
export const finalSystemPrompt = `You are given the question: {question}. You have an answer to the question: {answer}. Rephrase the answer into a standalone answer.
Standalone Answer:`
packages/components/nodes/agents/AutoGPT/AutoGPT.ts
+2 -1
View File
@@ -8,6 +8,7 @@ import { flatten } from 'lodash'
class AutoGPT_Agents implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -18,6 +19,7 @@ class AutoGPT_Agents implements INode {
constructor() {
this.label = 'AutoGPT'
this.name = 'autoGPT'
this.version = 1.0
this.type = 'AutoGPT'
this.category = 'Agents'
this.icon = 'autogpt.png'
@@ -90,7 +92,6 @@ class AutoGPT_Agents implements INode {
const res = await executor.run([input])
return res || 'I have completed all my tasks.'
} catch (e) {
console.error(e)
throw new Error(e)
}
}
packages/components/nodes/agents/BabyAGI/BabyAGI.ts
+2
View File
@@ -6,6 +6,7 @@ import { VectorStore } from 'langchain/vectorstores'
class BabyAGI_Agents implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -16,6 +17,7 @@ class BabyAGI_Agents implements INode {
constructor() {
this.label = 'BabyAGI'
this.name = 'babyAGI'
this.version = 1.0
this.type = 'BabyAGI'
this.category = 'Agents'
this.icon = 'babyagi.jpg'
packages/components/nodes/agents/CSVAgent/CSVAgent.ts
+151
View File
@@ -0,0 +1,151 @@
import { ICommonObject, INode, INodeData, INodeParams, PromptTemplate } from '../../../src/Interface'
import { AgentExecutor } from 'langchain/agents'
import { getBaseClasses } from '../../../src/utils'
import { LoadPyodide, finalSystemPrompt, systemPrompt } from './core'
import { LLMChain } from 'langchain/chains'
import { BaseLanguageModel } from 'langchain/base_language'
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
class CSV_Agents implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
category: string
baseClasses: string[]
inputs: INodeParams[]
constructor() {
this.label = 'CSV Agent'
this.name = 'csvAgent'
this.version = 1.0
this.type = 'AgentExecutor'
this.category = 'Agents'
this.icon = 'csvagent.png'
this.description = 'Agent used to to answer queries on CSV data'
this.baseClasses = [this.type, ...getBaseClasses(AgentExecutor)]
this.inputs = [
{
label: 'Csv File',
name: 'csvFile',
type: 'file',
fileType: '.csv'
},
{
label: 'Language Model',
name: 'model',
type: 'BaseLanguageModel'
}
]
}
async init(): Promise<any> {
// Not used
return undefined
}
async run(nodeData: INodeData, input: string, options: ICommonObject): Promise<string> {
const csvFileBase64 = nodeData.inputs?.csvFile as string
const model = nodeData.inputs?.model as BaseLanguageModel
const loggerHandler = new ConsoleCallbackHandler(options.logger)
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId)
let files: string[] = []
if (csvFileBase64.startsWith('[') && csvFileBase64.endsWith(']')) {
files = JSON.parse(csvFileBase64)
} else {
files = [csvFileBase64]
}
let base64String = ''
for (const file of files) {
const splitDataURI = file.split(',')
splitDataURI.pop()
base64String = splitDataURI.pop() ?? ''
}
const pyodide = await LoadPyodide()
// First load the csv file and get the dataframe dictionary of column types
// For example using titanic.csv: {'PassengerId': 'int64', 'Survived': 'int64', 'Pclass': 'int64', 'Name': 'object', 'Sex': 'object', 'Age': 'float64', 'SibSp': 'int64', 'Parch': 'int64', 'Ticket': 'object', 'Fare': 'float64', 'Cabin': 'object', 'Embarked': 'object'}
let dataframeColDict = ''
try {
const code = `import pandas as pd
import base64
from io import StringIO
import json
base64_string = "${base64String}"
decoded_data = base64.b64decode(base64_string)
csv_data = StringIO(decoded_data.decode('utf-8'))
df = pd.read_csv(csv_data)
my_dict = df.dtypes.astype(str).to_dict()
print(my_dict)
json.dumps(my_dict)`
dataframeColDict = await pyodide.runPythonAsync(code)
} catch (error) {
throw new Error(error)
}
// Then tell GPT to come out with ONLY python code
// For example: len(df), df[df['SibSp'] > 3]['PassengerId'].count()
let pythonCode = ''
if (dataframeColDict) {
const chain = new LLMChain({
llm: model,
prompt: PromptTemplate.fromTemplate(systemPrompt),
verbose: process.env.DEBUG === 'true' ? true : false
})
const inputs = {
dict: dataframeColDict,
question: input
}
const res = await chain.call(inputs, [loggerHandler])
pythonCode = res?.text
}
// Then run the code using Pyodide
let finalResult = ''
if (pythonCode) {
try {
const code = `import pandas as pd\n${pythonCode}`
finalResult = await pyodide.runPythonAsync(code)
} catch (error) {
throw new Error(`Sorry, I'm unable to find answer for question: "${input}" using follwoing code: "${pythonCode}"`)
}
}
// Finally, return a complete answer
if (finalResult) {
const chain = new LLMChain({
llm: model,
prompt: PromptTemplate.fromTemplate(finalSystemPrompt),
verbose: process.env.DEBUG === 'true' ? true : false
})
const inputs = {
question: input,
answer: finalResult
}
if (options.socketIO && options.socketIOClientId) {
const result = await chain.call(inputs, [loggerHandler, handler])
return result?.text
} else {
const result = await chain.call(inputs, [loggerHandler])
return result?.text
}
}
return pythonCode
}
}
module.exports = { nodeClass: CSV_Agents }
packages/components/nodes/agents/CSVAgent/core.ts
+29
View File
@@ -0,0 +1,29 @@
import type { PyodideInterface } from 'pyodide'
import * as path from 'path'
import { getUserHome } from '../../../src/utils'
let pyodideInstance: PyodideInterface | undefined
export async function LoadPyodide(): Promise<PyodideInterface> {
if (pyodideInstance === undefined) {
const { loadPyodide } = await import('pyodide')
const obj: any = { packageCacheDir: path.join(getUserHome(), '.flowise', 'pyodideCacheDir') }
pyodideInstance = await loadPyodide(obj)
await pyodideInstance.loadPackage(['pandas', 'numpy'])
}
return pyodideInstance
}
export const systemPrompt = `You are working with a pandas dataframe in Python. The name of the dataframe is df.
The columns and data types of a dataframe are given below as a Python dictionary with keys showing column names and values showing the data types.
{dict}
I will ask question, and you will output the Python code using pandas dataframe to answer my question. Do not provide any explanations. Do not respond with anything except the output of the code.
Question: {question}
Output Code:`
export const finalSystemPrompt = `You are given the question: {question}. You have an answer to the question: {answer}. Rephrase the answer into a standalone answer.
Standalone Answer:`
packages/components/nodes/agents/CSVAgent/csvagent.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.0 KiB

packages/components/nodes/agents/ConversationalAgent/ConversationalAgent.ts
+14 -15
View File
@@ -3,13 +3,22 @@ import { initializeAgentExecutorWithOptions, AgentExecutor, InitializeAgentExecu
import { Tool } from 'langchain/tools'
import { BaseChatMemory, ChatMessageHistory } from 'langchain/memory'
import { getBaseClasses } from '../../../src/utils'
import { AIChatMessage, HumanChatMessage } from 'langchain/schema'
import { AIMessage, HumanMessage } from 'langchain/schema'
import { BaseLanguageModel } from 'langchain/base_language'
import { flatten } from 'lodash'
const DEFAULT_PREFIX = `Assistant is a large language model trained by OpenAI.
Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.
Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.
Overall, Assistant is a powerful system that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.`
class ConversationalAgent_Agents implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -20,6 +29,7 @@ class ConversationalAgent_Agents implements INode {
constructor() {
this.label = 'Conversational Agent'
this.name = 'conversationalAgent'
this.version = 1.0
this.type = 'AgentExecutor'
this.category = 'Agents'
this.icon = 'agent.svg'
@@ -47,14 +57,7 @@ class ConversationalAgent_Agents implements INode {
name: 'systemMessage',
type: 'string',
rows: 4,
optional: true,
additionalParams: true
},
{
label: 'Human Message',
name: 'humanMessage',
type: 'string',
rows: 4,
default: DEFAULT_PREFIX,
optional: true,
additionalParams: true
}
@@ -66,7 +69,6 @@ class ConversationalAgent_Agents implements INode {
let tools = nodeData.inputs?.tools as Tool[]
tools = flatten(tools)
const memory = nodeData.inputs?.memory as BaseChatMemory
const humanMessage = nodeData.inputs?.humanMessage as string
const systemMessage = nodeData.inputs?.systemMessage as string
const obj: InitializeAgentExecutorOptions = {
@@ -75,9 +77,6 @@ class ConversationalAgent_Agents implements INode {
}
const agentArgs: any = {}
if (humanMessage) {
agentArgs.humanMessage = humanMessage
}
if (systemMessage) {
agentArgs.systemMessage = systemMessage
}
@@ -99,9 +98,9 @@ class ConversationalAgent_Agents implements INode {
for (const message of histories) {
if (message.type === 'apiMessage') {
chatHistory.push(new AIChatMessage(message.message))
chatHistory.push(new AIMessage(message.message))
} else if (message.type === 'userMessage') {
chatHistory.push(new HumanChatMessage(message.message))
chatHistory.push(new HumanMessage(message.message))
}
}
memory.chatHistory = new ChatMessageHistory(chatHistory)
packages/components/nodes/agents/MRKLAgentChat/MRKLAgentChat.ts
+2
View File
@@ -8,6 +8,7 @@ import { flatten } from 'lodash'
class MRKLAgentChat_Agents implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -18,6 +19,7 @@ class MRKLAgentChat_Agents implements INode {
constructor() {
this.label = 'MRKL Agent for Chat Models'
this.name = 'mrklAgentChat'
this.version = 1.0
this.type = 'AgentExecutor'
this.category = 'Agents'
this.icon = 'agent.svg'
packages/components/nodes/agents/MRKLAgentLLM/MRKLAgentLLM.ts
+2
View File
@@ -8,6 +8,7 @@ import { flatten } from 'lodash'
class MRKLAgentLLM_Agents implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -18,6 +19,7 @@ class MRKLAgentLLM_Agents implements INode {
constructor() {
this.label = 'MRKL Agent for LLMs'
this.name = 'mrklAgentLLM'
this.version = 1.0
this.type = 'AgentExecutor'
this.category = 'Agents'
this.icon = 'agent.svg'
packages/components/nodes/agents/OpenAIFunctionAgent/OpenAIFunctionAgent.ts
+11 -6
View File
@@ -1,14 +1,16 @@
import { ICommonObject, IMessage, INode, INodeData, INodeParams } from '../../../src/Interface'
import { initializeAgentExecutorWithOptions, AgentExecutor } from 'langchain/agents'
import { CustomChainHandler, getBaseClasses } from '../../../src/utils'
import { getBaseClasses } from '../../../src/utils'
import { BaseLanguageModel } from 'langchain/base_language'
import { flatten } from 'lodash'
import { BaseChatMemory, ChatMessageHistory } from 'langchain/memory'
import { AIChatMessage, HumanChatMessage } from 'langchain/schema'
import { AIMessage, HumanMessage } from 'langchain/schema'
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
class OpenAIFunctionAgent_Agents implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -19,6 +21,7 @@ class OpenAIFunctionAgent_Agents implements INode {
constructor() {
this.label = 'OpenAI Function Agent'
this.name = 'openAIFunctionAgent'
this.version = 1.0
this.type = 'AgentExecutor'
this.category = 'Agents'
this.icon = 'openai.png'
@@ -84,21 +87,23 @@ class OpenAIFunctionAgent_Agents implements INode {
for (const message of histories) {
if (message.type === 'apiMessage') {
chatHistory.push(new AIChatMessage(message.message))
chatHistory.push(new AIMessage(message.message))
} else if (message.type === 'userMessage') {
chatHistory.push(new HumanChatMessage(message.message))
chatHistory.push(new HumanMessage(message.message))
}
}
memory.chatHistory = new ChatMessageHistory(chatHistory)
executor.memory = memory
}
const loggerHandler = new ConsoleCallbackHandler(options.logger)
if (options.socketIO && options.socketIOClientId) {
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId)
const result = await executor.run(input, [handler])
const result = await executor.run(input, [loggerHandler, handler])
return result
} else {
const result = await executor.run(input)
const result = await executor.run(input, [loggerHandler])
return result
}
}
packages/components/nodes/chains/ApiChain/GETApiChain.ts
+8 -3
View File
@@ -1,8 +1,9 @@
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { APIChain } from 'langchain/chains'
import { CustomChainHandler, getBaseClasses } from '../../../src/utils'
import { getBaseClasses } from '../../../src/utils'
import { BaseLanguageModel } from 'langchain/base_language'
import { PromptTemplate } from 'langchain/prompts'
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
export const API_URL_RAW_PROMPT_TEMPLATE = `You are given the below API Documentation:
{api_docs}
@@ -18,6 +19,7 @@ export const API_RESPONSE_RAW_PROMPT_TEMPLATE =
class GETApiChain_Chains implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
@@ -28,6 +30,7 @@ class GETApiChain_Chains implements INode {
constructor() {
this.label = 'GET API Chain'
this.name = 'getApiChain'
this.version = 1.0
this.type = 'GETApiChain'
this.icon = 'apichain.svg'
this.category = 'Chains'
@@ -95,12 +98,14 @@ class GETApiChain_Chains implements INode {
const ansPrompt = nodeData.inputs?.ansPrompt as string
const chain = await getAPIChain(apiDocs, model, headers, urlPrompt, ansPrompt)
const loggerHandler = new ConsoleCallbackHandler(options.logger)
if (options.socketIO && options.socketIOClientId) {
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId, 2)
const res = await chain.run(input, [handler])
const res = await chain.run(input, [loggerHandler, handler])
return res
} else {
const res = await chain.run(input)
const res = await chain.run(input, [loggerHandler])
return res
}
}
packages/components/nodes/chains/ApiChain/OpenAPIChain.ts
+100
View File
@@ -0,0 +1,100 @@
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { APIChain, createOpenAPIChain } from 'langchain/chains'
import { getBaseClasses } from '../../../src/utils'
import { ChatOpenAI } from 'langchain/chat_models/openai'
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
class OpenApiChain_Chains implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
baseClasses: string[]
description: string
inputs: INodeParams[]
constructor() {
this.label = 'OpenAPI Chain'
this.name = 'openApiChain'
this.version = 1.0
this.type = 'OpenAPIChain'
this.icon = 'openapi.png'
this.category = 'Chains'
this.description = 'Chain that automatically select and call APIs based only on an OpenAPI spec'
this.baseClasses = [this.type, ...getBaseClasses(APIChain)]
this.inputs = [
{
label: 'ChatOpenAI Model',
name: 'model',
type: 'ChatOpenAI'
},
{
label: 'YAML Link',
name: 'yamlLink',
type: 'string',
placeholder: 'https://api.speak.com/openapi.yaml',
description: 'If YAML link is provided, uploaded YAML File will be ignored and YAML link will be used instead'
},
{
label: 'YAML File',
name: 'yamlFile',
type: 'file',
fileType: '.yaml',
description: 'If YAML link is provided, uploaded YAML File will be ignored and YAML link will be used instead'
},
{
label: 'Headers',
name: 'headers',
type: 'json',
additionalParams: true,
optional: true
}
]
}
async init(nodeData: INodeData): Promise<any> {
return await initChain(nodeData)
}
async run(nodeData: INodeData, input: string, options: ICommonObject): Promise<string> {
const chain = await initChain(nodeData)
const loggerHandler = new ConsoleCallbackHandler(options.logger)
if (options.socketIO && options.socketIOClientId) {
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId)
const res = await chain.run(input, [loggerHandler, handler])
return res
} else {
const res = await chain.run(input, [loggerHandler])
return res
}
}
}
const initChain = async (nodeData: INodeData) => {
const model = nodeData.inputs?.model as ChatOpenAI
const headers = nodeData.inputs?.headers as string
const yamlLink = nodeData.inputs?.yamlLink as string
const yamlFileBase64 = nodeData.inputs?.yamlFile as string
let yamlString = ''
if (yamlLink) {
yamlString = yamlLink
} else {
const splitDataURI = yamlFileBase64.split(',')
splitDataURI.pop()
const bf = Buffer.from(splitDataURI.pop() || '', 'base64')
yamlString = bf.toString('utf-8')
}
return await createOpenAPIChain(yamlString, {
llm: model,
headers: typeof headers === 'object' ? headers : headers ? JSON.parse(headers) : {},
verbose: process.env.DEBUG === 'true' ? true : false
})
}
module.exports = { nodeClass: OpenApiChain_Chains }
packages/components/nodes/chains/ApiChain/POSTApiChain.ts
+8 -3
View File
@@ -1,12 +1,14 @@
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { CustomChainHandler, getBaseClasses } from '../../../src/utils'
import { getBaseClasses } from '../../../src/utils'
import { BaseLanguageModel } from 'langchain/base_language'
import { PromptTemplate } from 'langchain/prompts'
import { API_RESPONSE_RAW_PROMPT_TEMPLATE, API_URL_RAW_PROMPT_TEMPLATE, APIChain } from './postCore'
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
class POSTApiChain_Chains implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
@@ -17,6 +19,7 @@ class POSTApiChain_Chains implements INode {
constructor() {
this.label = 'POST API Chain'
this.name = 'postApiChain'
this.version = 1.0
this.type = 'POSTApiChain'
this.icon = 'apichain.svg'
this.category = 'Chains'
@@ -84,12 +87,14 @@ class POSTApiChain_Chains implements INode {
const ansPrompt = nodeData.inputs?.ansPrompt as string
const chain = await getAPIChain(apiDocs, model, headers, urlPrompt, ansPrompt)
const loggerHandler = new ConsoleCallbackHandler(options.logger)
if (options.socketIO && options.socketIOClientId) {
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId, 2)
const res = await chain.run(input, [handler])
const res = await chain.run(input, [loggerHandler, handler])
return res
} else {
const res = await chain.run(input)
const res = await chain.run(input, [loggerHandler])
return res
}
}
packages/components/nodes/chains/ApiChain/openapi.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 24 KiB

packages/components/nodes/chains/ConversationChain/ConversationChain.ts
+39 -7
View File
@@ -1,16 +1,20 @@
import { ICommonObject, IMessage, INode, INodeData, INodeParams } from '../../../src/Interface'
import { ConversationChain } from 'langchain/chains'
import { CustomChainHandler, getBaseClasses } from '../../../src/utils'
import { getBaseClasses } from '../../../src/utils'
import { ChatPromptTemplate, HumanMessagePromptTemplate, MessagesPlaceholder, SystemMessagePromptTemplate } from 'langchain/prompts'
import { BufferMemory, ChatMessageHistory } from 'langchain/memory'
import { BaseChatModel } from 'langchain/chat_models/base'
import { AIChatMessage, HumanChatMessage } from 'langchain/schema'
import { AIMessage, HumanMessage } from 'langchain/schema'
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
import { flatten } from 'lodash'
import { Document } from 'langchain/document'
const systemMessage = `The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.`
let systemMessage = `The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.`
class ConversationChain_Chains implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
@@ -21,6 +25,7 @@ class ConversationChain_Chains implements INode {
constructor() {
this.label = 'Conversation Chain'
this.name = 'conversationChain'
this.version = 1.0
this.type = 'ConversationChain'
this.icon = 'chain.svg'
this.category = 'Chains'
@@ -37,6 +42,14 @@ class ConversationChain_Chains implements INode {
name: 'memory',
type: 'BaseMemory'
},
{
label: 'Document',
name: 'document',
type: 'Document',
description: 'Include whole document into the context window',
optional: true,
list: true
},
{
label: 'System Message',
name: 'systemMessagePrompt',
@@ -53,6 +66,23 @@ class ConversationChain_Chains implements INode {
const model = nodeData.inputs?.model as BaseChatModel
const memory = nodeData.inputs?.memory as BufferMemory
const prompt = nodeData.inputs?.systemMessagePrompt as string
const docs = nodeData.inputs?.document as Document[]
const flattenDocs = docs && docs.length ? flatten(docs) : []
const finalDocs = []
for (let i = 0; i < flattenDocs.length; i += 1) {
finalDocs.push(new Document(flattenDocs[i]))
}
let finalText = ''
for (let i = 0; i < finalDocs.length; i += 1) {
finalText += finalDocs[i].pageContent
}
const replaceChar: string[] = ['{', '}']
for (const char of replaceChar) finalText = finalText.replaceAll(char, '')
if (finalText) systemMessage = `${systemMessage}\nThe AI has the following context:\n${finalText}`
const obj: any = {
llm: model,
@@ -81,21 +111,23 @@ class ConversationChain_Chains implements INode {
for (const message of histories) {
if (message.type === 'apiMessage') {
chatHistory.push(new AIChatMessage(message.message))
chatHistory.push(new AIMessage(message.message))
} else if (message.type === 'userMessage') {
chatHistory.push(new HumanChatMessage(message.message))
chatHistory.push(new HumanMessage(message.message))
}
}
memory.chatHistory = new ChatMessageHistory(chatHistory)
chain.memory = memory
}
const loggerHandler = new ConsoleCallbackHandler(options.logger)
if (options.socketIO && options.socketIOClientId) {
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId)
const res = await chain.call({ input }, [handler])
const res = await chain.call({ input }, [loggerHandler, handler])
return res?.response
} else {
const res = await chain.call({ input })
const res = await chain.call({ input }, [loggerHandler])
return res?.response
}
}
packages/components/nodes/chains/ConversationalRetrievalQAChain/ConversationalRetrievalQAChain.ts
+98 -35
View File
@@ -1,28 +1,26 @@
import { BaseLanguageModel } from 'langchain/base_language'
import { ICommonObject, IMessage, INode, INodeData, INodeParams } from '../../../src/Interface'
import { CustomChainHandler, getBaseClasses } from '../../../src/utils'
import { ConversationalRetrievalQAChain } from 'langchain/chains'
import { AIChatMessage, BaseRetriever, HumanChatMessage } from 'langchain/schema'
import { BaseChatMemory, BufferMemory, ChatMessageHistory } from 'langchain/memory'
import { getBaseClasses } from '../../../src/utils'
import { ConversationalRetrievalQAChain, QAChainParams } from 'langchain/chains'
import { AIMessage, HumanMessage } from 'langchain/schema'
import { BaseRetriever } from 'langchain/schema/retriever'
import { BaseChatMemory, BufferMemory, ChatMessageHistory, BufferMemoryInput } from 'langchain/memory'
import { PromptTemplate } from 'langchain/prompts'
const default_qa_template = `Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.
{context}
Question: {question}
Helpful Answer:`
const qa_template = `Use the following pieces of context to answer the question at the end.
{context}
Question: {question}
Helpful Answer:`
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
import {
default_map_reduce_template,
default_qa_template,
qa_template,
map_reduce_template,
CUSTOM_QUESTION_GENERATOR_CHAIN_PROMPT,
refine_question_template,
refine_template
} from './prompts'
class ConversationalRetrievalQAChain_Chains implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
@@ -33,6 +31,7 @@ class ConversationalRetrievalQAChain_Chains implements INode {
constructor() {
this.label = 'Conversational Retrieval QA Chain'
this.name = 'conversationalRetrievalQAChain'
this.version = 1.0
this.type = 'ConversationalRetrievalQAChain'
this.icon = 'chain.svg'
this.category = 'Chains'
@@ -49,6 +48,13 @@ class ConversationalRetrievalQAChain_Chains implements INode {
name: 'vectorStoreRetriever',
type: 'BaseRetriever'
},
{
label: 'Memory',
name: 'memory',
type: 'BaseMemory',
optional: true,
description: 'If left empty, a default BufferMemory will be used'
},
{
label: 'Return Source Documents',
name: 'returnSourceDocuments',
@@ -99,22 +105,59 @@ class ConversationalRetrievalQAChain_Chains implements INode {
const systemMessagePrompt = nodeData.inputs?.systemMessagePrompt as string
const returnSourceDocuments = nodeData.inputs?.returnSourceDocuments as boolean
const chainOption = nodeData.inputs?.chainOption as string
const memory = nodeData.inputs?.memory
const obj: any = {
verbose: process.env.DEBUG === 'true' ? true : false,
qaChainOptions: {
type: 'stuff',
prompt: PromptTemplate.fromTemplate(systemMessagePrompt ? `${systemMessagePrompt}\n${qa_template}` : default_qa_template)
},
memory: new BufferMemory({
memoryKey: 'chat_history',
inputKey: 'question',
outputKey: 'text',
returnMessages: true
})
questionGeneratorChainOptions: {
template: CUSTOM_QUESTION_GENERATOR_CHAIN_PROMPT
}
}
if (returnSourceDocuments) obj.returnSourceDocuments = returnSourceDocuments
if (chainOption) obj.qaChainOptions = { ...obj.qaChainOptions, type: chainOption }
if (chainOption === 'map_reduce') {
obj.qaChainOptions = {
type: 'map_reduce',
combinePrompt: PromptTemplate.fromTemplate(
systemMessagePrompt ? `${systemMessagePrompt}\n${map_reduce_template}` : default_map_reduce_template
)
} as QAChainParams
} else if (chainOption === 'refine') {
const qprompt = new PromptTemplate({
inputVariables: ['context', 'question'],
template: refine_question_template(systemMessagePrompt)
})
const rprompt = new PromptTemplate({
inputVariables: ['context', 'question', 'existing_answer'],
template: refine_template
})
obj.qaChainOptions = {
type: 'refine',
questionPrompt: qprompt,
refinePrompt: rprompt
} as QAChainParams
} else {
obj.qaChainOptions = {
type: 'stuff',
prompt: PromptTemplate.fromTemplate(systemMessagePrompt ? `${systemMessagePrompt}\n${qa_template}` : default_qa_template)
} as QAChainParams
}
if (memory) {
memory.inputKey = 'question'
memory.memoryKey = 'chat_history'
if (chainOption === 'refine') memory.outputKey = 'output_text'
else memory.outputKey = 'text'
obj.memory = memory
} else {
const fields: BufferMemoryInput = {
memoryKey: 'chat_history',
inputKey: 'question',
returnMessages: true
}
if (chainOption === 'refine') fields.outputKey = 'output_text'
else fields.outputKey = 'text'
obj.memory = new BufferMemory(fields)
}
const chain = ConversationalRetrievalQAChain.fromLLM(model, vectorStoreRetriever, obj)
return chain
@@ -123,6 +166,9 @@ class ConversationalRetrievalQAChain_Chains implements INode {
async run(nodeData: INodeData, input: string, options: ICommonObject): Promise<string | ICommonObject> {
const chain = nodeData.instance as ConversationalRetrievalQAChain
const returnSourceDocuments = nodeData.inputs?.returnSourceDocuments as boolean
const memory = nodeData.inputs?.memory
const chainOption = nodeData.inputs?.chainOption as string
let model = nodeData.inputs?.model
// Temporary fix: https://github.com/hwchase17/langchainjs/issues/754
@@ -131,29 +177,46 @@ class ConversationalRetrievalQAChain_Chains implements INode {
const obj = { question: input }
if (chain.memory && options && options.chatHistory) {
// If external memory like Zep, Redis is being used, ignore below
if (!memory && chain.memory && options && options.chatHistory) {
const chatHistory = []
const histories: IMessage[] = options.chatHistory
const memory = chain.memory as BaseChatMemory
for (const message of histories) {
if (message.type === 'apiMessage') {
chatHistory.push(new AIChatMessage(message.message))
chatHistory.push(new AIMessage(message.message))
} else if (message.type === 'userMessage') {
chatHistory.push(new HumanChatMessage(message.message))
chatHistory.push(new HumanMessage(message.message))
}
}
memory.chatHistory = new ChatMessageHistory(chatHistory)
chain.memory = memory
}
const loggerHandler = new ConsoleCallbackHandler(options.logger)
if (options.socketIO && options.socketIOClientId) {
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId, undefined, returnSourceDocuments)
const res = await chain.call(obj, [handler])
const handler = new CustomChainHandler(
options.socketIO,
options.socketIOClientId,
chainOption === 'refine' ? 4 : undefined,
returnSourceDocuments
)
const res = await chain.call(obj, [loggerHandler, handler])
if (chainOption === 'refine') {
if (res.output_text && res.sourceDocuments) {
return {
text: res.output_text,
sourceDocuments: res.sourceDocuments
}
}
return res?.output_text
}
if (res.text && res.sourceDocuments) return res
return res?.text
} else {
const res = await chain.call(obj)
const res = await chain.call(obj, [loggerHandler])
if (res.text && res.sourceDocuments) return res
return res?.text
}
packages/components/nodes/chains/ConversationalRetrievalQAChain/prompts.ts
+64
View File
@@ -0,0 +1,64 @@
export const default_qa_template = `Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.
{context}
Question: {question}
Helpful Answer:`
export const qa_template = `Use the following pieces of context to answer the question at the end.
{context}
Question: {question}
Helpful Answer:`
export const default_map_reduce_template = `Given the following extracted parts of a long document and a question, create a final answer.
If you don't know the answer, just say that you don't know. Don't try to make up an answer.
{summaries}
Question: {question}
Helpful Answer:`
export const map_reduce_template = `Given the following extracted parts of a long document and a question, create a final answer.
{summaries}
Question: {question}
Helpful Answer:`
export const refine_question_template = (sysPrompt?: string) => {
let returnPrompt = ''
if (sysPrompt)
returnPrompt = `Context information is below.
---------------------
{context}
---------------------
Given the context information and not prior knowledge, ${sysPrompt}
Answer the question: {question}.
Answer:`
if (!sysPrompt)
returnPrompt = `Context information is below.
---------------------
{context}
---------------------
Given the context information and not prior knowledge, answer the question: {question}.
Answer:`
return returnPrompt
}
export const refine_template = `The original question is as follows: {question}
We have provided an existing answer: {existing_answer}
We have the opportunity to refine the existing answer (only if needed) with some more context below.
------------
{context}
------------
Given the new context, refine the original answer to better answer the question.
If you can't find answer from the context, return the original answer.`
export const CUSTOM_QUESTION_GENERATOR_CHAIN_PROMPT = `Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question, answer in the same language as the follow up question. include it in the standalone question.
Chat History:
{chat_history}
Follow Up Input: {question}
Standalone question:`
packages/components/nodes/chains/LLMChain/LLMChain.ts
+36 -32
View File
@@ -1,11 +1,13 @@
import { ICommonObject, INode, INodeData, INodeOutputsValue, INodeParams } from '../../../src/Interface'
import { CustomChainHandler, getBaseClasses } from '../../../src/utils'
import { getBaseClasses, handleEscapeCharacters } from '../../../src/utils'
import { LLMChain } from 'langchain/chains'
import { BaseLanguageModel } from 'langchain/base_language'
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
class LLMChain_Chains implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
@@ -17,6 +19,7 @@ class LLMChain_Chains implements INode {
constructor() {
this.label = 'LLM Chain'
this.name = 'llmChain'
this.version = 1.0
this.type = 'LLMChain'
this.icon = 'chain.svg'
this.category = 'Chains'
@@ -50,12 +53,12 @@ class LLMChain_Chains implements INode {
{
label: 'Output Prediction',
name: 'outputPrediction',
baseClasses: ['string']
baseClasses: ['string', 'json']
}
]
}
async init(nodeData: INodeData, input: string): Promise<any> {
async init(nodeData: INodeData, input: string, options: ICommonObject): Promise<any> {
const model = nodeData.inputs?.model as BaseLanguageModel
const prompt = nodeData.inputs?.prompt
const output = nodeData.outputs?.output as string
@@ -67,12 +70,17 @@ class LLMChain_Chains implements INode {
} else if (output === 'outputPrediction') {
const chain = new LLMChain({ llm: model, prompt, verbose: process.env.DEBUG === 'true' ? true : false })
const inputVariables = chain.prompt.inputVariables as string[] // ["product"]
const res = await runPrediction(inputVariables, chain, input, promptValues)
const res = await runPrediction(inputVariables, chain, input, promptValues, options)
// eslint-disable-next-line no-console
console.log('\x1b[92m\x1b[1m\n*****OUTPUT PREDICTION*****\n\x1b[0m\x1b[0m')
// eslint-disable-next-line no-console
console.log(res)
return res
/**
* Apply string transformation to convert special chars:
* FROM: hello i am ben\n\n\thow are you?
* TO: hello i am benFLOWISE_NEWLINEFLOWISE_NEWLINEFLOWISE_TABhow are you?
*/
return handleEscapeCharacters(res, false)
}
}
@@ -80,10 +88,7 @@ class LLMChain_Chains implements INode {
const inputVariables = nodeData.instance.prompt.inputVariables as string[] // ["product"]
const chain = nodeData.instance as LLMChain
const promptValues = nodeData.inputs?.prompt.promptValues as ICommonObject
const res = options.socketIO
? await runPrediction(inputVariables, chain, input, promptValues, true, options.socketIO, options.socketIOClientId)
: await runPrediction(inputVariables, chain, input, promptValues)
const res = await runPrediction(inputVariables, chain, input, promptValues, options)
// eslint-disable-next-line no-console
console.log('\x1b[93m\x1b[1m\n*****FINAL RESULT*****\n\x1b[0m\x1b[0m')
// eslint-disable-next-line no-console
@@ -96,21 +101,22 @@ const runPrediction = async (
inputVariables: string[],
chain: LLMChain,
input: string,
promptValues: ICommonObject,
isStreaming?: boolean,
socketIO?: any,
socketIOClientId = ''
promptValuesRaw: ICommonObject,
options: ICommonObject
) => {
if (inputVariables.length === 1) {
if (isStreaming) {
const handler = new CustomChainHandler(socketIO, socketIOClientId)
const res = await chain.run(input, [handler])
return res
} else {
const res = await chain.run(input)
return res
}
} else if (inputVariables.length > 1) {
const loggerHandler = new ConsoleCallbackHandler(options.logger)
const isStreaming = options.socketIO && options.socketIOClientId
const socketIO = isStreaming ? options.socketIO : undefined
const socketIOClientId = isStreaming ? options.socketIOClientId : ''
/**
* Apply string transformation to reverse converted special chars:
* FROM: { "value": "hello i am benFLOWISE_NEWLINEFLOWISE_NEWLINEFLOWISE_TABhow are you?" }
* TO: { "value": "hello i am ben\n\n\thow are you?" }
*/
const promptValues = handleEscapeCharacters(promptValuesRaw, true)
if (promptValues && inputVariables.length > 0) {
let seen: string[] = []
for (const variable of inputVariables) {
@@ -122,15 +128,13 @@ const runPrediction = async (
if (seen.length === 0) {
// All inputVariables have fixed values specified
const options = {
...promptValues
}
const options = { ...promptValues }
if (isStreaming) {
const handler = new CustomChainHandler(socketIO, socketIOClientId)
const res = await chain.call(options, [handler])
const res = await chain.call(options, [loggerHandler, handler])
return res?.text
} else {
const res = await chain.call(options)
const res = await chain.call(options, [loggerHandler])
return res?.text
}
} else if (seen.length === 1) {
@@ -143,10 +147,10 @@ const runPrediction = async (
}
if (isStreaming) {
const handler = new CustomChainHandler(socketIO, socketIOClientId)
const res = await chain.call(options, [handler])
const res = await chain.call(options, [loggerHandler, handler])
return res?.text
} else {
const res = await chain.call(options)
const res = await chain.call(options, [loggerHandler])
return res?.text
}
} else {
@@ -155,10 +159,10 @@ const runPrediction = async (
} else {
if (isStreaming) {
const handler = new CustomChainHandler(socketIO, socketIOClientId)
const res = await chain.run(input, [handler])
const res = await chain.run(input, [loggerHandler, handler])
return res
} else {
const res = await chain.run(input)
const res = await chain.run(input, [loggerHandler])
return res
}
}
packages/components/nodes/chains/MultiPromptChain/MultiPromptChain.ts
+8 -3
View File
@@ -1,11 +1,13 @@
import { BaseLanguageModel } from 'langchain/base_language'
import { ICommonObject, INode, INodeData, INodeParams, PromptRetriever } from '../../../src/Interface'
import { CustomChainHandler, getBaseClasses } from '../../../src/utils'
import { getBaseClasses } from '../../../src/utils'
import { MultiPromptChain } from 'langchain/chains'
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
class MultiPromptChain_Chains implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
@@ -16,6 +18,7 @@ class MultiPromptChain_Chains implements INode {
constructor() {
this.label = 'Multi Prompt Chain'
this.name = 'multiPromptChain'
this.version = 1.0
this.type = 'MultiPromptChain'
this.icon = 'chain.svg'
this.category = 'Chains'
@@ -63,12 +66,14 @@ class MultiPromptChain_Chains implements INode {
const chain = nodeData.instance as MultiPromptChain
const obj = { input }
const loggerHandler = new ConsoleCallbackHandler(options.logger)
if (options.socketIO && options.socketIOClientId) {
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId, 2)
const res = await chain.call(obj, [handler])
const res = await chain.call(obj, [loggerHandler, handler])
return res?.text
} else {
const res = await chain.call(obj)
const res = await chain.call(obj, [loggerHandler])
return res?.text
}
}
packages/components/nodes/chains/MultiRetrievalQAChain/MultiRetrievalQAChain.ts
+7 -3
View File
@@ -1,11 +1,13 @@
import { BaseLanguageModel } from 'langchain/base_language'
import { ICommonObject, INode, INodeData, INodeParams, VectorStoreRetriever } from '../../../src/Interface'
import { CustomChainHandler, getBaseClasses } from '../../../src/utils'
import { getBaseClasses } from '../../../src/utils'
import { MultiRetrievalQAChain } from 'langchain/chains'
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
class MultiRetrievalQAChain_Chains implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
@@ -16,6 +18,7 @@ class MultiRetrievalQAChain_Chains implements INode {
constructor() {
this.label = 'Multi Retrieval QA Chain'
this.name = 'multiRetrievalQAChain'
this.version = 1.0
this.type = 'MultiRetrievalQAChain'
this.icon = 'chain.svg'
this.category = 'Chains'
@@ -71,14 +74,15 @@ class MultiRetrievalQAChain_Chains implements INode {
const returnSourceDocuments = nodeData.inputs?.returnSourceDocuments as boolean
const obj = { input }
const loggerHandler = new ConsoleCallbackHandler(options.logger)
if (options.socketIO && options.socketIOClientId) {
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId, 2, returnSourceDocuments)
const res = await chain.call(obj, [handler])
const res = await chain.call(obj, [loggerHandler, handler])
if (res.text && res.sourceDocuments) return res
return res?.text
} else {
const res = await chain.call(obj)
const res = await chain.call(obj, [loggerHandler])
if (res.text && res.sourceDocuments) return res
return res?.text
}
packages/components/nodes/chains/RetrievalQAChain/RetrievalQAChain.ts
+8 -4
View File
@@ -1,12 +1,14 @@
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { RetrievalQAChain } from 'langchain/chains'
import { BaseRetriever } from 'langchain/schema'
import { CustomChainHandler, getBaseClasses } from '../../../src/utils'
import { BaseRetriever } from 'langchain/schema/retriever'
import { getBaseClasses } from '../../../src/utils'
import { BaseLanguageModel } from 'langchain/base_language'
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
class RetrievalQAChain_Chains implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
@@ -17,6 +19,7 @@ class RetrievalQAChain_Chains implements INode {
constructor() {
this.label = 'Retrieval QA Chain'
this.name = 'retrievalQAChain'
this.version = 1.0
this.type = 'RetrievalQAChain'
this.icon = 'chain.svg'
this.category = 'Chains'
@@ -49,13 +52,14 @@ class RetrievalQAChain_Chains implements INode {
const obj = {
query: input
}
const loggerHandler = new ConsoleCallbackHandler(options.logger)
if (options.socketIO && options.socketIOClientId) {
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId)
const res = await chain.call(obj, [handler])
const res = await chain.call(obj, [loggerHandler, handler])
return res?.text
} else {
const res = await chain.call(obj)
const res = await chain.call(obj, [loggerHandler])
return res?.text
}
}
packages/components/nodes/chains/SqlDatabaseChain/SqlDatabaseChain.ts
+10 -5
View File
@@ -1,13 +1,15 @@
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { SqlDatabaseChain, SqlDatabaseChainInput } from 'langchain/chains'
import { CustomChainHandler, getBaseClasses } from '../../../src/utils'
import { SqlDatabaseChain, SqlDatabaseChainInput } from 'langchain/chains/sql_db'
import { getBaseClasses } from '../../../src/utils'
import { DataSource } from 'typeorm'
import { SqlDatabase } from 'langchain/sql_db'
import { BaseLanguageModel } from 'langchain/base_language'
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
class SqlDatabaseChain_Chains implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
@@ -18,6 +20,7 @@ class SqlDatabaseChain_Chains implements INode {
constructor() {
this.label = 'Sql Database Chain'
this.name = 'sqlDatabaseChain'
this.version = 1.0
this.type = 'SqlDatabaseChain'
this.icon = 'sqlchain.svg'
this.category = 'Chains'
@@ -65,12 +68,14 @@ class SqlDatabaseChain_Chains implements INode {
const dbFilePath = nodeData.inputs?.dbFilePath
const chain = await getSQLDBChain(databaseType, dbFilePath, model)
const loggerHandler = new ConsoleCallbackHandler(options.logger)
if (options.socketIO && options.socketIOClientId) {
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId)
const res = await chain.run(input, [handler])
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId, 2)
const res = await chain.run(input, [loggerHandler, handler])
return res
} else {
const res = await chain.run(input)
const res = await chain.run(input, [loggerHandler])
return res
}
}
packages/components/nodes/chains/VectorDBQAChain/VectorDBQAChain.ts
+8 -3
View File
@@ -1,12 +1,14 @@
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { CustomChainHandler, getBaseClasses } from '../../../src/utils'
import { getBaseClasses } from '../../../src/utils'
import { VectorDBQAChain } from 'langchain/chains'
import { BaseLanguageModel } from 'langchain/base_language'
import { VectorStore } from 'langchain/vectorstores'
import { ConsoleCallbackHandler, CustomChainHandler } from '../../../src/handler'
class VectorDBQAChain_Chains implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
@@ -17,6 +19,7 @@ class VectorDBQAChain_Chains implements INode {
constructor() {
this.label = 'VectorDB QA Chain'
this.name = 'vectorDBQAChain'
this.version = 1.0
this.type = 'VectorDBQAChain'
this.icon = 'chain.svg'
this.category = 'Chains'
@@ -53,12 +56,14 @@ class VectorDBQAChain_Chains implements INode {
query: input
}
const loggerHandler = new ConsoleCallbackHandler(options.logger)
if (options.socketIO && options.socketIOClientId) {
const handler = new CustomChainHandler(options.socketIO, options.socketIOClientId)
const res = await chain.call(obj, [handler])
const res = await chain.call(obj, [loggerHandler, handler])
return res?.text
} else {
const res = await chain.call(obj)
const res = await chain.call(obj, [loggerHandler])
return res?.text
}
}
packages/components/nodes/chatmodels/AzureChatOpenAI/Azure.svg
+1 -5
View File
@@ -1,5 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-brand-azure" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
<path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
<path d="M6 7.5l-4 9.5h4l6 -15z"></path>
<path d="M22 20l-7 -15l-3 7l4 5l-8 3z"></path>
</svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 48" width="96px" height="96px"><path fill="#035bda" d="M46 40L29.317 10.852 22.808 23.96 34.267 37.24 13 39.655zM13.092 18.182L2 36.896 11.442 35.947 28.033 5.678z"/></svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 392 B

After

Width:  |  Height:  |  Size: 229 B

packages/components/nodes/chatmodels/AzureChatOpenAI/AzureChatOpenAI.ts
+30 -39
View File
@@ -1,32 +1,36 @@
import { OpenAIBaseInput } from 'langchain/dist/types/openai-types'
import { INode, INodeData, INodeParams } from '../../../src/Interface'
import { getBaseClasses } from '../../../src/utils'
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { getBaseClasses, getCredentialData, getCredentialParam } from '../../../src/utils'
import { AzureOpenAIInput, ChatOpenAI } from 'langchain/chat_models/openai'
class AzureChatOpenAI_ChatModels implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
description: string
baseClasses: string[]
credential: INodeParams
inputs: INodeParams[]
constructor() {
this.label = 'Azure ChatOpenAI'
this.name = 'azureChatOpenAI'
this.version = 1.0
this.type = 'AzureChatOpenAI'
this.icon = 'Azure.svg'
this.category = 'Chat Models'
this.description = 'Wrapper around Azure OpenAI large language models that use the Chat endpoint'
this.baseClasses = [this.type, ...getBaseClasses(ChatOpenAI)]
this.credential = {
label: 'Connect Credential',
name: 'credential',
type: 'credential',
credentialNames: ['azureOpenAIApi']
}
this.inputs = [
{
label: 'Azure OpenAI Api Key',
name: 'azureOpenAIApiKey',
type: 'password'
},
{
label: 'Model Name',
name: 'modelName',
@@ -43,6 +47,10 @@ class AzureChatOpenAI_ChatModels implements INode {
{
label: 'gpt-35-turbo',
name: 'gpt-35-turbo'
},
{
label: 'gpt-35-turbo-16k',
name: 'gpt-35-turbo-16k'
}
],
default: 'gpt-35-turbo',
@@ -52,37 +60,15 @@ class AzureChatOpenAI_ChatModels implements INode {
label: 'Temperature',
name: 'temperature',
type: 'number',
step: 0.1,
default: 0.9,
optional: true
},
{
label: 'Azure OpenAI Api Instance Name',
name: 'azureOpenAIApiInstanceName',
type: 'string',
placeholder: 'YOUR-INSTANCE-NAME'
},
{
label: 'Azure OpenAI Api Deployment Name',
name: 'azureOpenAIApiDeploymentName',
type: 'string',
placeholder: 'YOUR-DEPLOYMENT-NAME'
},
{
label: 'Azure OpenAI Api Version',
name: 'azureOpenAIApiVersion',
type: 'options',
options: [
{
label: '2023-03-15-preview',
name: '2023-03-15-preview'
}
],
default: '2023-03-15-preview'
},
{
label: 'Max Tokens',
name: 'maxTokens',
type: 'number',
step: 1,
optional: true,
additionalParams: true
},
@@ -90,6 +76,7 @@ class AzureChatOpenAI_ChatModels implements INode {
label: 'Frequency Penalty',
name: 'frequencyPenalty',
type: 'number',
step: 0.1,
optional: true,
additionalParams: true
},
@@ -97,6 +84,7 @@ class AzureChatOpenAI_ChatModels implements INode {
label: 'Presence Penalty',
name: 'presencePenalty',
type: 'number',
step: 0.1,
optional: true,
additionalParams: true
},
@@ -104,27 +92,30 @@ class AzureChatOpenAI_ChatModels implements INode {
label: 'Timeout',
name: 'timeout',
type: 'number',
step: 1,
optional: true,
additionalParams: true
}
]
}
async init(nodeData: INodeData): Promise<any> {
const azureOpenAIApiKey = nodeData.inputs?.azureOpenAIApiKey as string
async init(nodeData: INodeData, _: string, options: ICommonObject): Promise<any> {
const modelName = nodeData.inputs?.modelName as string
const temperature = nodeData.inputs?.temperature as string
const azureOpenAIApiInstanceName = nodeData.inputs?.azureOpenAIApiInstanceName as string
const azureOpenAIApiDeploymentName = nodeData.inputs?.azureOpenAIApiDeploymentName as string
const azureOpenAIApiVersion = nodeData.inputs?.azureOpenAIApiVersion as string
const maxTokens = nodeData.inputs?.maxTokens as string
const frequencyPenalty = nodeData.inputs?.frequencyPenalty as string
const presencePenalty = nodeData.inputs?.presencePenalty as string
const timeout = nodeData.inputs?.timeout as string
const streaming = nodeData.inputs?.streaming as boolean
const credentialData = await getCredentialData(nodeData.credential ?? '', options)
const azureOpenAIApiKey = getCredentialParam('azureOpenAIApiKey', credentialData, nodeData)
const azureOpenAIApiInstanceName = getCredentialParam('azureOpenAIApiInstanceName', credentialData, nodeData)
const azureOpenAIApiDeploymentName = getCredentialParam('azureOpenAIApiDeploymentName', credentialData, nodeData)
const azureOpenAIApiVersion = getCredentialParam('azureOpenAIApiVersion', credentialData, nodeData)
const obj: Partial<AzureOpenAIInput> & Partial<OpenAIBaseInput> = {
temperature: parseInt(temperature, 10),
temperature: parseFloat(temperature),
modelName,
azureOpenAIApiKey,
azureOpenAIApiInstanceName,
@@ -134,8 +125,8 @@ class AzureChatOpenAI_ChatModels implements INode {
}
if (maxTokens) obj.maxTokens = parseInt(maxTokens, 10)
if (frequencyPenalty) obj.frequencyPenalty = parseInt(frequencyPenalty, 10)
if (presencePenalty) obj.presencePenalty = parseInt(presencePenalty, 10)
if (frequencyPenalty) obj.frequencyPenalty = parseFloat(frequencyPenalty)
if (presencePenalty) obj.presencePenalty = parseFloat(presencePenalty)
if (timeout) obj.timeout = parseInt(timeout, 10)
const model = new ChatOpenAI(obj)
packages/components/nodes/chatmodels/ChatAnthropic/ChatAnthropic.ts
+33 -13
View File
@@ -1,36 +1,50 @@
import { INode, INodeData, INodeParams } from '../../../src/Interface'
import { getBaseClasses } from '../../../src/utils'
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { getBaseClasses, getCredentialData, getCredentialParam } from '../../../src/utils'
import { AnthropicInput, ChatAnthropic } from 'langchain/chat_models/anthropic'
class ChatAnthropic_ChatModels implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
description: string
baseClasses: string[]
credential: INodeParams
inputs: INodeParams[]
constructor() {
this.label = 'ChatAnthropic'
this.name = 'chatAnthropic'
this.version = 1.0
this.type = 'ChatAnthropic'
this.icon = 'chatAnthropic.png'
this.category = 'Chat Models'
this.description = 'Wrapper around ChatAnthropic large language models that use the Chat endpoint'
this.baseClasses = [this.type, ...getBaseClasses(ChatAnthropic)]
this.credential = {
label: 'Connect Credential',
name: 'credential',
type: 'credential',
credentialNames: ['anthropicApi']
}
this.inputs = [
{
label: 'ChatAnthropic Api Key',
name: 'anthropicApiKey',
type: 'password'
},
{
label: 'Model Name',
name: 'modelName',
type: 'options',
options: [
{
label: 'claude-2',
name: 'claude-2',
description: 'Claude 2 latest major version, automatically get updates to the model as they are released'
},
{
label: 'claude-instant-1',
name: 'claude-instant-1',
description: 'Claude Instant latest major version, automatically get updates to the model as they are released'
},
{
label: 'claude-v1',
name: 'claude-v1'
@@ -76,13 +90,14 @@ class ChatAnthropic_ChatModels implements INode {
name: 'claude-instant-v1.1-100k'
}
],
default: 'claude-v1',
default: 'claude-2',
optional: true
},
{
label: 'Temperature',
name: 'temperature',
type: 'number',
step: 0.1,
default: 0.9,
optional: true
},
@@ -90,6 +105,7 @@ class ChatAnthropic_ChatModels implements INode {
label: 'Max Tokens',
name: 'maxTokensToSample',
type: 'number',
step: 1,
optional: true,
additionalParams: true
},
@@ -97,6 +113,7 @@ class ChatAnthropic_ChatModels implements INode {
label: 'Top P',
name: 'topP',
type: 'number',
step: 0.1,
optional: true,
additionalParams: true
},
@@ -104,31 +121,34 @@ class ChatAnthropic_ChatModels implements INode {
label: 'Top K',
name: 'topK',
type: 'number',
step: 0.1,
optional: true,
additionalParams: true
}
]
}
async init(nodeData: INodeData): Promise<any> {
async init(nodeData: INodeData, _: string, options: ICommonObject): Promise<any> {
const temperature = nodeData.inputs?.temperature as string
const modelName = nodeData.inputs?.modelName as string
const anthropicApiKey = nodeData.inputs?.anthropicApiKey as string
const maxTokensToSample = nodeData.inputs?.maxTokensToSample as string
const topP = nodeData.inputs?.topP as string
const topK = nodeData.inputs?.topK as string
const streaming = nodeData.inputs?.streaming as boolean
const credentialData = await getCredentialData(nodeData.credential ?? '', options)
const anthropicApiKey = getCredentialParam('anthropicApiKey', credentialData, nodeData)
const obj: Partial<AnthropicInput> & { anthropicApiKey?: string } = {
temperature: parseInt(temperature, 10),
temperature: parseFloat(temperature),
modelName,
anthropicApiKey,
streaming: streaming ?? true
}
if (maxTokensToSample) obj.maxTokensToSample = parseInt(maxTokensToSample, 10)
if (topP) obj.topP = parseInt(topP, 10)
if (topK) obj.topK = parseInt(topK, 10)
if (topP) obj.topP = parseFloat(topP)
if (topK) obj.topK = parseFloat(topK)
const model = new ChatAnthropic(obj)
return model
packages/components/nodes/chatmodels/ChatHuggingFace/ChatHuggingFace.ts
+37 -14
View File
@@ -1,41 +1,56 @@
import { INode, INodeData, INodeParams } from '../../../src/Interface'
import { getBaseClasses } from '../../../src/utils'
import { HFInput, HuggingFaceInference } from 'langchain/llms/hf'
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { getBaseClasses, getCredentialData, getCredentialParam } from '../../../src/utils'
import { HFInput, HuggingFaceInference } from './core'
class ChatHuggingFace_ChatModels implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
description: string
baseClasses: string[]
credential: INodeParams
inputs: INodeParams[]
constructor() {
this.label = 'ChatHuggingFace'
this.name = 'chatHuggingFace'
this.version = 1.0
this.type = 'ChatHuggingFace'
this.icon = 'huggingface.png'
this.category = 'Chat Models'
this.description = 'Wrapper around HuggingFace large language models'
this.baseClasses = [this.type, 'BaseChatModel', ...getBaseClasses(HuggingFaceInference)]
this.credential = {
label: 'Connect Credential',
name: 'credential',
type: 'credential',
credentialNames: ['huggingFaceApi']
}
this.inputs = [
{
label: 'Model',
name: 'model',
type: 'string',
placeholder: 'gpt2'
description: 'If using own inference endpoint, leave this blank',
placeholder: 'gpt2',
optional: true
},
{
label: 'HuggingFace Api Key',
name: 'apiKey',
type: 'password'
label: 'Endpoint',
name: 'endpoint',
type: 'string',
placeholder: 'https://xyz.eu-west-1.aws.endpoints.huggingface.cloud/gpt2',
description: 'Using your own inference endpoint',
optional: true
},
{
label: 'Temperature',
name: 'temperature',
type: 'number',
step: 0.1,
description: 'Temperature parameter may not apply to certain model. Please check available model parameters',
optional: true,
additionalParams: true
@@ -44,6 +59,7 @@ class ChatHuggingFace_ChatModels implements INode {
label: 'Max Tokens',
name: 'maxTokens',
type: 'number',
step: 1,
description: 'Max Tokens parameter may not apply to certain model. Please check available model parameters',
optional: true,
additionalParams: true
@@ -52,6 +68,7 @@ class ChatHuggingFace_ChatModels implements INode {
label: 'Top Probability',
name: 'topP',
type: 'number',
step: 0.1,
description: 'Top Probability parameter may not apply to certain model. Please check available model parameters',
optional: true,
additionalParams: true
@@ -60,6 +77,7 @@ class ChatHuggingFace_ChatModels implements INode {
label: 'Top K',
name: 'hfTopK',
type: 'number',
step: 0.1,
description: 'Top K parameter may not apply to certain model. Please check available model parameters',
optional: true,
additionalParams: true
@@ -68,6 +86,7 @@ class ChatHuggingFace_ChatModels implements INode {
label: 'Frequency Penalty',
name: 'frequencyPenalty',
type: 'number',
step: 0.1,
description: 'Frequency Penalty parameter may not apply to certain model. Please check available model parameters',
optional: true,
additionalParams: true
@@ -75,25 +94,29 @@ class ChatHuggingFace_ChatModels implements INode {
]
}
async init(nodeData: INodeData): Promise<any> {
async init(nodeData: INodeData, _: string, options: ICommonObject): Promise<any> {
const model = nodeData.inputs?.model as string
const apiKey = nodeData.inputs?.apiKey as string
const temperature = nodeData.inputs?.temperature as string
const maxTokens = nodeData.inputs?.maxTokens as string
const topP = nodeData.inputs?.topP as string
const hfTopK = nodeData.inputs?.hfTopK as string
const frequencyPenalty = nodeData.inputs?.frequencyPenalty as string
const endpoint = nodeData.inputs?.endpoint as string
const credentialData = await getCredentialData(nodeData.credential ?? '', options)
const huggingFaceApiKey = getCredentialParam('huggingFaceApiKey', credentialData, nodeData)
const obj: Partial<HFInput> = {
model,
apiKey
apiKey: huggingFaceApiKey
}
if (temperature) obj.temperature = parseInt(temperature, 10)
if (temperature) obj.temperature = parseFloat(temperature)
if (maxTokens) obj.maxTokens = parseInt(maxTokens, 10)
if (topP) obj.topP = parseInt(topP, 10)
if (hfTopK) obj.topK = parseInt(hfTopK, 10)
if (frequencyPenalty) obj.frequencyPenalty = parseInt(frequencyPenalty, 10)
if (topP) obj.topP = parseFloat(topP)
if (hfTopK) obj.topK = parseFloat(hfTopK)
if (frequencyPenalty) obj.frequencyPenalty = parseFloat(frequencyPenalty)
if (endpoint) obj.endpoint = endpoint
const huggingFace = new HuggingFaceInference(obj)
return huggingFace
packages/components/nodes/chatmodels/ChatHuggingFace/core.ts
+113
View File
@@ -0,0 +1,113 @@
import { getEnvironmentVariable } from '../../../src/utils'
import { LLM, BaseLLMParams } from 'langchain/llms/base'
export interface HFInput {
/** Model to use */
model: string
/** Sampling temperature to use */
temperature?: number
/**
* Maximum number of tokens to generate in the completion.
*/
maxTokens?: number
/** Total probability mass of tokens to consider at each step */
topP?: number
/** Integer to define the top tokens considered within the sample operation to create new text. */
topK?: number
/** Penalizes repeated tokens according to frequency */
frequencyPenalty?: number
/** API key to use. */
apiKey?: string
/** Private endpoint to use. */
endpoint?: string
}
export class HuggingFaceInference extends LLM implements HFInput {
get lc_secrets(): { [key: string]: string } | undefined {
return {
apiKey: 'HUGGINGFACEHUB_API_KEY'
}
}
model = 'gpt2'
temperature: number | undefined = undefined
maxTokens: number | undefined = undefined
topP: number | undefined = undefined
topK: number | undefined = undefined
frequencyPenalty: number | undefined = undefined
apiKey: string | undefined = undefined
endpoint: string | undefined = undefined
constructor(fields?: Partial<HFInput> & BaseLLMParams) {
super(fields ?? {})
this.model = fields?.model ?? this.model
this.temperature = fields?.temperature ?? this.temperature
this.maxTokens = fields?.maxTokens ?? this.maxTokens
this.topP = fields?.topP ?? this.topP
this.topK = fields?.topK ?? this.topK
this.frequencyPenalty = fields?.frequencyPenalty ?? this.frequencyPenalty
this.endpoint = fields?.endpoint ?? ''
this.apiKey = fields?.apiKey ?? getEnvironmentVariable('HUGGINGFACEHUB_API_KEY')
if (!this.apiKey) {
throw new Error(
'Please set an API key for HuggingFace Hub in the environment variable HUGGINGFACEHUB_API_KEY or in the apiKey field of the HuggingFaceInference constructor.'
)
}
}
_llmType() {
return 'hf'
}
/** @ignore */
async _call(prompt: string, options: this['ParsedCallOptions']): Promise<string> {
const { HfInference } = await HuggingFaceInference.imports()
const hf = new HfInference(this.apiKey)
const obj: any = {
parameters: {
// make it behave similar to openai, returning only the generated text
return_full_text: false,
temperature: this.temperature,
max_new_tokens: this.maxTokens,
top_p: this.topP,
top_k: this.topK,
repetition_penalty: this.frequencyPenalty
},
inputs: prompt
}
if (this.endpoint) {
hf.endpoint(this.endpoint)
} else {
obj.model = this.model
}
const res = await this.caller.callWithOptions({ signal: options.signal }, hf.textGeneration.bind(hf), obj)
return res.generated_text
}
/** @ignore */
static async imports(): Promise<{
HfInference: typeof import('@huggingface/inference').HfInference
}> {
try {
const { HfInference } = await import('@huggingface/inference')
return { HfInference }
} catch (e) {
throw new Error('Please install huggingface as a dependency with, e.g. `yarn add @huggingface/inference`')
}
}
}
packages/components/nodes/chatmodels/ChatLocalAI/ChatLocalAI.ts
+8 -2
View File
@@ -6,6 +6,7 @@ import { OpenAIChatInput } from 'langchain/chat_models/openai'
class ChatLocalAI_ChatModels implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
@@ -16,6 +17,7 @@ class ChatLocalAI_ChatModels implements INode {
constructor() {
this.label = 'ChatLocalAI'
this.name = 'chatLocalAI'
this.version = 1.0
this.type = 'ChatLocalAI'
this.icon = 'localai.png'
this.category = 'Chat Models'
@@ -38,6 +40,7 @@ class ChatLocalAI_ChatModels implements INode {
label: 'Temperature',
name: 'temperature',
type: 'number',
step: 0.1,
default: 0.9,
optional: true
},
@@ -45,6 +48,7 @@ class ChatLocalAI_ChatModels implements INode {
label: 'Max Tokens',
name: 'maxTokens',
type: 'number',
step: 1,
optional: true,
additionalParams: true
},
@@ -52,6 +56,7 @@ class ChatLocalAI_ChatModels implements INode {
label: 'Top Probability',
name: 'topP',
type: 'number',
step: 0.1,
optional: true,
additionalParams: true
},
@@ -59,6 +64,7 @@ class ChatLocalAI_ChatModels implements INode {
label: 'Timeout',
name: 'timeout',
type: 'number',
step: 1,
optional: true,
additionalParams: true
}
@@ -74,13 +80,13 @@ class ChatLocalAI_ChatModels implements INode {
const basePath = nodeData.inputs?.basePath as string
const obj: Partial<OpenAIChatInput> & { openAIApiKey?: string } = {
temperature: parseInt(temperature, 10),
temperature: parseFloat(temperature),
modelName,
openAIApiKey: 'sk-'
}
if (maxTokens) obj.maxTokens = parseInt(maxTokens, 10)
if (topP) obj.topP = parseInt(topP, 10)
if (topP) obj.topP = parseFloat(topP)
if (timeout) obj.timeout = parseInt(timeout, 10)
const model = new OpenAIChat(obj, { basePath })
packages/components/nodes/chatmodels/ChatOpenAI/ChatOpenAI.ts
+25 -13
View File
@@ -1,31 +1,35 @@
import { INode, INodeData, INodeParams } from '../../../src/Interface'
import { getBaseClasses } from '../../../src/utils'
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { getBaseClasses, getCredentialData, getCredentialParam } from '../../../src/utils'
import { ChatOpenAI, OpenAIChatInput } from 'langchain/chat_models/openai'
class ChatOpenAI_ChatModels implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
description: string
baseClasses: string[]
credential: INodeParams
inputs: INodeParams[]
constructor() {
this.label = 'ChatOpenAI'
this.name = 'chatOpenAI'
this.version = 1.0
this.type = 'ChatOpenAI'
this.icon = 'openai.png'
this.category = 'Chat Models'
this.description = 'Wrapper around OpenAI large language models that use the Chat endpoint'
this.baseClasses = [this.type, ...getBaseClasses(ChatOpenAI)]
this.credential = {
label: 'Connect Credential',
name: 'credential',
type: 'credential',
credentialNames: ['openAIApi']
}
this.inputs = [
{
label: 'OpenAI Api Key',
name: 'openAIApiKey',
type: 'password'
},
{
label: 'Model Name',
name: 'modelName',
@@ -71,6 +75,7 @@ class ChatOpenAI_ChatModels implements INode {
label: 'Temperature',
name: 'temperature',
type: 'number',
step: 0.1,
default: 0.9,
optional: true
},
@@ -78,6 +83,7 @@ class ChatOpenAI_ChatModels implements INode {
label: 'Max Tokens',
name: 'maxTokens',
type: 'number',
step: 1,
optional: true,
additionalParams: true
},
@@ -85,6 +91,7 @@ class ChatOpenAI_ChatModels implements INode {
label: 'Top Probability',
name: 'topP',
type: 'number',
step: 0.1,
optional: true,
additionalParams: true
},
@@ -92,6 +99,7 @@ class ChatOpenAI_ChatModels implements INode {
label: 'Frequency Penalty',
name: 'frequencyPenalty',
type: 'number',
step: 0.1,
optional: true,
additionalParams: true
},
@@ -99,6 +107,7 @@ class ChatOpenAI_ChatModels implements INode {
label: 'Presence Penalty',
name: 'presencePenalty',
type: 'number',
step: 0.1,
optional: true,
additionalParams: true
},
@@ -106,6 +115,7 @@ class ChatOpenAI_ChatModels implements INode {
label: 'Timeout',
name: 'timeout',
type: 'number',
step: 1,
optional: true,
additionalParams: true
},
@@ -119,10 +129,9 @@ class ChatOpenAI_ChatModels implements INode {
]
}
async init(nodeData: INodeData): Promise<any> {
async init(nodeData: INodeData, _: string, options: ICommonObject): Promise<any> {
const temperature = nodeData.inputs?.temperature as string
const modelName = nodeData.inputs?.modelName as string
const openAIApiKey = nodeData.inputs?.openAIApiKey as string
const maxTokens = nodeData.inputs?.maxTokens as string
const topP = nodeData.inputs?.topP as string
const frequencyPenalty = nodeData.inputs?.frequencyPenalty as string
@@ -131,17 +140,20 @@ class ChatOpenAI_ChatModels implements INode {
const streaming = nodeData.inputs?.streaming as boolean
const basePath = nodeData.inputs?.basepath as string
const credentialData = await getCredentialData(nodeData.credential ?? '', options)
const openAIApiKey = getCredentialParam('openAIApiKey', credentialData, nodeData)
const obj: Partial<OpenAIChatInput> & { openAIApiKey?: string } = {
temperature: parseInt(temperature, 10),
temperature: parseFloat(temperature),
modelName,
openAIApiKey,
streaming: streaming ?? true
}
if (maxTokens) obj.maxTokens = parseInt(maxTokens, 10)
if (topP) obj.topP = parseInt(topP, 10)
if (frequencyPenalty) obj.frequencyPenalty = parseInt(frequencyPenalty, 10)
if (presencePenalty) obj.presencePenalty = parseInt(presencePenalty, 10)
if (topP) obj.topP = parseFloat(topP)
if (frequencyPenalty) obj.frequencyPenalty = parseFloat(frequencyPenalty)
if (presencePenalty) obj.presencePenalty = parseFloat(presencePenalty)
if (timeout) obj.timeout = parseInt(timeout, 10)
const model = new ChatOpenAI(obj, {
packages/components/nodes/documentloaders/API/APILoader.ts
+200
View File
@@ -0,0 +1,200 @@
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { TextSplitter } from 'langchain/text_splitter'
import { BaseDocumentLoader } from 'langchain/document_loaders/base'
import { Document } from 'langchain/document'
import axios, { AxiosRequestConfig } from 'axios'
class API_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
category: string
baseClasses: string[]
inputs?: INodeParams[]
constructor() {
this.label = 'API Loader'
this.name = 'apiLoader'
this.version = 1.0
this.type = 'Document'
this.icon = 'api-loader.png'
this.category = 'Document Loaders'
this.description = `Load data from an API`
this.baseClasses = [this.type]
this.inputs = [
{
label: 'Text Splitter',
name: 'textSplitter',
type: 'TextSplitter',
optional: true
},
{
label: 'Method',
name: 'method',
type: 'options',
options: [
{
label: 'GET',
name: 'GET'
},
{
label: 'POST',
name: 'POST'
}
]
},
{
label: 'URL',
name: 'url',
type: 'string'
},
{
label: 'Headers',
name: 'headers',
type: 'json',
additionalParams: true,
optional: true
},
{
label: 'Body',
name: 'body',
type: 'json',
description:
'JSON body for the POST request. If not specified, agent will try to figure out itself from AIPlugin if provided',
additionalParams: true,
optional: true
}
]
}
async init(nodeData: INodeData): Promise<any> {
const headers = nodeData.inputs?.headers as string
const url = nodeData.inputs?.url as string
const body = nodeData.inputs?.body as string
const method = nodeData.inputs?.method as string
const textSplitter = nodeData.inputs?.textSplitter as TextSplitter
const metadata = nodeData.inputs?.metadata
const options: ApiLoaderParams = {
url,
method
}
if (headers) {
const parsedHeaders = typeof headers === 'object' ? headers : JSON.parse(headers)
options.headers = parsedHeaders
}
if (body) {
const parsedBody = typeof body === 'object' ? body : JSON.parse(body)
options.body = parsedBody
}
const loader = new ApiLoader(options)
let docs = []
if (textSplitter) {
docs = await loader.loadAndSplit(textSplitter)
} else {
docs = await loader.load()
}
if (metadata) {
const parsedMetadata = typeof metadata === 'object' ? metadata : JSON.parse(metadata)
let finaldocs = []
for (const doc of docs) {
const newdoc = {
...doc,
metadata: {
...doc.metadata,
...parsedMetadata
}
}
finaldocs.push(newdoc)
}
return finaldocs
}
return docs
}
}
interface ApiLoaderParams {
url: string
method: string
headers?: ICommonObject
body?: ICommonObject
}
class ApiLoader extends BaseDocumentLoader {
public readonly url: string
public readonly headers?: ICommonObject
public readonly body?: ICommonObject
public readonly method: string
constructor({ url, headers, body, method }: ApiLoaderParams) {
super()
this.url = url
this.headers = headers
this.body = body
this.method = method
}
public async load(): Promise<Document[]> {
if (this.method === 'POST') {
return this.executePostRequest(this.url, this.headers, this.body)
} else {
return this.executeGetRequest(this.url, this.headers)
}
}
protected async executeGetRequest(url: string, headers?: ICommonObject): Promise<Document[]> {
try {
const config: AxiosRequestConfig = {}
if (headers) {
config.headers = headers
}
const response = await axios.get(url, config)
const responseJsonString = JSON.stringify(response.data, null, 2)
const doc = new Document({
pageContent: responseJsonString,
metadata: {
url
}
})
return [doc]
} catch (error) {
throw new Error(`Failed to fetch ${url}: ${error}`)
}
}
protected async executePostRequest(url: string, headers?: ICommonObject, body?: ICommonObject): Promise<Document[]> {
try {
const config: AxiosRequestConfig = {}
if (headers) {
config.headers = headers
}
const response = await axios.post(url, body ?? {}, config)
const responseJsonString = JSON.stringify(response.data, null, 2)
const doc = new Document({
pageContent: responseJsonString,
metadata: {
url
}
})
return [doc]
} catch (error) {
throw new Error(`Failed to post ${url}: ${error}`)
}
}
}
module.exports = {
nodeClass: API_DocumentLoaders
}
packages/components/nodes/documentloaders/API/api-loader.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.4 KiB

packages/components/nodes/documentloaders/Airtable/Airtable.ts
+230
View File
@@ -0,0 +1,230 @@
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { TextSplitter } from 'langchain/text_splitter'
import { BaseDocumentLoader } from 'langchain/document_loaders/base'
import { Document } from 'langchain/document'
import axios from 'axios'
import { getCredentialData, getCredentialParam } from '../../../src/utils'
class Airtable_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
category: string
baseClasses: string[]
credential: INodeParams
inputs?: INodeParams[]
constructor() {
this.label = 'Airtable'
this.name = 'airtable'
this.version = 1.0
this.type = 'Document'
this.icon = 'airtable.svg'
this.category = 'Document Loaders'
this.description = `Load data from Airtable table`
this.baseClasses = [this.type]
this.credential = {
label: 'Connect Credential',
name: 'credential',
type: 'credential',
credentialNames: ['airtableApi']
}
this.inputs = [
{
label: 'Text Splitter',
name: 'textSplitter',
type: 'TextSplitter',
optional: true
},
{
label: 'Base Id',
name: 'baseId',
type: 'string',
placeholder: 'app11RobdGoX0YNsC',
description:
'If your table URL looks like: https://airtable.com/app11RobdGoX0YNsC/tblJdmvbrgizbYICO/viw9UrP77Id0CE4ee, app11RovdGoX0YNsC is the base id'
},
{
label: 'Table Id',
name: 'tableId',
type: 'string',
placeholder: 'tblJdmvbrgizbYICO',
description:
'If your table URL looks like: https://airtable.com/app11RobdGoX0YNsC/tblJdmvbrgizbYICO/viw9UrP77Id0CE4ee, tblJdmvbrgizbYICO is the table id'
},
{
label: 'Return All',
name: 'returnAll',
type: 'boolean',
default: true,
additionalParams: true,
description: 'If all results should be returned or only up to a given limit'
},
{
label: 'Limit',
name: 'limit',
type: 'number',
default: 100,
additionalParams: true,
description: 'Number of results to return'
},
{
label: 'Metadata',
name: 'metadata',
type: 'json',
optional: true,
additionalParams: true
}
]
}
async init(nodeData: INodeData, _: string, options: ICommonObject): Promise<any> {
const baseId = nodeData.inputs?.baseId as string
const tableId = nodeData.inputs?.tableId as string
const returnAll = nodeData.inputs?.returnAll as boolean
const limit = nodeData.inputs?.limit as string
const textSplitter = nodeData.inputs?.textSplitter as TextSplitter
const metadata = nodeData.inputs?.metadata
const credentialData = await getCredentialData(nodeData.credential ?? '', options)
const accessToken = getCredentialParam('accessToken', credentialData, nodeData)
const airtableOptions: AirtableLoaderParams = {
baseId,
tableId,
returnAll,
accessToken,
limit: limit ? parseInt(limit, 10) : 100
}
const loader = new AirtableLoader(airtableOptions)
let docs = []
if (textSplitter) {
docs = await loader.loadAndSplit(textSplitter)
} else {
docs = await loader.load()
}
if (metadata) {
const parsedMetadata = typeof metadata === 'object' ? metadata : JSON.parse(metadata)
let finaldocs = []
for (const doc of docs) {
const newdoc = {
...doc,
metadata: {
...doc.metadata,
...parsedMetadata
}
}
finaldocs.push(newdoc)
}
return finaldocs
}
return docs
}
}
interface AirtableLoaderParams {
baseId: string
tableId: string
accessToken: string
limit?: number
returnAll?: boolean
}
interface AirtableLoaderResponse {
records: AirtableLoaderPage[]
offset?: string
}
interface AirtableLoaderPage {
id: string
createdTime: string
fields: ICommonObject
}
class AirtableLoader extends BaseDocumentLoader {
public readonly baseId: string
public readonly tableId: string
public readonly accessToken: string
public readonly limit: number
public readonly returnAll: boolean
constructor({ baseId, tableId, accessToken, limit = 100, returnAll = false }: AirtableLoaderParams) {
super()
this.baseId = baseId
this.tableId = tableId
this.accessToken = accessToken
this.limit = limit
this.returnAll = returnAll
}
public async load(): Promise<Document[]> {
if (this.returnAll) {
return this.loadAll()
}
return this.loadLimit()
}
protected async fetchAirtableData(url: string, params: ICommonObject): Promise<AirtableLoaderResponse> {
try {
const headers = {
Authorization: `Bearer ${this.accessToken}`,
'Content-Type': 'application/json',
Accept: 'application/json'
}
const response = await axios.get(url, { params, headers })
return response.data
} catch (error) {
throw new Error(`Failed to fetch ${url} from Airtable: ${error}`)
}
}
private createDocumentFromPage(page: AirtableLoaderPage): Document {
// Generate the URL
const pageUrl = `https://api.airtable.com/v0/${this.baseId}/${this.tableId}/${page.id}`
// Return a langchain document
return new Document({
pageContent: JSON.stringify(page.fields, null, 2),
metadata: {
url: pageUrl
}
})
}
private async loadLimit(): Promise<Document[]> {
const params = { maxRecords: this.limit }
const data = await this.fetchAirtableData(`https://api.airtable.com/v0/${this.baseId}/${this.tableId}`, params)
if (data.records.length === 0) {
return []
}
return data.records.map((page) => this.createDocumentFromPage(page))
}
private async loadAll(): Promise<Document[]> {
const params: ICommonObject = { pageSize: 100 }
let data: AirtableLoaderResponse
let returnPages: AirtableLoaderPage[] = []
do {
data = await this.fetchAirtableData(`https://api.airtable.com/v0/${this.baseId}/${this.tableId}`, params)
returnPages.push.apply(returnPages, data.records)
params.offset = data.offset
} while (data.offset !== undefined)
return returnPages.map((page) => this.createDocumentFromPage(page))
}
}
module.exports = {
nodeClass: Airtable_DocumentLoaders
}
packages/components/nodes/documentloaders/Airtable/airtable.svg
+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg width="256px" height="215px" viewBox="0 0 256 215" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" preserveAspectRatio="xMidYMid">
<g>
<path d="M114.25873,2.70101695 L18.8604023,42.1756384 C13.5552723,44.3711638 13.6102328,51.9065311 18.9486282,54.0225085 L114.746142,92.0117514 C123.163769,95.3498757 132.537419,95.3498757 140.9536,92.0117514 L236.75256,54.0225085 C242.08951,51.9065311 242.145916,44.3711638 236.83934,42.1756384 L141.442459,2.70101695 C132.738459,-0.900338983 122.961284,-0.900338983 114.25873,2.70101695" fill="#FFBF00"></path>
<path d="M136.349071,112.756863 L136.349071,207.659101 C136.349071,212.173089 140.900664,215.263892 145.096461,213.600615 L251.844122,172.166219 C254.281184,171.200072 255.879376,168.845451 255.879376,166.224705 L255.879376,71.3224678 C255.879376,66.8084791 251.327783,63.7176768 247.131986,65.3809537 L140.384325,106.815349 C137.94871,107.781496 136.349071,110.136118 136.349071,112.756863" fill="#26B5F8"></path>
<path d="M111.422771,117.65355 L79.742409,132.949912 L76.5257763,134.504714 L9.65047684,166.548104 C5.4112904,168.593211 0.000578531073,165.503855 0.000578531073,160.794612 L0.000578531073,71.7210757 C0.000578531073,70.0173017 0.874160452,68.5463864 2.04568588,67.4384994 C2.53454463,66.9481944 3.08848814,66.5446689 3.66412655,66.2250305 C5.26231864,65.2661153 7.54173107,65.0101153 9.47981017,65.7766689 L110.890522,105.957098 C116.045234,108.002206 116.450206,115.225166 111.422771,117.65355" fill="#ED3049"></path>
<path d="M111.422771,117.65355 L79.742409,132.949912 L2.04568588,67.4384994 C2.53454463,66.9481944 3.08848814,66.5446689 3.66412655,66.2250305 C5.26231864,65.2661153 7.54173107,65.0101153 9.47981017,65.7766689 L110.890522,105.957098 C116.045234,108.002206 116.450206,115.225166 111.422771,117.65355" fill-opacity="0.25" fill="#000000"></path>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.9 KiB

packages/components/nodes/documentloaders/Cheerio/Cheerio.ts
+47 -21
View File
@@ -2,11 +2,12 @@ import { INode, INodeData, INodeParams } from '../../../src/Interface'
import { TextSplitter } from 'langchain/text_splitter'
import { CheerioWebBaseLoader } from 'langchain/document_loaders/web/cheerio'
import { test } from 'linkifyjs'
import { getAvailableURLs } from '../../../src'
import { webCrawl, xmlScrape } from '../../../src'
class Cheerio_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -17,6 +18,7 @@ class Cheerio_DocumentLoaders implements INode {
constructor() {
this.label = 'Cheerio Web Scraper'
this.name = 'cheerioWebScraper'
this.version = 1.0
this.type = 'Document'
this.icon = 'cheerio.svg'
this.category = 'Document Loaders'
@@ -35,19 +37,34 @@ class Cheerio_DocumentLoaders implements INode {
optional: true
},
{
label: 'Web Scrap for Relative Links',
name: 'webScrap',
type: 'boolean',
label: 'Get Relative Links Method',
name: 'relativeLinksMethod',
type: 'options',
description: 'Select a method to retrieve relative links',
options: [
{
label: 'Web Crawl',
name: 'webCrawl',
description: 'Crawl relative links from HTML URL'
},
{
label: 'Scrape XML Sitemap',
name: 'scrapeXMLSitemap',
description: 'Scrape relative links from XML sitemap URL'
}
],
optional: true,
additionalParams: true
},
{
label: 'Web Scrap Links Limit',
label: 'Get Relative Links Limit',
name: 'limit',
type: 'number',
default: 10,
optional: true,
additionalParams: true
additionalParams: true,
description:
'Only used when "Get Relative Links Method" is selected. Set 0 to retrieve all relative links, default limit is 10.',
warning: `Retreiving all links might take long time, and all links will be upserted again if the flow's state changed (eg: different URL, chunk size, etc)`
},
{
label: 'Metadata',
@@ -62,7 +79,7 @@ class Cheerio_DocumentLoaders implements INode {
async init(nodeData: INodeData): Promise<any> {
const textSplitter = nodeData.inputs?.textSplitter as TextSplitter
const metadata = nodeData.inputs?.metadata
const webScrap = nodeData.inputs?.webScrap as boolean
const relativeLinksMethod = nodeData.inputs?.relativeLinksMethod as string
let limit = nodeData.inputs?.limit as string
let url = nodeData.inputs?.url as string
@@ -71,25 +88,34 @@ class Cheerio_DocumentLoaders implements INode {
throw new Error('Invalid URL')
}
const cheerioLoader = async (url: string): Promise<any> => {
let docs = []
const loader = new CheerioWebBaseLoader(url)
if (textSplitter) {
docs = await loader.loadAndSplit(textSplitter)
} else {
docs = await loader.load()
async function cheerioLoader(url: string): Promise<any> {
try {
let docs = []
const loader = new CheerioWebBaseLoader(url)
if (textSplitter) {
docs = await loader.loadAndSplit(textSplitter)
} else {
docs = await loader.load()
}
return docs
} catch (err) {
if (process.env.DEBUG === 'true') console.error(`error in CheerioWebBaseLoader: ${err.message}, on page: ${url}`)
}
return docs
}
let availableUrls: string[]
let docs = []
if (webScrap) {
if (relativeLinksMethod) {
if (process.env.DEBUG === 'true') console.info(`Start ${relativeLinksMethod}`)
if (!limit) limit = '10'
availableUrls = await getAvailableURLs(url, parseInt(limit))
for (let i = 0; i < availableUrls.length; i++) {
docs.push(...(await cheerioLoader(availableUrls[i])))
else if (parseInt(limit) < 0) throw new Error('Limit cannot be less than 0')
const pages: string[] =
relativeLinksMethod === 'webCrawl' ? await webCrawl(url, parseInt(limit)) : await xmlScrape(url, parseInt(limit))
if (process.env.DEBUG === 'true') console.info(`pages: ${JSON.stringify(pages)}, length: ${pages.length}`)
if (!pages || pages.length === 0) throw new Error('No relative links found')
for (const page of pages) {
docs.push(...(await cheerioLoader(page)))
}
if (process.env.DEBUG === 'true') console.info(`Finish ${relativeLinksMethod}`)
} else {
docs = await cheerioLoader(url)
}
packages/components/nodes/documentloaders/Confluence/Confluence.ts
+21 -19
View File
@@ -1,25 +1,35 @@
import { INode, INodeData, INodeParams } from '../../../src/Interface'
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { TextSplitter } from 'langchain/text_splitter'
import { ConfluencePagesLoader, ConfluencePagesLoaderParams } from 'langchain/document_loaders/web/confluence'
import { getCredentialData, getCredentialParam } from '../../../src'
class Confluence_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
category: string
baseClasses: string[]
credential: INodeParams
inputs: INodeParams[]
constructor() {
this.label = 'Confluence'
this.name = 'confluence'
this.version = 1.0
this.type = 'Document'
this.icon = 'confluence.png'
this.category = 'Document Loaders'
this.description = `Load data from a Confluence Document`
this.baseClasses = [this.type]
this.credential = {
label: 'Connect Credential',
name: 'credential',
type: 'credential',
credentialNames: ['confluenceApi']
}
this.inputs = [
{
label: 'Text Splitter',
@@ -27,18 +37,6 @@ class Confluence_DocumentLoaders implements INode {
type: 'TextSplitter',
optional: true
},
{
label: 'Username',
name: 'username',
type: 'string',
placeholder: '<CONFLUENCE_USERNAME>'
},
{
label: 'Access Token',
name: 'accessToken',
type: 'password',
placeholder: '<CONFLUENCE_ACCESS_TOKEN>'
},
{
label: 'Base URL',
name: 'baseUrl',
@@ -49,7 +47,9 @@ class Confluence_DocumentLoaders implements INode {
label: 'Space Key',
name: 'spaceKey',
type: 'string',
placeholder: '~EXAMPLE362906de5d343d49dcdbae5dEXAMPLE'
placeholder: '~EXAMPLE362906de5d343d49dcdbae5dEXAMPLE',
description:
'Refer to <a target="_blank" href="https://community.atlassian.com/t5/Confluence-questions/How-to-find-the-key-for-a-space/qaq-p/864760">official guide</a> on how to get Confluence Space Key'
},
{
label: 'Limit',
@@ -68,16 +68,18 @@ class Confluence_DocumentLoaders implements INode {
]
}
async init(nodeData: INodeData): Promise<any> {
const username = nodeData.inputs?.username as string
const accessToken = nodeData.inputs?.accessToken as string
async init(nodeData: INodeData, _: string, options: ICommonObject): Promise<any> {
const spaceKey = nodeData.inputs?.spaceKey as string
const baseUrl = nodeData.inputs?.baseUrl as string
const limit = nodeData.inputs?.limit as number
const textSplitter = nodeData.inputs?.textSplitter as TextSplitter
const metadata = nodeData.inputs?.metadata
const options: ConfluencePagesLoaderParams = {
const credentialData = await getCredentialData(nodeData.credential ?? '', options)
const accessToken = getCredentialParam('accessToken', credentialData, nodeData)
const username = getCredentialParam('username', credentialData, nodeData)
const confluenceOptions: ConfluencePagesLoaderParams = {
username,
accessToken,
baseUrl,
@@ -85,7 +87,7 @@ class Confluence_DocumentLoaders implements INode {
limit
}
const loader = new ConfluencePagesLoader(options)
const loader = new ConfluencePagesLoader(confluenceOptions)
let docs = []
packages/components/nodes/documentloaders/Csv/Csv.ts
+2
View File
@@ -5,6 +5,7 @@ import { CSVLoader } from 'langchain/document_loaders/fs/csv'
class Csv_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -15,6 +16,7 @@ class Csv_DocumentLoaders implements INode {
constructor() {
this.label = 'Csv File'
this.name = 'csvFile'
this.version = 1.0
this.type = 'Document'
this.icon = 'Csv.png'
this.category = 'Document Loaders'
packages/components/nodes/documentloaders/Docx/Docx.ts
+2
View File
@@ -5,6 +5,7 @@ import { DocxLoader } from 'langchain/document_loaders/fs/docx'
class Docx_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -15,6 +16,7 @@ class Docx_DocumentLoaders implements INode {
constructor() {
this.label = 'Docx File'
this.name = 'docxFile'
this.version = 1.0
this.type = 'Document'
this.icon = 'Docx.png'
this.category = 'Document Loaders'
packages/components/nodes/documentloaders/Figma/Figma.ts
+25 -15
View File
@@ -1,42 +1,50 @@
import { INode, INodeData, INodeParams } from '../../../src/Interface'
import { getCredentialData, getCredentialParam } from '../../../src'
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { FigmaFileLoader, FigmaLoaderParams } from 'langchain/document_loaders/web/figma'
class Figma_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
category: string
baseClasses: string[]
credential: INodeParams
inputs: INodeParams[]
constructor() {
this.label = 'Figma'
this.name = 'figma'
this.version = 1.0
this.type = 'Document'
this.icon = 'figma.png'
this.icon = 'figma.svg'
this.category = 'Document Loaders'
this.description = 'Load data from a Figma file'
this.baseClasses = [this.type]
this.credential = {
label: 'Connect Credential',
name: 'credential',
type: 'credential',
credentialNames: ['figmaApi']
}
this.inputs = [
{
label: 'Access Token',
name: 'accessToken',
type: 'password',
placeholder: '<FIGMA_ACCESS_TOKEN>'
},
{
label: 'File Key',
name: 'fileKey',
type: 'string',
placeholder: 'key'
placeholder: 'key',
description:
'The file key can be read from any Figma file URL: https://www.figma.com/file/:key/:title. For example, in https://www.figma.com/file/12345/Website, the file key is 12345'
},
{
label: 'Node IDs',
name: 'nodeIds',
type: 'string',
placeholder: '0, 1, 2'
placeholder: '0, 1, 2',
description:
'A list of Node IDs, seperated by comma. Refer to <a target="_blank" href="https://www.figma.com/community/plugin/758276196886757462/Node-Inspector">official guide</a> on how to get Node IDs'
},
{
label: 'Recursive',
@@ -60,18 +68,20 @@ class Figma_DocumentLoaders implements INode {
]
}
async init(nodeData: INodeData): Promise<any> {
const accessToken = nodeData.inputs?.accessToken as string
const nodeIds = (nodeData.inputs?.nodeIds as string)?.split(',') || []
async init(nodeData: INodeData, _: string, options: ICommonObject): Promise<any> {
const nodeIds = (nodeData.inputs?.nodeIds as string)?.trim().split(',') || []
const fileKey = nodeData.inputs?.fileKey as string
const options: FigmaLoaderParams = {
const credentialData = await getCredentialData(nodeData.credential ?? '', options)
const accessToken = getCredentialParam('accessToken', credentialData, nodeData)
const figmaOptions: FigmaLoaderParams = {
accessToken,
nodeIds,
fileKey
}
const loader = new FigmaFileLoader(options)
const loader = new FigmaFileLoader(figmaOptions)
const docs = await loader.load()
return docs
packages/components/nodes/documentloaders/Figma/figma.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 172 KiB

packages/components/nodes/documentloaders/Figma/figma.svg
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 48" width="96px" height="96px"><path fill="#e64a19" d="M26,17h-8c-3.866,0-7-3.134-7-7v0c0-3.866,3.134-7,7-7h8V17z"/><path fill="#7c4dff" d="M25,31h-7c-3.866,0-7-3.134-7-7v0c0-3.866,3.134-7,7-7h7V31z"/><path fill="#66bb6a" d="M18,45L18,45c-3.866,0-7-3.134-7-7v0c0-3.866,3.134-7,7-7h7v7C25,41.866,21.866,45,18,45z"/><path fill="#ff7043" d="M32,17h-7V3h7c3.866,0,7,3.134,7,7v0C39,13.866,35.866,17,32,17z"/><circle cx="32" cy="24" r="7" fill="#29b6f6"/></svg>
Side by Side

After

Width:  |  Height:  |  Size: 512 B

packages/components/nodes/documentloaders/Folder/Folder.ts
+2
View File
@@ -10,6 +10,7 @@ import { DocxLoader } from 'langchain/document_loaders/fs/docx'
class Folder_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -20,6 +21,7 @@ class Folder_DocumentLoaders implements INode {
constructor() {
this.label = 'Folder with Files'
this.name = 'folderFiles'
this.version = 1.0
this.type = 'Document'
this.icon = 'folder.svg'
this.category = 'Document Loaders'
packages/components/nodes/documentloaders/Gitbook/Gitbook.ts
+84
View File
@@ -0,0 +1,84 @@
import { INode, INodeData, INodeParams } from '../../../src/Interface'
import { TextSplitter } from 'langchain/text_splitter'
import { GitbookLoader } from 'langchain/document_loaders/web/gitbook'
class Gitbook_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
category: string
baseClasses: string[]
inputs?: INodeParams[]
constructor() {
this.label = 'GitBook'
this.name = 'gitbook'
this.version = 1.0
this.type = 'Document'
this.icon = 'gitbook.svg'
this.category = 'Document Loaders'
this.description = `Load data from GitBook`
this.baseClasses = [this.type]
this.inputs = [
{
label: 'Web Path',
name: 'webPath',
type: 'string',
placeholder: 'https://docs.gitbook.com/product-tour/navigation',
description: 'If want to load all paths from the GitBook provide only root path e.g.https://docs.gitbook.com/ '
},
{
label: 'Should Load All Paths',
name: 'shouldLoadAllPaths',
type: 'boolean',
description: 'Load from all paths in a given GitBook',
optional: true
},
{
label: 'Text Splitter',
name: 'textSplitter',
type: 'TextSplitter',
optional: true
},
{
label: 'Metadata',
name: 'metadata',
type: 'json',
optional: true,
additionalParams: true
}
]
}
async init(nodeData: INodeData): Promise<any> {
const webPath = nodeData.inputs?.webPath as string
const shouldLoadAllPaths = nodeData.inputs?.shouldLoadAllPaths as boolean
const textSplitter = nodeData.inputs?.textSplitter as TextSplitter
const metadata = nodeData.inputs?.metadata
const loader = shouldLoadAllPaths ? new GitbookLoader(webPath, { shouldLoadAllPaths }) : new GitbookLoader(webPath)
const docs = textSplitter ? await loader.loadAndSplit() : await loader.load()
if (metadata) {
const parsedMetadata = typeof metadata === 'object' ? metadata : JSON.parse(metadata)
return docs.map((doc) => {
return {
...doc,
metadata: {
...doc.metadata,
...parsedMetadata
}
}
})
}
return docs
}
}
module.exports = {
nodeClass: Gitbook_DocumentLoaders
}
packages/components/nodes/documentloaders/Gitbook/gitbook.svg
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="64" height="64"><switch><g><path d="M28.8 47.4c1 0 1.9.8 1.9 1.9 0 1-.8 1.9-1.9 1.9-1 0-1.9-.8-1.9-1.9 0-1.1.9-1.9 1.9-1.9m29.4-11.6c-1 0-1.9-.8-1.9-1.9 0-1 .8-1.9 1.9-1.9 1 0 1.9.8 1.9 1.9 0 1-.9 1.9-1.9 1.9m0-7.7c-3.2 0-5.8 2.6-5.8 5.8 0 .6.1 1.2.3 1.8L33.6 45.9c-1.1-1.6-2.9-2.5-4.8-2.5-2.2 0-4.2 1.3-5.2 3.2l-17.2-9c-1.8-1-3.2-3.9-3-6.7.1-1.4.6-2.5 1.3-2.9.5-.3 1-.2 1.7.1l.1.1c4.6 2.4 19.5 10.2 20.1 10.5 1 .4 1.5.6 3.2-.2l30.8-16c.5-.2 1-.6 1-1.3 0-.9-.9-1.3-.9-1.3-1.8-.8-4.5-2.1-7.1-3.3C48 14 41.6 11 38.8 9.5c-2.4-1.3-4.4-.2-4.7 0l-.7.3C20.7 16.2 3.9 24.5 2.9 25.1c-1.7 1-2.8 3.1-2.9 5.7-.2 4.1 1.9 8.4 4.9 9.9l18.2 9.4c.4 2.8 2.9 5 5.7 5 3.2 0 5.7-2.5 5.8-5.7l20-10.8c1 .8 2.3 1.2 3.6 1.2 3.2 0 5.8-2.6 5.8-5.8 0-3.3-2.6-5.9-5.8-5.9" fill="#4285fd"/></g></switch></svg>
Side by Side

After

Width:  |  Height:  |  Size: 826 B

packages/components/nodes/documentloaders/Github/Github.ts
+20 -13
View File
@@ -1,25 +1,37 @@
import { INode, INodeData, INodeParams } from '../../../src/Interface'
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { TextSplitter } from 'langchain/text_splitter'
import { GithubRepoLoader, GithubRepoLoaderParams } from 'langchain/document_loaders/web/github'
import { getCredentialData, getCredentialParam } from '../../../src'
class Github_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
category: string
baseClasses: string[]
credential: INodeParams
inputs: INodeParams[]
constructor() {
this.label = 'Github'
this.name = 'github'
this.version = 1.0
this.type = 'Document'
this.icon = 'github.png'
this.category = 'Document Loaders'
this.description = `Load data from a GitHub repository`
this.baseClasses = [this.type]
this.credential = {
label: 'Connect Credential',
name: 'credential',
type: 'credential',
description: 'Only needed when accessing private repo',
optional: true,
credentialNames: ['githubApi']
}
this.inputs = [
{
label: 'Repo Link',
@@ -33,13 +45,6 @@ class Github_DocumentLoaders implements INode {
type: 'string',
default: 'main'
},
{
label: 'Access Token',
name: 'accessToken',
type: 'password',
placeholder: '<GITHUB_ACCESS_TOKEN>',
optional: true
},
{
label: 'Recursive',
name: 'recursive',
@@ -62,23 +67,25 @@ class Github_DocumentLoaders implements INode {
]
}
async init(nodeData: INodeData): Promise<any> {
async init(nodeData: INodeData, _: string, options: ICommonObject): Promise<any> {
const repoLink = nodeData.inputs?.repoLink as string
const branch = nodeData.inputs?.branch as string
const recursive = nodeData.inputs?.recursive as boolean
const accessToken = nodeData.inputs?.accessToken as string
const textSplitter = nodeData.inputs?.textSplitter as TextSplitter
const metadata = nodeData.inputs?.metadata
const options: GithubRepoLoaderParams = {
const credentialData = await getCredentialData(nodeData.credential ?? '', options)
const accessToken = getCredentialParam('accessToken', credentialData, nodeData)
const githubOptions: GithubRepoLoaderParams = {
branch,
recursive,
unknown: 'warn'
}
if (accessToken) options.accessToken = accessToken
if (accessToken) githubOptions.accessToken = accessToken
const loader = new GithubRepoLoader(repoLink, options)
const loader = new GithubRepoLoader(repoLink, githubOptions)
const docs = textSplitter ? await loader.loadAndSplit(textSplitter) : await loader.load()
if (metadata) {
packages/components/nodes/documentloaders/Json/Json.ts
+2
View File
@@ -5,6 +5,7 @@ import { JSONLoader } from 'langchain/document_loaders/fs/json'
class Json_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -15,6 +16,7 @@ class Json_DocumentLoaders implements INode {
constructor() {
this.label = 'Json File'
this.name = 'jsonFile'
this.version = 1.0
this.type = 'Document'
this.icon = 'json.svg'
this.category = 'Document Loaders'
packages/components/nodes/documentloaders/Jsonlines/Jsonlines.ts
+2
View File
@@ -5,6 +5,7 @@ import { JSONLinesLoader } from 'langchain/document_loaders/fs/json'
class Jsonlines_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -15,6 +16,7 @@ class Jsonlines_DocumentLoaders implements INode {
constructor() {
this.label = 'Json Lines File'
this.name = 'jsonlinesFile'
this.version = 1.0
this.type = 'Document'
this.icon = 'jsonlines.svg'
this.category = 'Document Loaders'
packages/components/nodes/documentloaders/NotionDB/NotionDB.ts → packages/components/nodes/documentloaders/Notion/NotionDB.ts
+25 -26
View File
@@ -1,25 +1,35 @@
import { INode, INodeData, INodeParams } from '../../../src/Interface'
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { TextSplitter } from 'langchain/text_splitter'
import { NotionDBLoader, NotionDBLoaderParams } from 'langchain/document_loaders/web/notiondb'
import { NotionAPILoader, NotionAPILoaderOptions } from 'langchain/document_loaders/web/notionapi'
import { getCredentialData, getCredentialParam } from '../../../src'
class NotionDB_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
category: string
baseClasses: string[]
credential: INodeParams
inputs: INodeParams[]
constructor() {
this.label = 'Notion Database'
this.name = 'notionDB'
this.version = 1.0
this.type = 'Document'
this.icon = 'notion.png'
this.category = 'Document Loaders'
this.description = 'Load data from Notion Database ID'
this.description = 'Load data from Notion Database (each row is a separate document with all properties as metadata)'
this.baseClasses = [this.type]
this.credential = {
label: 'Connect Credential',
name: 'credential',
type: 'credential',
credentialNames: ['notionApi']
}
this.inputs = [
{
label: 'Text Splitter',
@@ -31,21 +41,7 @@ class NotionDB_DocumentLoaders implements INode {
label: 'Notion Database Id',
name: 'databaseId',
type: 'string',
description:
'If your URL looks like - https://www.notion.so/<long_hash_1>?v=<long_hash_2>, then <long_hash_1> is the database ID'
},
{
label: 'Notion Integration Token',
name: 'notionIntegrationToken',
type: 'password',
description:
'You can find integration token <a target="_blank" href="https://developers.notion.com/docs/create-a-notion-integration#step-1-create-an-integration">here</a>'
},
{
label: 'Page Size Limit',
name: 'pageSizeLimit',
type: 'number',
default: 10
description: 'If your URL looks like - https://www.notion.so/abcdefh?v=long_hash_2, then abcdefh is the database ID'
},
{
label: 'Metadata',
@@ -57,19 +53,22 @@ class NotionDB_DocumentLoaders implements INode {
]
}
async init(nodeData: INodeData): Promise<any> {
async init(nodeData: INodeData, _: string, options: ICommonObject): Promise<any> {
const textSplitter = nodeData.inputs?.textSplitter as TextSplitter
const databaseId = nodeData.inputs?.databaseId as string
const notionIntegrationToken = nodeData.inputs?.notionIntegrationToken as string
const pageSizeLimit = nodeData.inputs?.pageSizeLimit as string
const metadata = nodeData.inputs?.metadata
const obj: NotionDBLoaderParams = {
pageSizeLimit: pageSizeLimit ? parseInt(pageSizeLimit, 10) : 10,
databaseId,
notionIntegrationToken
const credentialData = await getCredentialData(nodeData.credential ?? '', options)
const notionIntegrationToken = getCredentialParam('notionIntegrationToken', credentialData, nodeData)
const obj: NotionAPILoaderOptions = {
clientOptions: {
auth: notionIntegrationToken
},
id: databaseId,
type: 'database'
}
const loader = new NotionDBLoader(obj)
const loader = new NotionAPILoader(obj)
let docs = []
if (textSplitter) {
packages/components/nodes/documentloaders/NotionFolder/NotionFolder.ts → packages/components/nodes/documentloaders/Notion/NotionFolder.ts
+2
View File
@@ -5,6 +5,7 @@ import { NotionLoader } from 'langchain/document_loaders/fs/notion'
class NotionFolder_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -15,6 +16,7 @@ class NotionFolder_DocumentLoaders implements INode {
constructor() {
this.label = 'Notion Folder'
this.name = 'notionFolder'
this.version = 1.0
this.type = 'Document'
this.icon = 'notion.png'
this.category = 'Document Loaders'
packages/components/nodes/documentloaders/Notion/NotionPage.ts
+101
View File
@@ -0,0 +1,101 @@
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { TextSplitter } from 'langchain/text_splitter'
import { NotionAPILoader, NotionAPILoaderOptions } from 'langchain/document_loaders/web/notionapi'
import { getCredentialData, getCredentialParam } from '../../../src'
class NotionPage_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
category: string
baseClasses: string[]
credential: INodeParams
inputs: INodeParams[]
constructor() {
this.label = 'Notion Page'
this.name = 'notionPage'
this.version = 1.0
this.type = 'Document'
this.icon = 'notion.png'
this.category = 'Document Loaders'
this.description = 'Load data from Notion Page (including child pages all as separate documents)'
this.baseClasses = [this.type]
this.credential = {
label: 'Connect Credential',
name: 'credential',
type: 'credential',
credentialNames: ['notionApi']
}
this.inputs = [
{
label: 'Text Splitter',
name: 'textSplitter',
type: 'TextSplitter',
optional: true
},
{
label: 'Notion Page Id',
name: 'pageId',
type: 'string',
description:
'The last The 32 char hex in the url path. For example: https://www.notion.so/skarard/LangChain-Notion-API-b34ca03f219c4420a6046fc4bdfdf7b4, b34ca03f219c4420a6046fc4bdfdf7b4 is the Page ID'
},
{
label: 'Metadata',
name: 'metadata',
type: 'json',
optional: true,
additionalParams: true
}
]
}
async init(nodeData: INodeData, _: string, options: ICommonObject): Promise<any> {
const textSplitter = nodeData.inputs?.textSplitter as TextSplitter
const pageId = nodeData.inputs?.pageId as string
const metadata = nodeData.inputs?.metadata
const credentialData = await getCredentialData(nodeData.credential ?? '', options)
const notionIntegrationToken = getCredentialParam('notionIntegrationToken', credentialData, nodeData)
const obj: NotionAPILoaderOptions = {
clientOptions: {
auth: notionIntegrationToken
},
id: pageId,
type: 'page'
}
const loader = new NotionAPILoader(obj)
let docs = []
if (textSplitter) {
docs = await loader.loadAndSplit(textSplitter)
} else {
docs = await loader.load()
}
if (metadata) {
const parsedMetadata = typeof metadata === 'object' ? metadata : JSON.parse(metadata)
let finaldocs = []
for (const doc of docs) {
const newdoc = {
...doc,
metadata: {
...doc.metadata,
...parsedMetadata
}
}
finaldocs.push(newdoc)
}
return finaldocs
}
return docs
}
}
module.exports = { nodeClass: NotionPage_DocumentLoaders }
packages/components/nodes/documentloaders/NotionDB/notion.png → packages/components/nodes/documentloaders/Notion/notion.png
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 11 KiB

packages/components/nodes/documentloaders/NotionFolder/notion.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 11 KiB

packages/components/nodes/documentloaders/Pdf/Pdf.ts
+2
View File
@@ -5,6 +5,7 @@ import { PDFLoader } from 'langchain/document_loaders/fs/pdf'
class Pdf_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -15,6 +16,7 @@ class Pdf_DocumentLoaders implements INode {
constructor() {
this.label = 'Pdf File'
this.name = 'pdfFile'
this.version = 1.0
this.type = 'Document'
this.icon = 'pdf.svg'
this.category = 'Document Loaders'
packages/components/nodes/documentloaders/Playwright/Playwright.ts
+47 -21
View File
@@ -2,11 +2,12 @@ import { INode, INodeData, INodeParams } from '../../../src/Interface'
import { TextSplitter } from 'langchain/text_splitter'
import { PlaywrightWebBaseLoader } from 'langchain/document_loaders/web/playwright'
import { test } from 'linkifyjs'
import { getAvailableURLs } from '../../../src'
import { webCrawl, xmlScrape } from '../../../src'
class Playwright_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -17,6 +18,7 @@ class Playwright_DocumentLoaders implements INode {
constructor() {
this.label = 'Playwright Web Scraper'
this.name = 'playwrightWebScraper'
this.version = 1.0
this.type = 'Document'
this.icon = 'playwright.svg'
this.category = 'Document Loaders'
@@ -35,19 +37,34 @@ class Playwright_DocumentLoaders implements INode {
optional: true
},
{
label: 'Web Scrap for Relative Links',
name: 'webScrap',
type: 'boolean',
label: 'Get Relative Links Method',
name: 'relativeLinksMethod',
type: 'options',
description: 'Select a method to retrieve relative links',
options: [
{
label: 'Web Crawl',
name: 'webCrawl',
description: 'Crawl relative links from HTML URL'
},
{
label: 'Scrape XML Sitemap',
name: 'scrapeXMLSitemap',
description: 'Scrape relative links from XML sitemap URL'
}
],
optional: true,
additionalParams: true
},
{
label: 'Web Scrap Links Limit',
label: 'Get Relative Links Limit',
name: 'limit',
type: 'number',
default: 10,
optional: true,
additionalParams: true
additionalParams: true,
description:
'Only used when "Get Relative Links Method" is selected. Set 0 to retrieve all relative links, default limit is 10.',
warning: `Retreiving all links might take long time, and all links will be upserted again if the flow's state changed (eg: different URL, chunk size, etc)`
},
{
label: 'Metadata',
@@ -62,7 +79,7 @@ class Playwright_DocumentLoaders implements INode {
async init(nodeData: INodeData): Promise<any> {
const textSplitter = nodeData.inputs?.textSplitter as TextSplitter
const metadata = nodeData.inputs?.metadata
const webScrap = nodeData.inputs?.webScrap as boolean
const relativeLinksMethod = nodeData.inputs?.relativeLinksMethod as string
let limit = nodeData.inputs?.limit as string
let url = nodeData.inputs?.url as string
@@ -71,25 +88,34 @@ class Playwright_DocumentLoaders implements INode {
throw new Error('Invalid URL')
}
const playwrightLoader = async (url: string): Promise<any> => {
let docs = []
const loader = new PlaywrightWebBaseLoader(url)
if (textSplitter) {
docs = await loader.loadAndSplit(textSplitter)
} else {
docs = await loader.load()
async function playwrightLoader(url: string): Promise<any> {
try {
let docs = []
const loader = new PlaywrightWebBaseLoader(url)
if (textSplitter) {
docs = await loader.loadAndSplit(textSplitter)
} else {
docs = await loader.load()
}
return docs
} catch (err) {
if (process.env.DEBUG === 'true') console.error(`error in PlaywrightWebBaseLoader: ${err.message}, on page: ${url}`)
}
return docs
}
let availableUrls: string[]
let docs = []
if (webScrap) {
if (relativeLinksMethod) {
if (process.env.DEBUG === 'true') console.info(`Start ${relativeLinksMethod}`)
if (!limit) limit = '10'
availableUrls = await getAvailableURLs(url, parseInt(limit))
for (let i = 0; i < availableUrls.length; i++) {
docs.push(...(await playwrightLoader(availableUrls[i])))
else if (parseInt(limit) < 0) throw new Error('Limit cannot be less than 0')
const pages: string[] =
relativeLinksMethod === 'webCrawl' ? await webCrawl(url, parseInt(limit)) : await xmlScrape(url, parseInt(limit))
if (process.env.DEBUG === 'true') console.info(`pages: ${JSON.stringify(pages)}, length: ${pages.length}`)
if (!pages || pages.length === 0) throw new Error('No relative links found')
for (const page of pages) {
docs.push(...(await playwrightLoader(page)))
}
if (process.env.DEBUG === 'true') console.info(`Finish ${relativeLinksMethod}`)
} else {
docs = await playwrightLoader(url)
}
packages/components/nodes/documentloaders/Puppeteer/Puppeteer.ts
+52 -26
View File
@@ -2,11 +2,12 @@ import { INode, INodeData, INodeParams } from '../../../src/Interface'
import { TextSplitter } from 'langchain/text_splitter'
import { PuppeteerWebBaseLoader } from 'langchain/document_loaders/web/puppeteer'
import { test } from 'linkifyjs'
import { getAvailableURLs } from '../../../src'
import { webCrawl, xmlScrape } from '../../../src'
class Puppeteer_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -17,6 +18,7 @@ class Puppeteer_DocumentLoaders implements INode {
constructor() {
this.label = 'Puppeteer Web Scraper'
this.name = 'puppeteerWebScraper'
this.version = 1.0
this.type = 'Document'
this.icon = 'puppeteer.svg'
this.category = 'Document Loaders'
@@ -35,19 +37,34 @@ class Puppeteer_DocumentLoaders implements INode {
optional: true
},
{
label: 'Web Scrape for Relative Links',
name: 'webScrape',
type: 'boolean',
label: 'Get Relative Links Method',
name: 'relativeLinksMethod',
type: 'options',
description: 'Select a method to retrieve relative links',
options: [
{
label: 'Web Crawl',
name: 'webCrawl',
description: 'Crawl relative links from HTML URL'
},
{
label: 'Scrape XML Sitemap',
name: 'scrapeXMLSitemap',
description: 'Scrape relative links from XML sitemap URL'
}
],
optional: true,
additionalParams: true
},
{
label: 'Web Scrape Links Limit',
label: 'Get Relative Links Limit',
name: 'limit',
type: 'number',
default: 10,
optional: true,
additionalParams: true
additionalParams: true,
description:
'Only used when "Get Relative Links Method" is selected. Set 0 to retrieve all relative links, default limit is 10.',
warning: `Retreiving all links might take long time, and all links will be upserted again if the flow's state changed (eg: different URL, chunk size, etc)`
},
{
label: 'Metadata',
@@ -62,7 +79,7 @@ class Puppeteer_DocumentLoaders implements INode {
async init(nodeData: INodeData): Promise<any> {
const textSplitter = nodeData.inputs?.textSplitter as TextSplitter
const metadata = nodeData.inputs?.metadata
const webScrape = nodeData.inputs?.webScrape as boolean
const relativeLinksMethod = nodeData.inputs?.relativeLinksMethod as string
let limit = nodeData.inputs?.limit as string
let url = nodeData.inputs?.url as string
@@ -71,30 +88,39 @@ class Puppeteer_DocumentLoaders implements INode {
throw new Error('Invalid URL')
}
const puppeteerLoader = async (url: string): Promise<any> => {
let docs = []
const loader = new PuppeteerWebBaseLoader(url)
if (textSplitter) {
docs = await loader.loadAndSplit(textSplitter)
} else {
docs = await loader.load()
async function puppeteerLoader(url: string): Promise<any> {
try {
let docs = []
const loader = new PuppeteerWebBaseLoader(url, {
launchOptions: {
args: ['--no-sandbox'],
headless: 'new'
}
})
if (textSplitter) {
docs = await loader.loadAndSplit(textSplitter)
} else {
docs = await loader.load()
}
return docs
} catch (err) {
if (process.env.DEBUG === 'true') console.error(`error in PuppeteerWebBaseLoader: ${err.message}, on page: ${url}`)
}
return docs
}
let availableUrls: string[]
let docs = []
if (webScrape) {
if (relativeLinksMethod) {
if (process.env.DEBUG === 'true') console.info(`Start ${relativeLinksMethod}`)
if (!limit) limit = '10'
availableUrls = await getAvailableURLs(url, parseInt(limit))
for (let i = 0; i < availableUrls.length; i++) {
try {
docs.push(...(await puppeteerLoader(availableUrls[i])))
} catch (error) {
console.error('Error loading url with puppeteer. URL: ', availableUrls[i], 'Error: ', error)
continue
}
else if (parseInt(limit) < 0) throw new Error('Limit cannot be less than 0')
const pages: string[] =
relativeLinksMethod === 'webCrawl' ? await webCrawl(url, parseInt(limit)) : await xmlScrape(url, parseInt(limit))
if (process.env.DEBUG === 'true') console.info(`pages: ${JSON.stringify(pages)}, length: ${pages.length}`)
if (!pages || pages.length === 0) throw new Error('No relative links found')
for (const page of pages) {
docs.push(...(await puppeteerLoader(page)))
}
if (process.env.DEBUG === 'true') console.info(`Finish ${relativeLinksMethod}`)
} else {
docs = await puppeteerLoader(url)
}
packages/components/nodes/documentloaders/Subtitles/Subtitles.ts
+2
View File
@@ -5,6 +5,7 @@ import { SRTLoader } from 'langchain/document_loaders/fs/srt'
class Subtitles_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -15,6 +16,7 @@ class Subtitles_DocumentLoaders implements INode {
constructor() {
this.label = 'Subtitles File'
this.name = 'subtitlesFile'
this.version = 1.0
this.type = 'Document'
this.icon = 'subtitlesFile.svg'
this.category = 'Document Loaders'
packages/components/nodes/documentloaders/Text/Text.ts
+2
View File
@@ -5,6 +5,7 @@ import { TextLoader } from 'langchain/document_loaders/fs/text'
class Text_DocumentLoaders implements INode {
label: string
name: string
version: number
description: string
type: string
icon: string
@@ -15,6 +16,7 @@ class Text_DocumentLoaders implements INode {
constructor() {
this.label = 'Text File'
this.name = 'textFile'
this.version = 1.0
this.type = 'Document'
this.icon = 'textFile.svg'
this.category = 'Document Loaders'
packages/components/nodes/embeddings/AzureOpenAIEmbedding/Azure.svg
+1 -5
View File
@@ -1,5 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-brand-azure" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
<path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
<path d="M6 7.5l-4 9.5h4l6 -15z"></path>
<path d="M22 20l-7 -15l-3 7l4 5l-8 3z"></path>
</svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 48" width="96px" height="96px"><path fill="#035bda" d="M46 40L29.317 10.852 22.808 23.96 34.267 37.24 13 39.655zM13.092 18.182L2 36.896 11.442 35.947 28.033 5.678z"/></svg>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 392 B

After

Width:  |  Height:  |  Size: 229 B

packages/components/nodes/embeddings/AzureOpenAIEmbedding/AzureOpenAIEmbedding.ts
+18 -32
View File
@@ -1,52 +1,36 @@
import { AzureOpenAIInput } from 'langchain/chat_models/openai'
import { INode, INodeData, INodeParams } from '../../../src/Interface'
import { getBaseClasses } from '../../../src/utils'
import { ICommonObject, INode, INodeData, INodeParams } from '../../../src/Interface'
import { getBaseClasses, getCredentialData, getCredentialParam } from '../../../src/utils'
import { OpenAIEmbeddings, OpenAIEmbeddingsParams } from 'langchain/embeddings/openai'
class AzureOpenAIEmbedding_Embeddings implements INode {
label: string
name: string
version: number
type: string
icon: string
category: string
description: string
baseClasses: string[]
credential: INodeParams
inputs: INodeParams[]
constructor() {
this.label = 'Azure OpenAI Embeddings'
this.name = 'azureOpenAIEmbeddings'
this.version = 1.0
this.type = 'AzureOpenAIEmbeddings'
this.icon = 'Azure.svg'
this.category = 'Embeddings'
this.description = 'Azure OpenAI API to generate embeddings for a given text'
this.baseClasses = [this.type, ...getBaseClasses(OpenAIEmbeddings)]
this.credential = {
label: 'Connect Credential',
name: 'credential',
type: 'credential',
credentialNames: ['azureOpenAIApi']
}
this.inputs = [
{
label: 'Azure OpenAI Api Key',
name: 'azureOpenAIApiKey',
type: 'password'
},
{
label: 'Azure OpenAI Api Instance Name',
name: 'azureOpenAIApiInstanceName',
type: 'string',
placeholder: 'YOUR-INSTANCE-NAME'
},
{
label: 'Azure OpenAI Api Deployment Name',
name: 'azureOpenAIApiDeploymentName',
type: 'string',
placeholder: 'YOUR-DEPLOYMENT-NAME'
},
{
label: 'Azure OpenAI Api Version',
name: 'azureOpenAIApiVersion',
type: 'string',
placeholder: 'YOUR-API-VERSION',
description:
'Description of Supported API Versions. Please refer <a target="_blank" href="https://learn.microsoft.com/en-us/azure/cognitive-services/openai/reference#embeddings">examples</a>'
},
{
label: 'Batch Size',
name: 'batchSize',
@@ -65,14 +49,16 @@ class AzureOpenAIEmbedding_Embeddings implements INode {
]
}
async init(nodeData: INodeData): Promise<any> {
const azureOpenAIApiKey = nodeData.inputs?.azureOpenAIApiKey as string
const azureOpenAIApiInstanceName = nodeData.inputs?.azureOpenAIApiInstanceName as string
const azureOpenAIApiDeploymentName = nodeData.inputs?.azureOpenAIApiDeploymentName as string
const azureOpenAIApiVersion = nodeData.inputs?.azureOpenAIApiVersion as string
async init(nodeData: INodeData, _: string, options: ICommonObject): Promise<any> {
const batchSize = nodeData.inputs?.batchSize as string
const timeout = nodeData.inputs?.timeout as string
const credentialData = await getCredentialData(nodeData.credential ?? '', options)
const azureOpenAIApiKey = getCredentialParam('azureOpenAIApiKey', credentialData, nodeData)
const azureOpenAIApiInstanceName = getCredentialParam('azureOpenAIApiInstanceName', credentialData, nodeData)
const azureOpenAIApiDeploymentName = getCredentialParam('azureOpenAIApiDeploymentName', credentialData, nodeData)
const azureOpenAIApiVersion = getCredentialParam('azureOpenAIApiVersion', credentialData, nodeData)
const obj: Partial<OpenAIEmbeddingsParams> & Partial<AzureOpenAIInput> = {
azureOpenAIApiKey,
azureOpenAIApiInstanceName,

Some files were not shown because too many files have changed in this diff Show More