Commit Graph

192 Commits

Author SHA1 Message Date
Jérémy JOURDIN 15d59a9052 [Prostgres Vector Store] Add PGVector Driver option + Fix null character issue w/ TypeORM Driver (#3367)
* Add PGVector Driver option + Fix null character issue w/ TypeORM Driver

* Handle file upload case with PGVector

* Cleanup

* Fix data filtering for chatflow uploaded files

* Add distanceStrategy parameter

* Fix query to improve chatflow uploaded files filtering

* Ensure PGVector release connections

* Await client connected

* Make Postgres credentials optionnal when set on env variables

* Document env variables in nodes directories

* Prevent reuse client

* Fix empty metadataFilter

* Update CONTRIBUTING.md

* Update Postgres.ts

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-11-01 18:13:45 +00:00
Henry Heng 53e504c32f Feature/Full File Uploads & Message Delete API (#3314)
* add functionality for full file uploads, add remove messages from view dialog and API

* add attachments swagger

* update question to include uploadedFilesContent

* make config dialog modal lg size
2024-10-23 11:00:46 +01:00
Vincelwt 4a9ffe7b99 Lunary: feedback tracking (#3332)
* Lunary: feedback tracking

* fix incorrect param order
2024-10-21 11:32:51 +01:00
Henry Heng a5d4b55bbd Bugfix/Follow up prompts status (#3341)
bugfixes of follow up prompts status, unauthroized error message and rate limit ui
2024-10-11 15:43:28 +01:00
thiagolealassis 7868c8c3a1 Added S3_FORCE_PATH_STYLE environment variable support for AWS S3 client configuration to ensure compatibility with custom S3 endpoints and path-style URLs. This change sets the forcePathStyle option based on the environment variable, defaulting to false if not provided. (#3315)
Co-authored-by: Thiago Assis <thiago.assis@go9.tech>
2024-10-07 16:00:48 +01:00
Ilango c9d8b8716b Feature: Follow-up Prompts (#3280)
* Add migrations - add follow up prompts column to chatflow and chat message

* Add configuration tab for follow-up prompts

* Add follow up prompts functionality

* Pin zod version in components - this was causing a type error with structured outputs

* Generate follow up prompts if enabled and return it in stream, response, and save to database

* Show follow up prompts after getting response

* Add google gen ai for generating follow up prompts and fix issues

* Add config for google gen ai and update model options

* Update follow-up prompts ui and styles

* Release/2.1.0 (#3204)

flowise@2.1.0 release

* Chore/update flowise embed version to 2.0.0 (#3205)

* update flowise-embed version on lock file

* add agent messages to share chatbot

* Update pnpm-lock.yaml

* update flowise-embed version

* update flowise-embed to 1.3.9

* update embed version to 2.0

* Bugfix/CodeInterpreter E2B Credential (#3206)

* Base changes for ServerSide Events (instead of socket.io)

* lint fixes

* adding of interface and separate methods for streaming events

* lint

* first draft, handles both internal and external prediction end points.

* lint fixes

* additional internal end point for streaming and associated changes

* return streamresponse as true to build agent flow

* 1) JSON formatting for internal events
2) other fixes

* 1) convert internal event to metadata to maintain consistency with external response

* fix action and metadata streaming

* fix for error when agent flow is aborted

* prevent subflows from streaming and other code cleanup

* prevent streaming from enclosed tools

* add fix for preventing chaintool streaming

* update lock file

* add open when hidden to sse

* Streaming errors

* Streaming errors

* add fix for showing error message

* add code interpreter

* add artifacts to view message dialog

* Update pnpm-lock.yaml

* uncomment e2b credential

---------

Co-authored-by: Vinod Paidimarry <vinodkiran@outlook.in>

* Release/2.1.0 (#3207)

* flowise@2.1.0 release

* update flowise-components@2.1.1

* Bugfix/Add artifacts migration script to other database types (#3210)

add artifacts migration script to other database types

* Release/2.1.1 (#3213)

release @2.1.1

* Bugfix/Add header to allow sse on nginx (#3214)

add header to allow sse on nginx

* Bugfix/remove invalid markdown (#3219)

remove invalid markdown

* Correct "as" casing (#3216)

* Correct "as" casing

* Remove "version" line from docker compose file

* Update docker-compose.yml

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>

* chore: update unstructured API url and doc reference (#3224)

chore: udpate unstructured API url and doc reference

* Feature/add ability to specify dynamic metadata to jsonlines (#3238)

* add ability to specify dynamic metadata to jsonlines

* fix additional metadata

* Bugfix/Buffer Memory for Anthropic (#3242)

fix buffer memory

* Added env vars to ui and api URL  (#3141)

* feat: add environment vars to split application in different deployments for better scalability

* update: package.json

added start script ui

---------

Co-authored-by: patrick <patrick.alves@br.experian.com>

* Added 1-click deployment link for Alibaba Cloud.  (#3251)

* Added a link for Alibaba Cloud Deployment

* change service name

---------

Co-authored-by: yehan <gn398171@alibaba-inc.com>

* Chore/Groq Llama3.2 (#3255)

* add gemini flash

* add gemin flash to vertex

* add gemin-1.5-flash-preview to vertex

* add azure gpt 4o

* add claude 3.5 sonnet

* add mistral nemo

* add groq llama3.1

* add gpt4o-mini to azure

* o1 mini

* add groq llama 3.2

* Bugfix/Prevent streaming of chatflow tool and chain tool (#3257)

prevent streaming of chatflow tool and chain tool

* Bugfix/Enable Custom Tool Optional Input Schema (#3258)

* prevent streaming of chatflow tool and chain tool

* enable optional input schema

* Bugfix/Searxng tool not working (#3263)

fix searxng tool not working

* LunaryAI automatic Thread and User tracking (#3233)

* Lunary Thread/User tracking

* Clean console logs

* Clean

* Remove commented lines

* Remove commented line

* feat: enable autofocus to the `new chatflow title` to improve usability (#3260)

This dialog has only one input and it is the primary one, there is no need for an extra click to be able to set the title

* feat: save a new Chatflow when the `ENTER` key is pressed (#3261)

This simple event handler improve the usability of the UI by avoiding having to use the mouse or having to tab twice and then hit enter to save a flow

* feat: save Chatflow title when the `ENTER` key is pressed or discard upon `ESC` is pressed (#3265)

This simple event handler improves the usability of the UI by avoiding having to use the mouse to save or dicard title changes

* feat: enable autofocus to the `edit chatflow title` field to improve UI usability (#3264)

feat: enable autofocus to the `edit chatflow title` field to improve usability

The canvas header has only one input and it is the primary one, there is no need for an extra click to be able to edit the title

* feat: add search keyboard shortcut based on the current platform (#3267)

* feat: highlight valid/invalid connection between nodes (#3266)

Change the inputs background to green/red to hint compatible connections, in adition to the `not-allowed` mouse cursor for incompatible connections

* Bugfix/add fixes for search of view header (#3271)

add fixes for search of view header

* fix: warning when passing a boolean to border property of a Card (#3275)

By default MainCard wrappers like NodeCardWrapper and CardWrapper add a a solid border of 1px, but if the `MainCard.border` prop is used (`false`) the border prop was wrongly set to a boolean instead of string

* feat: add shortcut text hint to the search field (#3269)

* feat: add shortcut text hint to the search field

* fix: search box width to fit the shortcut hint text

* fix: error when not running on Mac due to an undefined `os` variable

* fix: warning when a non-boolean values was used to set `checked` prop of a SwitchInput component (#3276)

fix: warning when a non-boolean values was used to set`checked` prop of SwitchInput component

The problem was that in the useEffect hook the plain value was used without validation like in useState

* Bugfix/Throw error to prevent SSE from retrying (#3281)

throw error to prevent SSE from retrying

* Pin zod version in components - this was causing a type error with structured outputs

* Fix conflicts in pnpm lock

* fix ui changes for follow up prompts

* Fix button disable state in follow-up prompts configuration

* Fix follow-up prompts not showing up for agent flows

* Show follow up prompts if last message is apiMessage and follow up prompts are available

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
Co-authored-by: Vinod Paidimarry <vinodkiran@outlook.in>
Co-authored-by: Cross <github@dillfrescott.com>
Co-authored-by: cragwolfe <cragcw@gmail.com>
Co-authored-by: patrickreinan <patrickreinan@gmail.com>
Co-authored-by: patrick <patrick.alves@br.experian.com>
Co-authored-by: yehan <34835250+yehanyh@users.noreply.github.com>
Co-authored-by: yehan <gn398171@alibaba-inc.com>
Co-authored-by: Vincelwt <vincelwt@users.noreply.github.com>
Co-authored-by: Humberto Rodríguez A. <rhumbertgz@users.noreply.github.com>
Co-authored-by: Henry <hzj94@hotmail.com>
2024-10-04 16:44:28 +01:00
Henry Heng 04e8d02a99 Bugfix/Upsert files extension to input field (#3288)
get upsert files extension to input field
2024-09-30 22:52:02 +01:00
Vincelwt 18f916a7e1 LunaryAI automatic Thread and User tracking (#3233)
* Lunary Thread/User tracking

* Clean console logs

* Clean

* Remove commented lines

* Remove commented line
2024-09-26 12:01:33 +01:00
Henry Heng bc76886178 Bugfix/Enable Custom Tool Optional Input Schema (#3258)
* prevent streaming of chatflow tool and chain tool

* enable optional input schema
2024-09-26 01:09:35 +01:00
Henry Heng 1219c2ff12 Bugfix/remove invalid markdown (#3219)
remove invalid markdown
2024-09-19 14:56:13 +01:00
Henry Heng b02f279e9d Feature/Code Interpreter (#3183)
* Base changes for ServerSide Events (instead of socket.io)

* lint fixes

* adding of interface and separate methods for streaming events

* lint

* first draft, handles both internal and external prediction end points.

* lint fixes

* additional internal end point for streaming and associated changes

* return streamresponse as true to build agent flow

* 1) JSON formatting for internal events
2) other fixes

* 1) convert internal event to metadata to maintain consistency with external response

* fix action and metadata streaming

* fix for error when agent flow is aborted

* prevent subflows from streaming and other code cleanup

* prevent streaming from enclosed tools

* add fix for preventing chaintool streaming

* update lock file

* add open when hidden to sse

* Streaming errors

* Streaming errors

* add fix for showing error message

* add code interpreter

* add artifacts to view message dialog

* Update pnpm-lock.yaml

---------

Co-authored-by: Vinod Paidimarry <vinodkiran@outlook.in>
2024-09-17 08:44:56 +01:00
Vinod Kiran 26444ac3ae Feature/sse (#3125)
* Base changes for ServerSide Events (instead of socket.io)

* lint fixes

* adding of interface and separate methods for streaming events

* lint

* first draft, handles both internal and external prediction end points.

* lint fixes

* additional internal end point for streaming and associated changes

* return streamresponse as true to build agent flow

* 1) JSON formatting for internal events
2) other fixes

* 1) convert internal event to metadata to maintain consistency with external response

* fix action and metadata streaming

* fix for error when agent flow is aborted

* prevent subflows from streaming and other code cleanup

* prevent streaming from enclosed tools

* add fix for preventing chaintool streaming

* update lock file

* add open when hidden to sse

* Streaming errors

* Streaming errors

* add fix for showing error message

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-09-17 08:01:25 +01:00
Henry Heng 7a4c7efcab Chore/changed vm2 to @flowiseai/nodevm (#3200)
* changed vm2 to @flowiseai/nodevm

* update set-value and lunary version
2024-09-17 01:57:02 +01:00
Vinod Kiran 8bd3de4153 Bugfix: Check for relative path when saving file, to prevent unauthorised writes (#3172)
* Check for relative path when saving file, to prevent unauthorised writes

* preventing relative paths for all modes (s3/local)

* preventing relative paths for all modes (s3/local)

* Update storageUtils.ts

* changing the code to sanitizing filenames.

* fix lock file

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
Co-authored-by: Henry <hzj94@hotmail.com>
2024-09-14 13:58:52 +01:00
Henry Heng 474793486b Bugfix/Prevent sending non image file (#3173)
* bugfix to prevent sending non image file

* fix typo
2024-09-09 21:02:47 +01:00
Henry Heng 4c1378c670 Feature/Append uploaded images to message history (#3105)
append uploaded images to message history
2024-09-02 11:27:45 +01:00
Jrakru 2e45851822 Passing state to tool so that we can use them in custom tools (#3103) 2024-08-30 20:50:16 +01:00
Meltcap eb595d2a66 Bugfix/langwatch missing returnids (#3077)
* Update handler.ts

Add missing langWatch: {} to onLLMStart and onLLMStart

* Update handler.ts

Revert changes so I can make bugfix branch

* Update handler.ts - missing langWatch

Add missing langWatch object to returnIds for missing 'span' bug.
2024-08-25 13:39:19 +01:00
Henry Heng 66acd0c000 Feature/add ability to upload file from chat (#3059)
add ability to upload file from chat
2024-08-25 13:22:48 +01:00
Henry Heng 0a36aa7ef4 Feature/Add message history to agents (#3031)
add message history to agents
2024-08-17 19:28:01 +01:00
Vinod Kiran c0bae635b0 Document Store - Phase 2 (#2912)
* Document Store - Phase 2

* Adding additional columns for vector store config, document store phase 2

* Adding additional columns for vector store config, document store phase 2

* Document Store - Phase 2 - Upsert and Query

* ux cleanup

* retrieval settings and more ux changes

* adding MMR params to execution

* Making the upsert process async.

* add upsert history changes

* making the searchParams dynamic

* removing unnecessary params

* add ability to delete data from vector store

* update margin for vector store query

* adding option to save config in the retrieval playground

* adding chunk number for query return chunks

* Adding a Document Store node in the VectorStore category

* update doc store status, ui touchup

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-08-07 18:59:52 +01:00
Henry Heng 70d7420d02 Feature/Seq Agent Nodes Update (#2942)
update all seg agent nodes to 2.0 for UI UX improvements
2024-08-06 11:30:08 +01:00
Henry Heng 27cb143fef Feature/Community Node (#2902)
add feature to enable community node
2024-07-29 16:52:47 +01:00
Henry Heng b5d969b2bb Feature/prepend messages to agent graph (#2897)
prepend messages to agent graph
2024-07-29 11:09:38 +01:00
Henry Heng 1c730323e2 Feat/add s3 custom url (#2891)
add s3 custom url
2024-07-26 15:39:24 +01:00
Henry Heng 2dadf2e42b Bugfix/undefined returned by agent for lowercase (#2885)
fix undefined returned by agent for lowercase
2024-07-26 00:46:54 +01:00
Henry Heng 368c69cbc5 Bugfix/Move summarization as optional for multi agents (#2858)
add summarization as optional for multi agents
2024-07-23 15:15:41 +01:00
Henry Heng bca4de0c63 Feature/seq agents (#2798)
* update build functions

* sequential agents

* update langchain to 0.2, added sequential agent nodes

* add marketplace templates

* update howto wordings

* Merge branch 'main' into feature/Seq-Agents

# Conflicts:
#	pnpm-lock.yaml

* update deprecated functions and add new sequential nodes

* add marketplace templates

* update marketplace templates, add structured output to llm node

* add multi agents template

* update llm node with bindmodels

* update cypress version

* update templates sticky note wordings

* update tool node to include human in loop action

* update structured outputs error from models

* update cohere package to resolve google genai pipeThrough bug

* update mistral package version, added message reconstruction before invoke seq agent

* add HITL to agent

* update state messages restructuring

* update load and split methods for s3 directory
2024-07-22 17:46:14 +01:00
Neal Beeken 9e88c45051 feat: add driverInfo to mongodb component (#2779)
* feat: add driverInfo to mongodb component

NODE-6240

* chore: add a getVersion utility function
2024-07-15 12:24:00 +01:00
Henry Heng 363d1bfc44 Chore/update deprecating nodes (#2540)
* update deprecating nodes

* add filters use cases to marketplace

* update log level
2024-07-12 18:37:57 +01:00
Arun Lodhi b1e38783e4 Bugfix/observation-includes-not-function (#2744)
* Bugfix: observation?.includes is not a function

* Check type of observation before checking source document prefix

* lint-fix

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-07-05 11:26:05 +01:00
Rogério Chaves 0fc5e3d0c5 Add LangWatch integration (#2677)
Add langwatch integration
2024-06-21 16:26:55 +01:00
Henry Heng 842bfc66fe Feature/Add ability to upload image url when calling API (#2683)
add ability to upload image url when calling API
2024-06-19 23:26:56 +01:00
Rafael Reis eb738a1552 Fix for Whisper Error: 'File is not defined' when using Speech to Text (#2526)
* tested ok

* update localai stt file

* update toFile method for OpenAI Assistant uploads

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
Co-authored-by: Henry <hzj94@hotmail.com>
2024-05-30 12:54:55 +01:00
Henry Heng 059eae4268 Bugfix/model list unreachable (#2522)
* fallback to fetch local models json file

* enable user to specify local path to load models.json

* Update pnpm-lock.yaml
2024-05-30 11:59:19 +01:00
Daniel D'Abate 912c8f3d5b Feature: Support role-based authentication for AWS (#2470)
* Storage, DynamoDBChatMemory - Make AWS credentials optional to support role-based authentication

* Lint fix
2024-05-29 23:40:01 +01:00
patrickreinan 2878af69e4 Added LOG_JSON_SPACES to control json beautify (#2483)
* Added: environment var LOG_JSON_SPACES with default value 2. Used to no beautify JSON on handler.ts
Fix: logger.verbose was not working because default log level was info

* Update handler.ts

---------

Co-authored-by: patrick <patrick.alves@br.experian.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-05-29 18:25:32 +01:00
Vinod Kiran ff2381741e Chore/upgrade llamaindex version (#2440)
* updates to loader to support file upload

* adding a todo

* upgrade llamaindex

* update groq icon

* update azure models

* update llamaindex version

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-05-22 13:35:08 +01:00
Henry Heng 8ebc4dcfd5 Feature/lang graph (#2319)
* add langgraph

* datasource: initial commit

* datasource: datasource details and chunks

* datasource: Document Store Node

* more changes

* Document Store - Base functionality

* Document Store Loader Component

* Document Store Loader Component

* before merging the modularity PR

* after merging the modularity PR

* preview mode

* initial draft PR

* fixes

* minor updates and  fixes

* preview with loader and splitter

* preview with credential

* show stored chunks

* preview update...

* edit config

* save, preview and other changes

* save, preview and other changes

* save, process and other changes

* save, process and other changes

* alpha1 - for internal testing

* rerouting urls

* bug fix on new leader create

* pagination support for chunks

* delete document store

* Update pnpm-lock.yaml

* doc store card view

* Update store files to use updated storage functions, Document Store Table View and other changes

* ui changes

* add expanded chunk dialog, improve ui

* change throw Error to InternalError

* Bug Fixes and removal of subFolder, adding of view chunks for store

* lint fixes

* merge changes

* DocumentStoreStatus component

* ui changes for doc store

* add remove metadata key field, add custom document loader

* add chatflows used doc store chips

* add types/interfaces to DocumentStore Services

* document loader list dialog title bar color change

* update interfaces

* Whereused Chatflow Name and Added chunkNo to retain order of created chunks.

* use typeorm order chunkNo, ui changes

* update tabler icons react

* cleanup agents

* add pysandbox tool

* add abort functionality, loading next agent

* add empty view svg

* update chatflow tool with chatId

* rename to agentflows

* update worker for prompt input values

* update dashboard to agentflows, agentcanvas

* fix marketplace use template

* add agentflow templates

* resolve merge conflict

* update baseURL

---------

Co-authored-by: vinodkiran <vinodkiran@usa.net>
Co-authored-by: Vinod Paidimarry <vinodkiran@outlook.in>
2024-05-21 16:36:42 +01:00
Henry Heng 8caca472ba Feature/Add prepend messages to memory (#2410)
add prepend messages to memory
2024-05-20 17:08:34 +01:00
Henry Heng b5e502f3b6 Feature/Multer to s3 (#2408)
* add ability to store files from multer to s3

* add check to bypass doc loader
2024-05-15 19:41:37 +01:00
Henry Heng c022972cf8 Bugfix/Local Models Json (#2416)
fallback to fetch local models json file
2024-05-15 19:39:05 +01:00
clates d3f03e380e [FEATURE] Added support for LocalAI Speech To Text configuration (#2376)
* Added support for LocalAI to the Speech To Text configuration. Added a few debug statements around speech to text conversion. Finally, refactored the speechToTextProviders a bit to try and remove some magic strings that have undocumented rules around naming.

* LocalAI STT - PR Feedback - Updated LocalAI Image, changed casing, and updated the default model to whisper-1.
2024-05-13 12:21:27 +01:00
Vinod Kiran 40e36d1b39 Feature/DocumentStore (#2106)
* datasource: initial commit

* datasource: datasource details and chunks

* datasource: Document Store Node

* more changes

* Document Store - Base functionality

* Document Store Loader Component

* Document Store Loader Component

* before merging the modularity PR

* after merging the modularity PR

* preview mode

* initial draft PR

* fixes

* minor updates and  fixes

* preview with loader and splitter

* preview with credential

* show stored chunks

* preview update...

* edit config

* save, preview and other changes

* save, preview and other changes

* save, process and other changes

* save, process and other changes

* alpha1 - for internal testing

* rerouting urls

* bug fix on new leader create

* pagination support for chunks

* delete document store

* Update pnpm-lock.yaml

* doc store card view

* Update store files to use updated storage functions, Document Store Table View and other changes

* ui changes

* add expanded chunk dialog, improve ui

* change throw Error to InternalError

* Bug Fixes and removal of subFolder, adding of view chunks for store

* lint fixes

* merge changes

* DocumentStoreStatus component

* ui changes for doc store

* add remove metadata key field, add custom document loader

* add chatflows used doc store chips

* add types/interfaces to DocumentStore Services

* document loader list dialog title bar color change

* update interfaces

* Whereused Chatflow Name and Added chunkNo to retain order of created chunks.

* use typeorm order chunkNo, ui changes

---------

Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-05-06 15:23:27 +01:00
Ahmed Osman 2254d16c3a FIX Web Crawler doesn't include search params in URLs (#2300) 2024-05-02 18:42:00 +01:00
Vinod Kiran d5a97060e2 FEATURE: Adding File Upload to Unstructured Loader (#2304)
* initial commit

* updates to loader to support file upload

* updates to loader to support file upload

* update unstructured file

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-05-02 18:34:32 +01:00
Henry Heng 7360d1d9a6 Feature/OpenAI Assistant V2 (#2258)
* add gpt4 turbo to assistant

* OpenAI Assistant V2

* update langfuse handler
2024-04-25 20:14:04 +01:00
Vinod Kiran 7006d64de0 Feature/s3 storage (#2226)
* centralizing file writing....

* allowing s3 as storage option

* allowing s3 as storage option

* update s3 storage

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-04-23 11:35:38 +01:00
Henry Heng 95beaba9d9 Chore/Update langchain version, openai, mistral, vertex, anthropic (#2180)
* update langchain version, openai, mistral, vertex, anthropic, introduced toolagent

* upgrade @google/generative-ai 0.7.0, replicate and faiss-node

* update cohere ver

* adding chatCohere to streaming

* update gemini to have image upload

* update google genai, remove aiplugin
2024-04-20 02:20:30 +01:00
Vinod Kiran 6bd8aaefc8 Bugfix/speech input on Safari/iOS (#1971)
* debug to identify

* Safari sends audio file as mp4 and nor webp

* Safari on iOS needs special handling

* lint fixes

* updated condition

* Remove unused import

---------

Co-authored-by: Ilango <rajagopalilango@gmail.com>
2024-04-19 01:28:09 +01:00