Commit Graph

477 Commits

Author SHA1 Message Date
YISH 83ecc88b35 feat (server): add support for setting listening host (#2604) 2024-06-21 14:51:03 +01:00
Henry Heng 842bfc66fe Feature/Add ability to upload image url when calling API (#2683)
add ability to upload image url when calling API
2024-06-19 23:26:56 +01:00
YISH 371e632a2c Fix apikey not URL safe (#2602) 2024-06-14 11:35:25 +01:00
Henry Heng 3ab0d99711 Bugfix/Correctly throw 401 error when unauthorized (#2626)
correctly throw 401 error when unauthorized
2024-06-12 19:10:42 +01:00
Jared McQueen 5e4d640ed7 fix totalChars accumulator for undefined pageContent (#2619) 2024-06-11 22:05:04 +01:00
Mingxin Hou 88ee9b09a7 Fireworks ai chat model (#2596)
* fireworks chat model

* add chatFireworks to streamAvaliableLLMs

* add model parameter input

* Update ChatFireworks.ts

* fix linting

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-06-11 21:56:22 +01:00
不知火 Shiranui c8939dc2a6 Add data source adapter of MariaDB (#2595)
fix(datasource.ts): add mariadb
2024-06-07 12:14:06 +01:00
Daniel D'Abate 34251fa336 Bugfix - Add x-forwarded-proto as source for http protocol for base url and add base url field to ChatflowTool (#2592) 2024-06-07 12:04:25 +01:00
Henry Heng 582dcc8508 Feature/add status check ping (#2579)
add status check ping
2024-06-04 18:45:45 +01:00
Henry Heng 4ec8376efa Bugfix/get rid of double quotes when replacing variable value (#2577)
get rid of double quotes when replacing variable value
2024-06-04 15:44:27 +01:00
Daniel D'Abate 5ba9493b30 Feature - Env variable to disable ChatFlow reuse (#2559) 2024-06-04 10:11:57 +01:00
Jared McQueen bdbb6f850a adding a document loader returns the correct id (#2538) 2024-06-04 02:57:30 +01:00
Jared McQueen 7eb9341fdc adding url to source for all web scrapers (#2537) 2024-06-04 02:50:43 +01:00
Henry Heng 6a58ae4e80 Bugfix/In-mem VS not able to load document (#2542)
fix where in-mem VS not able to load document
2024-05-31 23:41:21 +01:00
YISH d50563765e Fix JSON escaping (better) (#2498)
Fix JSON escaping
2024-05-30 12:58:20 +01:00
Rafael Reis eb738a1552 Fix for Whisper Error: 'File is not defined' when using Speech to Text (#2526)
* tested ok

* update localai stt file

* update toFile method for OpenAI Assistant uploads

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
Co-authored-by: Henry <hzj94@hotmail.com>
2024-05-30 12:54:55 +01:00
Henry Heng 97386bc3b2 Bugfix/Files not removed when doc store loader is deleted (#2502)
fix files not removed when doc store loader is deleted
2024-05-28 22:36:12 +01:00
YISH 82899d9d5d Fix JSON escaping (#2461) 2024-05-24 02:26:34 +01:00
Henry Heng e32b643445 Bugfix/Regex check for auth middleware (#2469)
add regex check for auth middleware
2024-05-23 15:46:11 +01:00
Vinod Kiran ff2381741e Chore/upgrade llamaindex version (#2440)
* updates to loader to support file upload

* adding a todo

* upgrade llamaindex

* update groq icon

* update azure models

* update llamaindex version

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-05-22 13:35:08 +01:00
Henry Heng 8ebc4dcfd5 Feature/lang graph (#2319)
* add langgraph

* datasource: initial commit

* datasource: datasource details and chunks

* datasource: Document Store Node

* more changes

* Document Store - Base functionality

* Document Store Loader Component

* Document Store Loader Component

* before merging the modularity PR

* after merging the modularity PR

* preview mode

* initial draft PR

* fixes

* minor updates and  fixes

* preview with loader and splitter

* preview with credential

* show stored chunks

* preview update...

* edit config

* save, preview and other changes

* save, preview and other changes

* save, process and other changes

* save, process and other changes

* alpha1 - for internal testing

* rerouting urls

* bug fix on new leader create

* pagination support for chunks

* delete document store

* Update pnpm-lock.yaml

* doc store card view

* Update store files to use updated storage functions, Document Store Table View and other changes

* ui changes

* add expanded chunk dialog, improve ui

* change throw Error to InternalError

* Bug Fixes and removal of subFolder, adding of view chunks for store

* lint fixes

* merge changes

* DocumentStoreStatus component

* ui changes for doc store

* add remove metadata key field, add custom document loader

* add chatflows used doc store chips

* add types/interfaces to DocumentStore Services

* document loader list dialog title bar color change

* update interfaces

* Whereused Chatflow Name and Added chunkNo to retain order of created chunks.

* use typeorm order chunkNo, ui changes

* update tabler icons react

* cleanup agents

* add pysandbox tool

* add abort functionality, loading next agent

* add empty view svg

* update chatflow tool with chatId

* rename to agentflows

* update worker for prompt input values

* update dashboard to agentflows, agentcanvas

* fix marketplace use template

* add agentflow templates

* resolve merge conflict

* update baseURL

---------

Co-authored-by: vinodkiran <vinodkiran@usa.net>
Co-authored-by: Vinod Paidimarry <vinodkiran@outlook.in>
2024-05-21 16:36:42 +01:00
Henry Heng 8caca472ba Feature/Add prepend messages to memory (#2410)
add prepend messages to memory
2024-05-20 17:08:34 +01:00
Henry Heng b5e502f3b6 Feature/Multer to s3 (#2408)
* add ability to store files from multer to s3

* add check to bypass doc loader
2024-05-15 19:41:37 +01:00
clates d3f03e380e [FEATURE] Added support for LocalAI Speech To Text configuration (#2376)
* Added support for LocalAI to the Speech To Text configuration. Added a few debug statements around speech to text conversion. Finally, refactored the speechToTextProviders a bit to try and remove some magic strings that have undocumented rules around naming.

* LocalAI STT - PR Feedback - Updated LocalAI Image, changed casing, and updated the default model to whisper-1.
2024-05-13 12:21:27 +01:00
Saurabh Gupta 823cefb5c5 fix initDatabase function by proper use of await in try catch (#2360)
* fix initDatabase function by proper use of await in try catch

* lint-fix

---------

Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-05-13 12:15:22 +01:00
Henry Heng a9f9c8874c Bugfix/Save chunk's metadata (#2366)
save metadata chunk
2024-05-08 17:24:03 +01:00
Henry Heng d4a5474f48 Bugfix/Escape column name on postgres migration indexing (#2342)
escape column name on postgres migration indexing
2024-05-07 01:10:46 +01:00
Henry Heng 80f24ac30c Bugfix/Default file size limit (#2334)
update default flowise file size limit
2024-05-06 17:21:22 +01:00
Vinod Kiran 40e36d1b39 Feature/DocumentStore (#2106)
* datasource: initial commit

* datasource: datasource details and chunks

* datasource: Document Store Node

* more changes

* Document Store - Base functionality

* Document Store Loader Component

* Document Store Loader Component

* before merging the modularity PR

* after merging the modularity PR

* preview mode

* initial draft PR

* fixes

* minor updates and  fixes

* preview with loader and splitter

* preview with credential

* show stored chunks

* preview update...

* edit config

* save, preview and other changes

* save, preview and other changes

* save, process and other changes

* save, process and other changes

* alpha1 - for internal testing

* rerouting urls

* bug fix on new leader create

* pagination support for chunks

* delete document store

* Update pnpm-lock.yaml

* doc store card view

* Update store files to use updated storage functions, Document Store Table View and other changes

* ui changes

* add expanded chunk dialog, improve ui

* change throw Error to InternalError

* Bug Fixes and removal of subFolder, adding of view chunks for store

* lint fixes

* merge changes

* DocumentStoreStatus component

* ui changes for doc store

* add remove metadata key field, add custom document loader

* add chatflows used doc store chips

* add types/interfaces to DocumentStore Services

* document loader list dialog title bar color change

* update interfaces

* Whereused Chatflow Name and Added chunkNo to retain order of created chunks.

* use typeorm order chunkNo, ui changes

---------

Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-05-06 15:23:27 +01:00
Henry Heng 5a137a478c Feature/TogetherAI (#2318)
add togetherAI
2024-05-04 18:25:13 +01:00
Ilango db452cd74d Feature: Collect contact information from users inside the chatbot (#1948)
* Add leads settings to chatflow configuration

* Add leads tab to chatflow configuration with options for lead capture

* Add database entity and migrations for leads

* Add endpoint for adding and fetching leads

* Show lead capture form in UI chat window when enabled

* Add view leads dialog

* Make export leads functional

* Add input for configuring message on successful lead capture

* Add migrations for adding lead email in chat message if available

* show lead email in view messages

* ui touch up

* Remove unused code and update how lead email is shown in view messages dialog

* Fix lead not getting saved

* Disable input when lead form is shown and save lead info to localstorage

* Fix lead capture form not working

* disabled lead save button until at least one form field is turned on, get rid of local storage _LEAD

* add leads API to as whitelist public endpoint

* Send leadEmail in internal chat inputs

* Fix condition for disabling input field and related buttons when lead is enabled/disabled and when lead is saved

* update leads ui

* update error message and alter table add column sqlite migration

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-05-02 01:54:59 +01:00
Henry Heng 5775947586 Bugfix/IP API (#2283)
fix ip API
2024-04-29 22:19:31 +01:00
Naga31 51058b2a31 Fix for 1869 Issue of Update Crendential (#2260)
Fix 1869 Issue of Update Crendential
2024-04-26 16:10:47 +01:00
Henry Heng 7360d1d9a6 Feature/OpenAI Assistant V2 (#2258)
* add gpt4 turbo to assistant

* OpenAI Assistant V2

* update langfuse handler
2024-04-25 20:14:04 +01:00
Vinod Kiran 7006d64de0 Feature/s3 storage (#2226)
* centralizing file writing....

* allowing s3 as storage option

* allowing s3 as storage option

* update s3 storage

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-04-23 11:35:38 +01:00
Henry Heng 95beaba9d9 Chore/Update langchain version, openai, mistral, vertex, anthropic (#2180)
* update langchain version, openai, mistral, vertex, anthropic, introduced toolagent

* upgrade @google/generative-ai 0.7.0, replicate and faiss-node

* update cohere ver

* adding chatCohere to streaming

* update gemini to have image upload

* update google genai, remove aiplugin
2024-04-20 02:20:30 +01:00
YISH b7e4fc9517 Optimize getEndingNodes (#2133) 2024-04-19 02:12:24 +01:00
Vinod Kiran 6bd8aaefc8 Bugfix/speech input on Safari/iOS (#1971)
* debug to identify

* Safari sends audio file as mp4 and nor webp

* Safari on iOS needs special handling

* lint fixes

* updated condition

* Remove unused import

---------

Co-authored-by: Ilango <rajagopalilango@gmail.com>
2024-04-19 01:28:09 +01:00
automaton82 2cadd68a43 Changing 'Output Schema' to 'Input Schema' (#2199) 2024-04-17 10:30:41 +01:00
Vinod Kiran 788d40f26b Feature/externalize model list (#2113)
* fixes

* test json

* test json

* externalize model list

* externalize model list

* correcting awsChatBedRock Name

* lint fixes

* externalize models

* externalize chat models

* externalize chat models

* externalize llm models

* externalize llm models

* lint fixes

* addition of models for ChatMistral

* updating version numbers for all imapacted nodes.

* marketplace template updates

* update embeddings, marketplaces, add chatCohere

* config changes for new env variable

* removal of local models.json

* update models json file

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-04-12 12:46:10 +01:00
Henry Heng c33642cdf9 Feature/Buffer Memory SessionId (#2111)
* add sessionId to buffer memory, add conversation summary buffer memory

* add fix for conv retrieval qa chain
2024-04-11 11:18:39 +01:00
Octavian FlowiseAI d7194e8aaa Chore/consistent services and error handlers (#2101)
* Update index.ts

* Update buildChatflow.ts

* Update index.ts

* Update index.ts

* Update index.ts

* Update index.ts

* Update index.ts

* Update index.ts

* Update index.ts

* Update index.ts

* Consistency

* Rename ApiError to InternalServerError

* Use InternalFlowiseError in controllers

* Use InternalFlowiseError in services

* Catch routes without preconditioned parameters

* Reconfigure the route precondition checks

* Fix router precondition checks

* cleanup status codes, get proper error messages

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-04-11 11:11:01 +01:00
Henry Heng 19e14c4798 Bugfix/Undefined Flowdata (#2116)
add fix for undefined flowdata
2024-04-07 12:01:57 +01:00
Vinod Kiran 658fa3984e Feature/externalize files from chatflow - do not save as base64 (#1976)
* initial commit. Externalizing the file base64 string from flowData

* csv - docloader - Externalizing the file base64 string from flowData

* csv - docloader - Externalizing the file base64 string from flowData

* DocX - docloader - Externalizing the file base64 string from flowData

* Json - docloader - Externalizing the file base64 string from flowData

* Jsonlines - docloader - Externalizing the file base64 string from flowData

* PDF - docloader - Externalizing the file base64 string from flowData

* Vectara - vector store - Externalizing the file base64 string from flowData

* OpenAPIToolkit - tools - Externalizing the file base64 string from flowData

* OpenAPIChain - chain - Externalizing the file base64 string from flowData

* lint fixes

* datasource enabled - initial commit

* CSVAgent - agents - Externalizing the file base64 string from flowData

* Externalizing the file base64 string from flowData

* Externalizing the file base64 string from flowData

* add pnpm-lock.yaml

* update filerepository to add try catch

* Rename FileRepository.ts to fileRepository.ts

---------

Co-authored-by: Henry <hzj94@hotmail.com>
Co-authored-by: Henry Heng <henryheng@flowiseai.com>
2024-04-04 21:41:06 +05:30
falkor 87e30399d4 fix missing comma upon sql migration query (#2095)
* fix missing comma upon sql migration query

* fix missing comma
2024-04-03 23:46:10 +01:00
Henry Heng e422ce287b Feature/Indexing (#1802)
* indexing

* fix for multiple files upsert

* fix default Postgres port

* fix SQLite node description

* add MySQLRecordManager node

* fix MySQL unique index

* add upsert history

* update jsx ui

* lint-fix

* update dialog details

* update llamaindex pinecone

---------

Co-authored-by: chungyau97 <chungyau97@gmail.com>
2024-04-02 23:47:19 +01:00
Octavian FlowiseAI 957694a912 feature: modularized express routes for reusability, testability, composability and performance (#2030)
* transition GET /api/v1/apikey

* transition POST /api/v1/apikey

* transition PUT /api/v1/apikey/:id

* transition DELETE /api/v1/apikey/:id

* Enable e2e tests for api/v1/apikey routes

* remove unused addChatflowsCount

* Enable e2e tests for api/v1/variables routes

* Enable Cypress in GitHub Action

* Update main.yml

* Update main.yml

* Transition GET /api/v1/variables

* Enable cypress on github workflow

* Transition POST /api/v1/variables

* Transition PUT /api/v1/variables

* Transition DELETE /api/v1/variables

* Transition GET /api/v1/variables

* Transition GET /api/v1/chatflows

* Transition GET /api/v1/chatflows/:id

* Transition POST /api/v1/chatflows

* Transition DELETE /api/v1/chatflows/:id

* Transition PUT /api/v1/chatflows/:id

* Transition GET /api/v1/chatflows/apikey/:apiKey

* Transition GET /api/v1/credentials

* Transition POST /api/v1/credentials

* Transition GET /api/v1/credentials/:id

* Transition PUT /api/v1/credentials/:id

* Transition DELETE /api/v1/credentials/:id

* Transition GET /api/v1/tools

* Transition GET /api/v1/tools/:id

* Transition POST /api/v1/tools

* Transition PUT & DELETE /api/v1/tools/:id

* Transition /api/v1/assistants routes

* Transition /api/v1/nodes routes

* Transition GET /api/v1/chatflows-streaming/:id & GET /api/v1/chatflows-uploads/:id

* wip-all-routes

* Transition GET /api/v1/public-chatflows/:id & /api/v1/public-chatbotConfig/:id

* Remove ts-ignore annotations

* Transition GET /api/v1/chatmessage/:id

* Transition POST /api/v1/chatmessage/:id

* delete /api/v1/chatmessage/:id

* transition /api/v1/feedback/:id routes

* transition /api/v1/stats/:id

* Transition GET /api/v1/openai-assistants/:id

* Transition GET /api/v1/openai-assistants

* Transition POST /api/v1/openai-assistants-file

* transition GET /api/v1/get-upload-path

* transition GET /api/v1/get-upload-file

* transition GET /api/v1/flow-config/:id

* transition POST /api/v1/node-config

* transition GET /api/v1/version

* transition GET /api/v1/fetch-links

* transition POST /api/v1/vector/upsert/:id

* transition POST /api/v1/vector/internal-upsert/:id

* transition POST /api/v1/load-prompt

* Update index.ts

* transition POST /api/v1/prompts-list

* transition predictions

* Update index.ts

* transition GET /api/v1/marketplaces/templates

* Router update modularity cleanup

* extend request interface - express namespace

* Update index.ts

* add errorMiddleware

* Add custom application error handler

* Fix pnpm lock file

* prediction return and vector upsert

* Move the getUploadsConfig into its own file

* Remove lint warnings

* fix undefined variable value

* Fix node-load-method api call

* standardize the error message display

* Apply review comment bugfixes

* Update index.ts

* standardize error message display  in snack notifications

* Error message standard in the UI

* Rename flowXpressApp to appServer

* Upload middleware fix and axios update

* fix async await

---------

Co-authored-by: Henry <hzj94@hotmail.com>
2024-04-02 16:44:04 +01:00
Henry Heng 7e84049990 Bugfix/Postgres migration column type (#2026)
fix postgres migration column type
2024-03-24 19:56:07 +08:00
Emrah TOY 813f622f6d Fix #2000 (#2021)
* ChatId andSessionId returned back to varchar type.

* Will fix problems for the users whom already migrated to 1.6.2

---------

Co-authored-by: Emrah TOY <emrah.toy@mtekbilisim.com>
2024-03-23 15:11:43 +08:00
Vinod Paidimarry 0c0308b9c4 BugFix/Spelling correction in telemetry payload (#2001)
spelling correction
2024-03-21 01:51:50 +08:00