farcasclaudiu/Flowise
1
0
Fork 0
mirror of https://github.com/farcasclaudiu/Flowise.git synced 2026-06-28 23:01:09 +03:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #575 from atishamte/feature/FlowiseMultiDatabaseSupport

Browse Source 
Feature/flowise multi database support
This commit is contained in:
Henry Heng
2023-07-19 18:57:47 +01:00
committed by GitHub
parent 59a9651d73 7f5acf32aa
commit 0de108c0c2
9 changed files with 149 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+14 -7
View File
@@ -138,18 +138,25 @@ FLOWISE_PASSWORD=1234
Flowise support different environment variables to configure your instance. You can specify the following variables in the `.env` file inside `packages/server` folder. Read [more](https://docs.flowiseai.com/environment-variables) Flowise support different environment variables to configure your instance. You can specify the following variables in the `.env` file inside `packages/server` folder. Read [more](https://docs.flowiseai.com/environment-variables)
| Variable | Description | Type | Default | | Variable | Description | Type | Default |
| -------------------------- | ---------------------------------------------------------------- | ------------------------------------------------ | ----------------------------------- | | ---------------- | ---------------------------------------------------------------- | ------------------------------------------------ | ----------------------------------- |
| PORT | The HTTP port Flowise runs on | Number | 3000 | | PORT | The HTTP port Flowise runs on | Number | 3000 |
| FLOWISE_USERNAME | Username to login | String | | FLOWISE_USERNAME | Username to login | String | |
| FLOWISE_PASSWORD | Password to login | String | | FLOWISE_PASSWORD | Password to login | String | |
| DEBUG | Print logs onto terminal/console | Boolean | | DEBUG | Print logs from components | Boolean | |
| LOG_PATH | Location where log files are stored | String | `your-path/Flowise/packages/server` | | LOG_PATH | Location where log files are stored | String | `your-path/Flowise/logs` |
| LOG_LEVEL | Different log levels for loggers to be saved | Enum String: `error`, `info`, `verbose`, `debug` | `info` | | LOG_LEVEL | Different levels of logs | Enum String: `error`, `info`, `verbose`, `debug` | `info` |
| DATABASE_PATH | Location where database is saved | String | `your-home-dir/.flowise` |
| APIKEY_PATH | Location where api keys are saved | String | `your-path/Flowise/packages/server` | | APIKEY_PATH | Location where api keys are saved | String | `your-path/Flowise/packages/server` |
| EXECUTION_MODE | Whether predictions run in their own process or the main process | Enum String: `child`, `main` | `main` | | EXECUTION_MODE | Whether predictions run in their own process or the main process | Enum String: `child`, `main` | `main` |
| TOOL_FUNCTION_BUILTIN_DEP | NodeJS built-in modules to be used for Tool Function | String | | | TOOL_FUNCTION_BUILTIN_DEP | NodeJS built-in modules to be used for Tool Function | String | |
| TOOL_FUNCTION_EXTERNAL_DEP | External modules to be used for Tool Function | String | | | TOOL_FUNCTION_EXTERNAL_DEP | External modules to be used for Tool Function | String | |
| OVERRIDE_DATABASE | Override current database with default | Enum String: `true`, `false` | `true` |
| DATABASE_TYPE | Type of database to store the flowise data | Enum String: `sqlite`, `mysql`, `postgres` | `sqlite` |
| DATABASE_PATH | Location where database is saved (When DATABASE_TYPE is sqlite) | String | `your-home-dir/.flowise` |
| DATABASE_HOST | Host URL or IP address (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_PORT | Database port (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_USERNAME | Database username (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_PASSWORD | Database password (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_NAME | Database name (When DATABASE_TYPE is not sqlite) | String | |
You can also specify the env variables when using `npx`. For example: You can also specify the env variables when using `npx`. For example:
packages/server/.env.example
+14 -1
View File
@@ -2,10 +2,23 @@ PORT=3000
# FLOWISE_USERNAME=user # FLOWISE_USERNAME=user
# FLOWISE_PASSWORD=1234 # FLOWISE_PASSWORD=1234
# DEBUG=true # DEBUG=true
# DATABASE_PATH=/your_database_path/.flowise
# APIKEY_PATH=/your_api_key_path/.flowise # APIKEY_PATH=/your_api_key_path/.flowise
# LOG_PATH=/your_log_path/.flowise/logs # LOG_PATH=/your_log_path/.flowise/logs
# LOG_LEVEL=debug (error | warn | info | verbose | debug) # LOG_LEVEL=debug (error | warn | info | verbose | debug)
# EXECUTION_MODE=main (child | main) # EXECUTION_MODE=main (child | main)
OVERRIDE_DATABASE="true"
DATABASE_TYPE="sqlite" # sqlite, mysql, postgres
# When database is sqlite
# DATABASE_PATH=/your_database_path/.flowise
# When database is not sqlite
# DATABASE_PORT=""
# DATABASE_HOST=""
# DATABASE_NAME="flowise"
# DATABASE_USER=""
# DATABASE_PASSWORD=""
# TOOL_FUNCTION_BUILTIN_DEP=crypto,fs # TOOL_FUNCTION_BUILTIN_DEP=crypto,fs
# TOOL_FUNCTION_EXTERNAL_DEP=moment,lodash # TOOL_FUNCTION_EXTERNAL_DEP=moment,lodash
packages/server/README.md
+14 -7
View File
@@ -34,18 +34,25 @@ FLOWISE_PASSWORD=1234
Flowise support different environment variables to configure your instance. You can specify the following variables in the `.env` file inside `packages/server` folder. Read [more](https://docs.flowiseai.com/environment-variables) Flowise support different environment variables to configure your instance. You can specify the following variables in the `.env` file inside `packages/server` folder. Read [more](https://docs.flowiseai.com/environment-variables)
| Variable | Description | Type | Default | | Variable | Description | Type | Default |
| -------------------------- | ---------------------------------------------------------------- | ------------------------------------------------ | ----------------------------------- | | ---------------- | ---------------------------------------------------------------- | ------------------------------------------------ | ----------------------------------- |
| PORT | The HTTP port Flowise runs on | Number | 3000 | | PORT | The HTTP port Flowise runs on | Number | 3000 |
| FLOWISE_USERNAME | Username to login | String | | FLOWISE_USERNAME | Username to login | String | |
| FLOWISE_PASSWORD | Password to login | String | | FLOWISE_PASSWORD | Password to login | String | |
| DEBUG | Print logs onto terminal/console | Boolean | | DEBUG | Print logs from components | Boolean | |
| LOG_PATH | Location where log files are stored | String | `your-path/Flowise/packages/server` | | LOG_PATH | Location where log files are stored | String | `your-path/Flowise/logs` |
| LOG_LEVEL | Different log levels for loggers to be saved | Enum String: `error`, `info`, `verbose`, `debug` | `info` | | LOG_LEVEL | Different levels of logs | Enum String: `error`, `info`, `verbose`, `debug` | `info` |
| DATABASE_PATH | Location where database is saved | String | `your-home-dir/.flowise` |
| APIKEY_PATH | Location where api keys are saved | String | `your-path/Flowise/packages/server` | | APIKEY_PATH | Location where api keys are saved | String | `your-path/Flowise/packages/server` |
| EXECUTION_MODE | Whether predictions run in their own process or the main process | Enum String: `child`, `main` | `main` | | EXECUTION_MODE | Whether predictions run in their own process or the main process | Enum String: `child`, `main` | `main` |
| TOOL_FUNCTION_BUILTIN_DEP | NodeJS built-in modules to be used for Tool Function | String | | | TOOL_FUNCTION_BUILTIN_DEP | NodeJS built-in modules to be used for Tool Function | String | |
| TOOL_FUNCTION_EXTERNAL_DEP | External modules to be used for Tool Function | String | | | TOOL_FUNCTION_EXTERNAL_DEP | External modules to be used for Tool Function | String | |
| OVERRIDE_DATABASE | Override current database with default | Enum String: `true`, `false` | `true` |
| DATABASE_TYPE | Type of database to store the flowise data | Enum String: `sqlite`, `mysql`, `postgres` | `sqlite` |
| DATABASE_PATH | Location where database is saved (When DATABASE_TYPE is sqlite) | String | `your-home-dir/.flowise` |
| DATABASE_HOST | Host URL or IP address (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_PORT | Database port (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_USERNAME | Database username (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_PASSWORD | Database password (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_NAME | Database name (When DATABASE_TYPE is not sqlite) | String | |
You can also specify the env variables when using `npx`. For example: You can also specify the env variables when using `npx`. For example:
packages/server/package.json
+2
View File
@@ -55,6 +55,8 @@
"flowise-ui": "*", "flowise-ui": "*",
"moment-timezone": "^0.5.34", "moment-timezone": "^0.5.34",
"multer": "^1.4.5-lts.1", "multer": "^1.4.5-lts.1",
"mysql": "^2.18.1",
"pg": "^8.11.1",
"reflect-metadata": "^0.1.13", "reflect-metadata": "^0.1.13",
"socket.io": "^4.6.1", "socket.io": "^4.6.1",
"sqlite3": "^5.1.6", "sqlite3": "^5.1.6",
packages/server/src/ChildProcess.ts
+52 -8
View File
@@ -142,14 +142,58 @@ export class ChildProcess {
* @returns {DataSource} * @returns {DataSource}
*/ */
async function initDB() { async function initDB() {
const homePath = process.env.DATABASE_PATH ?? path.join(getUserHome(), '.flowise') let childAppDataSource
const childAppDataSource = new DataSource({ let homePath
type: 'sqlite', switch (process.env.DATABASE_TYPE) {
database: path.resolve(homePath, 'database.sqlite'), case 'sqlite':
synchronize: true, homePath = process.env.DATABASE_PATH ?? path.join(getUserHome(), '.flowise')
entities: [ChatFlow, ChatMessage, Tool], childAppDataSource = new DataSource({
migrations: [] type: 'sqlite',
}) database: path.resolve(homePath, 'database.sqlite'),
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
case 'mysql':
childAppDataSource = new DataSource({
type: 'mysql',
host: process.env.DATABASE_HOST,
port: parseInt(process.env.DATABASE_PORT || '3306'),
username: process.env.DATABASE_USER,
password: process.env.DATABASE_PASSWORD,
database: process.env.DATABASE_NAME,
charset: 'utf8mb4',
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
case 'postgres':
childAppDataSource = new DataSource({
type: 'postgres',
host: process.env.DATABASE_HOST,
port: parseInt(process.env.DATABASE_PORT || '5432'),
username: process.env.DATABASE_USER,
password: process.env.DATABASE_PASSWORD,
database: process.env.DATABASE_NAME,
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
default:
homePath = process.env.DATABASE_PATH ?? path.join(getUserHome(), '.flowise')
childAppDataSource = new DataSource({
type: 'sqlite',
database: path.resolve(homePath, 'database.sqlite'),
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
}
return await childAppDataSource.initialize() return await childAppDataSource.initialize()
} }
packages/server/src/DataSource.ts
+50 -9
View File
@@ -9,15 +9,56 @@ import { getUserHome } from './utils'
let appDataSource: DataSource let appDataSource: DataSource
export const init = async (): Promise<void> => { export const init = async (): Promise<void> => {
const homePath = process.env.DATABASE_PATH ?? path.join(getUserHome(), '.flowise') let homePath
switch (process.env.DATABASE_TYPE) {
appDataSource = new DataSource({ case 'sqlite':
type: 'sqlite', homePath = process.env.DATABASE_PATH ?? path.join(getUserHome(), '.flowise')
database: path.resolve(homePath, 'database.sqlite'), appDataSource = new DataSource({
synchronize: true, type: 'sqlite',
entities: [ChatFlow, ChatMessage, Tool], database: path.resolve(homePath, 'database.sqlite'),
migrations: [] synchronize: process.env.OVERRIDE_DATABASE == 'true',
}) entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
case 'mysql':
appDataSource = new DataSource({
type: 'mysql',
host: process.env.DATABASE_HOST,
port: parseInt(process.env.DATABASE_PORT || '3306'),
username: process.env.DATABASE_USER,
password: process.env.DATABASE_PASSWORD,
database: process.env.DATABASE_NAME,
charset: 'utf8mb4',
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
case 'postgres':
appDataSource = new DataSource({
type: 'postgres',
host: process.env.DATABASE_HOST,
port: parseInt(process.env.DATABASE_PORT || '5432'),
username: process.env.DATABASE_USER,
password: process.env.DATABASE_PASSWORD,
database: process.env.DATABASE_NAME,
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
default:
homePath = process.env.DATABASE_PATH ?? path.join(getUserHome(), '.flowise')
appDataSource = new DataSource({
type: 'sqlite',
database: path.resolve(homePath, 'database.sqlite'),
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
}
} }
export function getDataSource(): DataSource { export function getDataSource(): DataSource {
packages/server/src/entity/ChatFlow.ts
+1 -1
View File
@@ -10,7 +10,7 @@ export class ChatFlow implements IChatFlow {
@Column() @Column()
name: string name: string
@Column() @Column({ type: "text" })
flowData: string flowData: string
@Column({ nullable: true }) @Column({ nullable: true })
packages/server/src/entity/ChatMessage.ts
+1 -1
View File
@@ -14,7 +14,7 @@ export class ChatMessage implements IChatMessage {
@Column() @Column()
chatflowid: string chatflowid: string
@Column() @Column({ type: "text" })
content: string content: string
@Column({ nullable: true }) @Column({ nullable: true })
packages/server/src/entity/Tool.ts
+1 -1
View File
@@ -10,7 +10,7 @@ export class Tool implements ITool {
@Column() @Column()
name: string name: string
@Column() @Column({ type: "text" })
description: string description: string
@Column() @Column()