chore: Enhance Locust load testing and optimize database settings #6265
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Refactor locustfile to provide more robust and configurable load testing: - Add dynamic configuration via environment variables - Improve error handling and logging - Implement realistic flow run simulation - Add connection and timeout handling - Support API key authentication - Enhance stats tracking and error reporting
- Add `no_autoflush` context to prevent unnecessary database operations - Change transaction logging error from exception to error level logging - Simplify error handling in log_transaction function
Update project dependencies by adding Locust (version 2.32.9) to the development requirements, supporting load testing capabilities
…and scalability - Increase default pool_size from 10 to 20 for better connection handling - Adjust max_overflow to 40 to support higher concurrent connections - Extend db_connect_timeout from 20 to 30 seconds - Add pool_recycle and echo settings to db_connection_settings - Enhance documentation for database connection settings, highlighting SQLite limitations
- Introduce comprehensive Locust load testing target with configurable parameters - Support flexible testing scenarios with customizable users, spawn rate, and host - Enable headless and interactive testing modes - Add environment variable support for API key, flow ID, and other testing parameters - Provide sensible default values for load testing configuration
- Remove RETRY_DELAY and MAX_RETRIES environment variables - Simplify FlowRunUser configuration by eliminating unused retry settings - Maintain existing wait time configuration for load testing
- Add mandatory validation for FLOW_ID environment variable - Raise a clear ValueError if FLOW_ID is not provided - Remove default flow ID to ensure explicit configuration - Improve load testing configuration robustness
- Introduce `locust_request_timeout` parameter in Makefile - Update locustfile to use configurable request timeout from environment variable - Set dynamic connection and network timeout based on REQUEST_TIMEOUT - Improve request handling with flexible timeout configuration
dosubot
bot
added
the
size:L
jordanrfrazier
approved these changes
Feb 11, 2025
| @@ -71,16 +71,15 @@ class Settings(BaseSettings): | |||
| """If True, Langflow will run in development mode.""" | |||
| database_url: str | None = None | |||
| """Database URL for Langflow. If not provided, Langflow will use a SQLite database.""" | |||
| database_connection_retry: bool = False | |||
| database_connection_retry: bool = True | |||
There was a problem hiding this comment.
this is possibly misleading; I think it only retries the initial connection when creating the schema / startup, not for subsequent requests during normal operation
There was a problem hiding this comment.
You are correct.
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Improve Locust load testing capabilities with dynamic configurations, error handling, and realistic simulations. Optimize database connection settings for better performance and scalability. Enhance transaction logging and add comprehensive load testing support in the Makefile.