- Introduced a new crawling management feature allowing users to configure, execute, and log web crawls.
- Added CRUD operations for crawl configurations, including URL analysis and preview capabilities.
- Implemented a new service for handling crawling logic and scheduling tasks.
- Integrated cheerio for HTML parsing and axios for HTTP requests.
- Created a sample HTML page for testing crawling functionality.
This commit enhances the application's data collection capabilities from external websites.
- Replaced existing toast error messages with the new `showErrorToast` utility across multiple components, improving consistency in error reporting.
- Updated error messages to provide more specific guidance for users, enhancing the overall user experience during error scenarios.
- Ensured that all relevant error handling in batch management, external call configurations, cascading management, and screen management components now utilizes the new utility for better maintainability.
- Introduced new endpoints in FlowController for listing procedures and retrieving procedure parameters, enhancing the flow management capabilities.
- Updated FlowDataMoveService to support procedure calls during data movement, ensuring seamless integration with external and internal databases.
- Enhanced NodeFlowExecutionService to execute procedure call actions, allowing for dynamic execution of stored procedures within flow nodes.
- Updated frontend components to support procedure selection and parameter management, improving user experience in configuring flow steps.
- Added necessary types and API functions for handling procedure-related data, ensuring type safety and clarity in implementation.
kjs
DDD1542