exploratory_charters.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Exploratory Testing Charters</h3>\n",
    "<h4>Charter Creation Prompt</h4>"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "You are a software testing expert testing a software application that classifies wines by their chemical \n",
    "properties. The application is a flask web application that uses HTML for the UI and MySQL to maintain user \n",
    "login and profile information. The trained model is stored in pickle format. The application uses session tokens \n",
    "to manage sessions and encryption to protect user profile information and logging data. Based on the user \n",
    "stories delimited by hash marks and the existing risks delimited by ampersands, create a exploratory testing \n",
    "charter around error handling.\n",
    "\n",
    "###\n",
    "As a logged-in user, I want to be able to log out from my\n",
    "account so that I cannot classify wines until I log in again\n",
    "- Given I am logged into the system, when I select the Logout item from the top menu, then I am logged out and \n",
    "redirected to the login page\n",
    "\n",
    "As a logged-out user, I should not be able to access the logout functionality\n",
    "- Given I am logged out of the application, when I go to any application pages, then I cannot log out again\n",
    "\n",
    "As a logged-out user, I should not be able to access any of the pages that only a logged-in user can access\n",
    "- Given I am logged out of the application, when I try to access either the wine classification or profile pages, \n",
    "then, I am redirected to the Login page.\n",
    "###\n",
    "\n",
    "&&&\n",
    "Logout Functionality (Security): Session token may not be invalidated properly upon logout, allowing unauthorized access. &\n",
    "Logout Functionality (Privacy): User data might be cached or stored insecurely after logout, posing a privacy risk. &\n",
    "Logout Functionality (Performance): Logout process could be slow, impacting user experience. &\n",
    "Logout Access (Security): Logout button/link might be accessible to logged-out users, leading to unexpected behavior. &\n",
    "Protected Pages Access (Security): Logged-out users might be able to bypass authentication and access protected pages. &\n",
    "\n",
    "Protected Pages Access (Privacy): Sensitive user data may be exposed if protected pages are accessible to logged-out users. &\n",
    "Protected Pages Access (Performance): Checking authentication status on protected pages could impact page load times.\n",
    "&&&"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>Exploratory Testing Charter Prompt Response</h4>"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "Exploratory Testing Charter: Error Handling in Wine Classification Application\n",
    "Focus:  This charter focuses on exploring the application's error handling mechanisms, particularly in relation to \n",
    "login/logout functionality and access to protected pages.  We will investigate how the application handles \n",
    "various error scenarios, ensuring robustness, security, and a positive user experience.\n",
    "\n",
    "Scope: This testing effort will cover the following areas:\n",
    "\n",
    "* Login/Logout Errors: Handling of invalid credentials, network issues during login/logout, session management issues, \n",
    "and unexpected server responses.\n",
    "\n",
    "* Protected Page Access Errors: Handling of unauthorized access attempts to protected pages, including attempts by \n",
    "logged-out users, users with insufficient privileges, and users with expired sessions.\n",
    "\n",
    "* General Application Errors: Handling of unexpected errors within the application, such as database connection \n",
    "issues, model loading failures, and file processing errors.\n",
    "\n",
    "Test Ideas & Heuristics:\n",
    "\n",
    "Login/Logout:\n",
    "* Invalid Credentials: Try various invalid username/password combinations (empty, special characters, incorrect length) and verify appropriate error messages are displayed without revealing sensitive information. Explore brute-force attack resistance (account lockout, rate limiting).\n",
    "* Network Issues: Simulate network interruptions during login/logout. Verify the application gracefully handles these situations and displays informative messages to the user.\n",
    "* Session Management: Test session timeout scenarios. Verify that sessions are invalidated correctly and the user is redirected to the login page. Test concurrent logins from different browsers/devices.\n",
    "* Logout Access: Verify that the logout functionality is truly inaccessible to logged-out users. Attempt to access it directly via URL manipulation.\n",
    "* Error Message Clarity: Ensure error messages are user-friendly, informative, and avoid technical jargon. They should guide the user towards a resolution.\n",
    "* Security: Verify that sensitive information (e.g., database connection strings, internal error details) is not exposed in error messages.\n",
    "\n",
    "Protected Page Access:\n",
    "* Unauthorized Access: Attempt to access protected pages while logged out. Verify redirects to the login page and appropriate error handling.\n",
    "* Expired Sessions: Simulate expired sessions and attempt to access protected pages. Verify proper redirection and error handling.\n",
    "* Insufficient Privileges (if applicable): If the application has different user roles, test access to restricted pages with users having insufficient privileges.\n",
    "* Direct URL Manipulation: Attempt to bypass authentication by directly manipulating URLs. Verify that the application correctly validates access.\n",
    "* Error Message Security: Ensure error messages do not reveal information about the application's internal workings or vulnerabilities.\n",
    "\n",
    "General Application Errors:\n",
    "* Database Connection Issues: Simulate database connection failures. Verify the application handles these gracefully and displays appropriate error messages.\n",
    "* Model Loading Failures: Simulate failures in loading the trained model (pickle file). Verify appropriate error handling and user notification.\n",
    "* File Processing Errors: If the application handles file uploads or processing, test with corrupted or invalid files. Verify appropriate error handling and user feedback.\n",
    "* Unhandled Exceptions: Attempt to trigger unexpected errors within the application. Verify the application does not crash and displays a generic error message without revealing sensitive information. Check logs for detailed error information.\n",
    "\n",
    "Tools & Techniques:\n",
    "* Browser developer tools (for inspecting network requests and responses)\n",
    "* Network simulation tools (for simulating network interruptions)\n",
    "* Postman or similar tools (for sending HTTP requests directly)\n",
    "* SQL client (for interacting with the database)\n",
    "* Log analysis tools (for reviewing application logs)\n",
    "* Timebox:  This exploratory testing session will be timeboxed to [Insert Timebox Duration, e.g., 2 hours].\n",
    "\n",
    "Deliverables:\n",
    "* A list of identified issues, including bug reports with detailed steps to reproduce.\n",
    "* A summary of the application's error handling behavior, highlighting strengths and weaknesses.\n",
    "* Recommendations for improving error handling and security."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>Additional Charter Ideas Prompt</h4>"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "You are a software testing expert testing a software application that classifies wines by their chemical \n",
    "properties. The application is a flask web application that uses HTML for the UI and MySQL to maintain user \n",
    "login and profile information. The trained model is stored in pickle format. The application uses session tokens \n",
    "to manage sessions and encryption to protect user profile information and logging data. Based on the exploratory \n",
    "testing charter above, use the SFDPOP mnemonic to create additional testing ideas.  SFDPOP stands for:  Structure, \n",
    "Function, Data, Platform, Operations, Performance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>Additional Charter Ideas Prompt Response</h4>"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "SFDPOP Testing Ideas based on the Error Handling Charter\n",
    "Here are some additional testing ideas based on the SFDPOP mnemonic, expanding on the error handling charter:\n",
    "\n",
    "Structure (Architecture & Design):\n",
    "* Error Logging: Explore the structure of the application's error logs. Are they comprehensive? Do they include timestamps, user IDs, error messages, and stack traces (without revealing sensitive information)? Test log rotation and archiving mechanisms. Are logs secured appropriately?\n",
    "* Modular Design: How is the error handling implemented? Is it centralized or distributed? Test error handling in different modules (e.g., login, classification, profile management) to ensure consistency and robustness. Test error propagation and how errors are handled across different layers of the application.\n",
    "* Security: How are error messages constructed? Are they parameterized to prevent injection attacks? Are error codes used? How is sensitive data (e.g. database connection strings) protected in the code and during error conditions?\n",
    "\n",
    "Function (Features & Capabilities):\n",
    "* User Feedback: Focus on the user interface. Are error messages displayed clearly and concisely? Are they helpful and actionable? Do they provide enough information without being overwhelming or revealing technical details? Test different types of error messages (e.g., validation errors, system errors, network errors).\n",
    "* Error Recovery: Test the application's ability to recover from errors. Can the user retry the operation after an error? Does the application maintain its state? Test error boundaries—does an error in one part of the application affect other parts?\n",
    "* Specific Error Scenarios: Define specific error scenarios for each function. For example, for the classification function: invalid input data, missing input parameters, model loading failure, database connection error during data retrieval.\n",
    "\n",
    "Data (Input & Output):\n",
    "* Boundary Conditions: Test error handling with boundary conditions for input data. For example, very large or very small input values, special characters, empty inputs.\n",
    "* Data Validation: Test error handling related to data validation. Are validation errors handled gracefully? Are appropriate error messages displayed to the user? Test client-side and server-side validation.\n",
    "* Data Corruption: Simulate data corruption in the database or the pickle file. Test how the application handles these situations.\n",
    "\n",
    "Platform (Environment & Infrastructure):\n",
    "* Operating System: Test error handling on different operating systems (e.g., Windows, macOS, Linux). Are there any platform-specific error handling issues?\n",
    "* Browser Compatibility: Test error handling in different browsers (e.g., Chrome, Firefox, Safari). Are error messages displayed consistently across browsers?\n",
    "* Database: Test database-specific error scenarios (e.g., connection failures, query errors, deadlock conditions). How does the application handle these errors?\n",
    "* Flask Framework: Test error handling related to the Flask framework itself. How does the application handle exceptions raised by Flask components?\n",
    "\n",
    "Operations (Deployment & Maintenance):\n",
    "* Deployment Errors: Simulate deployment errors (e.g., missing dependencies, incorrect configuration). How does the application handle these errors during startup?\n",
    "* Maintenance: Test error handling during maintenance tasks (e.g., database backups, software updates). Does the application gracefully handle these situations?\n",
    "* Monitoring: Are errors logged and monitored effectively? Are alerts generated for critical errors? Test the application's ability to report errors to administrators.\n",
    "\n",
    "Performance (Speed & Resource Usage):\n",
    "* Error Handling Overhead: Measure the performance overhead of error handling. Does error handling significantly impact response times?\n",
    "* Resource Leaks: Test for resource leaks during error conditions. Are memory, CPU, or database connections released properly after an error?\n",
    "* Stress Testing: Perform stress testing and simulate a large number of errors. Does the application handle this gracefully? Does it remain stable?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>Freestyle UI Exploratory Testing Prompt</h4>"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "You are a software testing expert testing a software application that classifies wines by their chemical \n",
    "properties. The application is a flask web application that uses HTML for the UI and MySQL to maintain user \n",
    "login and profile information. The trained model is stored in pickle format. The application uses session tokens \n",
    "to manage sessions and encryption to protect user profile information and logging data. Please analyze the \n",
    "attached image of the login screen and identify any bugs, issues, or important things missing from a functionality,\n",
    "usability, or security perspective.\n",
    "\n",
    "[Attached UI image]\n"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}