Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

refactor gdrive #327

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

refactor gdrive #327

wants to merge 13 commits into from

Conversation

mabels
Copy link
Contributor

@mabels mabels commented Feb 27, 2025

Summary by CodeRabbit

  • New Features
    • Introduced a robust Google Drive integration for managing files—upload, retrieve, delete, and subscribe to updates.
    • Added a simplified connection method for integrating Google Drive with your database using secure authentication.
    • Enhanced package metadata for seamless support across different environments.
  • Chores
    • Updated project dependencies for improved browser testing and automation.
    • Streamlined module export configurations.
  • Tests
    • Added automated tests to ensure reliable file operations with the new Google Drive integration.

@mabels mabels closed this Feb 27, 2025
@mabels mabels deleted the mabels/gdrive branch February 27, 2025 21:26
@mabels mabels restored the mabels/gdrive branch February 27, 2025 21:26
@mabels mabels reopened this Feb 27, 2025
Copy link

coderabbitai bot commented Feb 27, 2025

Walkthrough

This pull request introduces a new Google Drive integration for the project. A dedicated JSON configuration file provides metadata for the @fireproof/drive module. New source files implement a Google Drive gateway class and a connection function along with tests to validate the store protocol registration and database operations. Additionally, the package manifest has been updated with new dependencies and adjusted export definitions, and a new library bundle configuration has been added.

Changes

File(s) Change Summary
package-drive.json New JSON configuration defining module metadata for @fireproof/drive including name, version, entry points for CommonJS/ES module/browser, keywords, dependencies, and TypeScript type definitions.
package.json Added dependencies "@vitest/browser": "^3.0.7" and "webdriverio": "^9.10.1"; removed the exports section previously used for the ucan module.
src/drive/{drive-gateway.ts, drive-gateway.test.ts, index.ts} Introduced Google Drive integration: added GDriveGateway class implementing file CRUD and subscription methods, a connection function that registers the drive protocol, and a test suite to validate store operations.
tsup.config.ts Added new library bundle configuration for @fireproof/drive with entry set to src/drive/index.ts, output directory dist/drive, specified formats (esm, cjs), esbuild plugins for replacements and path resolving, and a TypeScript declaration footer.

Sequence Diagram(s)

sequenceDiagram
    participant Client as Application/User
    participant Connect as "Connect Function (index.ts)"
    participant Gateway as "GDriveGateway"
    participant API as "Google Drive API"
    
    Client->>Connect: call connect(database, auth, url, name)
    Connect->>Gateway: register protocol & instantiate gateway
    Gateway->>API: start(uri) for initialization
    Client->>Gateway: perform operations (put/get/delete)
    Gateway->>API: forward CRUD requests
    API-->>Gateway: return operation results
    Gateway-->>Client: respond with data/status
Loading
✨ Finishing Touches
  • 📝 Generate Docstrings

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (7)
src/drive/index.ts (3)

24-25: Consider improving the default authentication token.

Using "yourtoken" as the default authentication token parameter value might lead to accidental use in production environments if developers forget to provide a real token. Consider using an empty string as the default value to encourage proper token provision.

-  auth = "yourtoken",
+  auth = "",

39-44: Consider adding error handling for connection failures.

The connection establishment code doesn't have any error handling for potential failures when connecting to Google Drive. Consider adding a try/catch block to handle connection failures gracefully.

  return connectionCache.get(urlObj.toString()).once(() => {
    makeKeyBagUrlExtractable(sthis);
-    const connection = connectionFactory(sthis, urlObj);
-    connection.connect_X(blockstore);
-    return connection;
+    try {
+      const connection = connectionFactory(sthis, urlObj);
+      connection.connect_X(blockstore);
+      return connection;
+    } catch (error) {
+      console.error("Failed to connect to Google Drive:", error);
+      throw new Error(`Failed to connect to Google Drive: ${error.message}`);
+    }
  });

21-21: Consider adding API documentation for the connect function.

Adding JSDoc comments for the exported connect function would improve developer experience by providing documentation directly in the IDE.

+/**
+ * Connect a Fireproof database to Google Drive
+ * @param db - The Fireproof database to connect
+ * @param auth - Google Drive authentication token
+ * @param url - Google Drive API URL
+ * @param remoteDbName - Name of the remote database
+ * @returns A connection to Google Drive
+ */
 export const connect: ConnectFunction = (
src/drive/drive-gateway.test.ts (1)

7-19: Consider adding more test cases to improve coverage.

While the basic smoke test is good, consider adding more specific test cases such as:

  1. Testing with invalid authentication tokens
  2. Testing with different database names
  3. Testing error conditions

This would ensure the Google Drive gateway is thoroughly tested.

src/drive/drive-gateway.ts (3)

25-26: Implement or remove the destroy method.
Currently, this method throws an error and remains unimplemented. Either provide a concrete implementation or remove it if destruction logic is not needed.

Would you like me to propose a possible implementation or open an issue to track this TODO?


179-207: Validate exponential backoff logic in subscribe.
This polling-based subscription will keep doubling interval until it hits maxInterval on unchanged data. This is fine for limited use. If you expect high concurrency or real-time updates, consider using webhooks or a streaming solution.


300-306: Remove or clarify commented-out code.
These lines appear unused and may confuse future maintainers. If they are no longer required, remove them; otherwise, include a reason for keeping them commented.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5ac9fc2 and 9cf1757.

📒 Files selected for processing (6)
  • package-drive.json (1 hunks)
  • package.json (2 hunks)
  • src/drive/drive-gateway.test.ts (1 hunks)
  • src/drive/drive-gateway.ts (1 hunks)
  • src/drive/index.ts (1 hunks)
  • tsup.config.ts (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • package-drive.json
🔇 Additional comments (7)
src/drive/index.ts (1)

29-31: LGTM! Good error handling for required database name.

The check for database name presence is important for proper functioning of the Google Drive connection.

src/drive/drive-gateway.test.ts (1)

18-18: LGTM! Good cleanup with unregistering the protocol.

Properly unregistering the protocol after the test is complete is a good practice to avoid side effects in other tests.

package.json (1)

114-114: LGTM! Good addition of browser testing dependencies.

The added dependencies (@vitest/browser and webdriverio) are appropriate for testing browser interactions with Google Drive.

Also verify that there's a corresponding update to package.json for the @fireproof/drive package:

#!/bin/bash
# Check if there's a package-drive.json file for the new @fireproof/drive package
ls -la package-drive.json || echo "package-drive.json not found"

Also applies to: 135-135

tsup.config.ts (1)

154-174: LGTM! Bundle configuration follows existing patterns.

The bundle configuration for @fireproof/drive follows the same pattern as other library bundles in the file, making it consistent with the project's structure.

src/drive/drive-gateway.ts (3)

104-109: Check behavior for 204 or empty response bodies.
Calling await response.json() on DELETE might fail if the API returns a 204 No Content or an empty body. Consider verifying response.ok and handling empty JSON gracefully, for example:

      const response = await fetch(url + fileId, {
        method: "DELETE",
        headers: headers,
      });
-     return await response.json();
+     if (response.ok && response.status !== 204) {
+       return await response.json();
+     }
+     return {};

296-296: Remove sensitive data from log statements.
Line 296 logs the auth token. Obfuscate or do not log tokens by default.
[security]

- return this.logger.Error().Any({ auth, fileName }).Err(err).Msg("Fetch Error").ResultError();
+ return this.logger.Error().Any({ auth: "<redacted>", fileName }).Err(err).Msg("Fetch Error").ResultError();

308-324: No major concerns with protocol registration logic.
Code successfully registers the GDriveGateway under the "gdrive:" protocol. The approach follows best practices by encapsulating default URIs and returning a gateway instance.

});
return await response.json();
} catch (err) {
return this.logger.Error().Url(url).Any("init", auth).Err(err).Msg("Could not delete").ResultError();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Avoid logging auth tokens in error messages.
Line 110 includes Any("init", auth) in the log, which may reveal bearer tokens. Consider obfuscating or removing sensitive information from logs to prevent accidental exposure.

[security]

- return this.logger.Error().Url(url).Any("init", auth).Err(err).Msg("Could not delete").ResultError();
+ return this.logger.Error().Url(url).Any("init", "<redacted>").Err(err).Msg("Could not delete").ResultError();
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
return this.logger.Error().Url(url).Any("init", auth).Err(err).Msg("Could not delete").ResultError();
return this.logger.Error().Url(url).Any("init", "<redacted>").Err(err).Msg("Could not delete").ResultError();

Comment on lines +76 to +78
if (!index) {
name += `-${index}`;
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Likely reversed condition for file indexing.
The code appends -undefined to the filename when no index is given. Instead, you probably want to append the index only if it exists, e.g.:

-if (!index) {
-  name += `-${index}`;
+if (index) {
+  name += `-${index}`;
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (!index) {
name += `-${index}`;
}
if (index) {
name += `-${index}`;
}

Comment on lines +116 to +137
let headers;
const url = BuildURI.from(this.params.driveURL);
headers = {
Authorization: `Bearer ${auth}`,
"Content-Type": "application/json",
};
if (type == "meta") {
response = await fetch(url + fileId, {
method: "GET",
headers,
});
return Result.Ok(new Uint8Array(await response.arrayBuffer()));
} else {
headers = {
Authorization: `Bearer ${auth}`,
};
response = await fetch(url.appendRelative(fileId).setParam("alt", "media").toString(), {
method: "GET",
headers,
});
return Result.Ok(new Uint8Array(await response.arrayBuffer()));
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Return errors if the fetch fails in #get.
There's no check for response.ok before returning Result.Ok(...). A 404 or other non-2xx status could silently fail, returning malformed data. Consider:

if (!response.ok) {
  return this.logger.Error().Any({ status: response.status }).Msg("GET failed").ResultError();
}
return Result.Ok(new Uint8Array(await response.arrayBuffer()));
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
let headers;
const url = BuildURI.from(this.params.driveURL);
headers = {
Authorization: `Bearer ${auth}`,
"Content-Type": "application/json",
};
if (type == "meta") {
response = await fetch(url + fileId, {
method: "GET",
headers,
});
return Result.Ok(new Uint8Array(await response.arrayBuffer()));
} else {
headers = {
Authorization: `Bearer ${auth}`,
};
response = await fetch(url.appendRelative(fileId).setParam("alt", "media").toString(), {
method: "GET",
headers,
});
return Result.Ok(new Uint8Array(await response.arrayBuffer()));
}
let headers;
const url = BuildURI.from(this.params.driveURL);
headers = {
Authorization: `Bearer ${auth}`,
"Content-Type": "application/json",
};
if (type == "meta") {
response = await fetch(url + fileId, {
method: "GET",
headers,
});
if (!response.ok) {
return this.logger.Error().Any({ status: response.status }).Msg("GET failed").ResultError();
}
return Result.Ok(new Uint8Array(await response.arrayBuffer()));
} else {
headers = {
Authorization: `Bearer ${auth}`,
};
response = await fetch(url.appendRelative(fileId).setParam("alt", "media").toString(), {
method: "GET",
headers,
});
if (!response.ok) {
return this.logger.Error().Any({ status: response.status }).Msg("GET failed").ResultError();
}
return Result.Ok(new Uint8Array(await response.arrayBuffer()));
}

@mabels mabels changed the title refactore gdrive refactor gdrive Feb 27, 2025
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants