Skip to content

Team push/pull: owner-only over rclone, remove WebDAV transfer #1604

Description

@phernandez

Summary

Team-workspace bm cloud push / bm cloud pull currently run over WebDAV (#1262). It is too slow to use. Replace it with the existing rclone path, restricted to the workspace owner. Non-owners get a clear error instead of push/pull.

Why

A user reported (support email, 2026-09-23) on a Team workspace with 209 files / 28 MB:

  • bm cloud pull with nothing to transfer: ~65–70 s
  • clean pull of all files: ~3m53s
  • one pull failed with HTTP 502 - Failed to list cloud project after ~82 s
  • bm project ls --cloud (one flat rclone listing): ~5.4 s

Reproduced against a Team project (680 files, 103 directories) from the US:

files=680 dirs=103 listing_total=125.3s
PROPFIND: n=103 median=0.93s max=5.99s
GET:      n=5   median=1.23s

Causes:

  1. list_project_files (cli/commands/cloud/webdav.py) walks the project one Depth: 1 PROPFIND per directory, serially, before any transfer decision.
  2. Each request costs ~0.8 s server-side (auth, workspace/project resolution, new S3 client, S3 list, plus a HEAD that always misses on directory paths without a trailing slash).
  3. webdav_project_transfer downloads one file at a time.

Network latency from Europe is secondary (~0.1–0.2 s per request).

Why not per-project storage credentials

Tested on dev Tigris with a throwaway bucket and one key per policy variant:

  • Object actions (GetObject/PutObject/DeleteObject) scope correctly to a prefix such as bucket/project-a/*. Access to project-b/ is denied.
  • ListBucket cannot be scoped. A key granted ListBucket only on bucket/project-a/* still listed project-b/ and the bucket root.
  • The AWS s3:prefix condition is rejected: Invalid policy document: unsupported condition: StringLike.

A per-project key would therefore expose every file path (note title) in the workspace to any member. The owner already has access to every project, so tenant-wide keys leak nothing to the owner.

Proposed change

  • In _run_directional_transfer (cli/commands/cloud/project_sync.py), for non-personal workspaces:
    • role == "owner": use the existing rclone path (remote_name_for_workspace, get_mount_info(workspace_id=...), project_diff, project_transfer). bm cloud setup --workspace <slug> already mints the owner's keys.
    • any other role: exit with a message that Team push/pull are owner-only.
  • Delete cli/commands/cloud/webdav_transfer.py and tests/cli/cloud/test_webdav_transfer.py.
  • Reduce cli/commands/cloud/webdav.py to webdav_path (still used by bm cloud upload), and trim tests/cli/cloud/test_webdav_client.py to match.
  • Remove WEBDAV_COMPARE_FAILURE and the compare_failure parameter of _check_plan.
  • Update docstrings and comments that describe the two-transport design (transfer.py, rclone_commands.py push/pull section, push/pull command help).
  • Replace the WebDAV routing tests in tests/cli/cloud/test_project_sync_command.py with: Team owner routes through rclone with the workspace's remote and bucket; Team non-owner exits before touching rclone or credentials.
  • Docs: note that Team push/pull are owner-only.

This reverses the member access added in #1262. That is intentional.

Out of scope / follow-ups

  • A cloud-to-local mirror mode for owners (rclone sync cloud → local only deletes local files), which would answer the user's deletion request. Separate issue if wanted.
  • bm project ls hides rclone's stderr (it showed only exit status 1 for a 403 caused by stale keys); bm cloud status should print the tenant/bucket.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcloudBasic Memory Cloud

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions