Refactoring R2 Bucket Manager: Splitting Large Files

by SLV Team 53 views
Refactoring R2 Bucket Manager: Splitting Large Files

Hey guys! We're diving into a pretty big refactor of the R2 Bucket Manager. The goal? To split some hefty files into smaller, more manageable modules. This is all about making the codebase easier to maintain and contribute to without sacrificing performance or security. Think of it as giving our codebase a serious organizational makeover!

Overview

This refactoring process involves breaking down two large files into smaller, more focused modules. We're talking about a methodical approach here, guys. We'll be using safe editing practices and conducting thorough testing at each stage. No cowboy coding here!

Important Notes

  • Token Consumption: Heads up! This process is expected to consume a good chunk of tokens. But, hey, correctness is our top priority, right?
  • No Shortcuts: We're not cutting any corners, guys. No placeholder code or skipped steps. We're talking full implementations only.
  • Safe Methods: We're sticking to proper file reading and careful editing throughout this process. Safety first!
  • One File at a Time: We'll be fully completing each module before moving on to the next. One step at a time!
  • Local Testing: All changes will be tested locally before we even think about pushing them to GitHub. Gotta make sure everything works!
  • Feature Branch: All the magic will happen in a feature branch, not the main branch. Keeping things nice and tidy.

Phase 1: Setup and Preparation

1.1 Creating a Feature Branch

First things first, let’s get our workspace set up. We'll create and checkout a feature branch named refactor/split-large-files. This keeps our work isolated and organized. We'll also make sure we're starting with a clean working directory – no stray changes hanging around. To keep track of our progress, we’ll document the current line counts as a baseline. This helps us measure how much we've refactored.

Think of this as setting the stage for a big performance – we need everything in place before the show begins!

1.2 Structuring the New Directory

Next up, we're creating a new directory structure to house our refactored code. This is like building the framework for a well-organized house. Here are the directories we're adding:

  • worker/routes/: This is where we'll put route handlers for our API endpoints. Think of it as the traffic control center for our worker.
  • worker/utils/: This directory will hold utility functions, such as those for authentication, signing, and other helper functions. It’s our toolbox for common tasks.
  • worker/types/: We’ll store TypeScript type definitions for our worker here. This helps us keep our code type-safe and less prone to errors.
  • src/hooks/: We'll add more custom React hooks to this existing directory. Hooks help us manage stateful logic in our components.
  • src/components/filegrid/: This new directory will house components specific to the FileGrid feature. It's like creating a dedicated space for a particular set of tools.

Phase 2: Worker Refactoring (worker/index.ts - 2067 lines)

Alright, guys, let's get into the thick of it! We're tackling the worker/index.ts file, which is currently sitting at a whopping 2067 lines. Our mission is to break this behemoth down into smaller, more manageable pieces. Think of it as defusing a bomb – carefully disconnecting each wire and component.

2.1 Extracting Type Definitions

Our first target is to extract all those interfaces and types from worker/index.ts and move them into a new file: worker/types/index.ts. This includes Env, R2-related types, and D1-related types. It's like creating a central registry for all our data structures. We need to ensure that all global type declarations are preserved, so nothing breaks in the process.

2.2 Pulling out Authentication Utilities

Next, we're extracting the validateAccessJWT() function, along with all its JWT validation logic, and placing it into worker/utils/auth.ts. This is our security checkpoint. By isolating the authentication logic, we make it easier to maintain and reason about. We'll export the function with proper typing, of course.

2.3 Isolating URL Signing Utilities

We're going to extract the generateSignature() and validateSignature() functions, which handle HMAC-SHA256 signing logic, and move them into worker/utils/signing.ts. This is like creating a vault for our digital signatures. Keeping this logic separate helps us manage how we secure our URLs.

2.4 Gathering Helper Utilities

Time to extract the getBucketSize() function and any other common utility functions into worker/utils/helpers.ts. This is our general-purpose toolbox. We'll also make sure to include proper error handling, so we know when things go sideways.

2.5 Managing CORS Configuration

We're going to extract the CORS headers configuration and create a function to generate CORS headers in worker/utils/cors.ts. This is about making sure our worker plays nicely with others. We'll also handle OPTIONS preflight requests, so browsers know what's up.

2.6 Carving Out Bucket Routes

Now, we’re extracting all bucket-related route handlers into worker/routes/buckets.ts. This includes handling GET, POST, DELETE, and PATCH requests for buckets. Think of this as setting up the specific routes for bucket operations. We'll maintain all authentication and authorization checks and export everything as a route handler function.

2.7 Detangling File Routes

Next, we're extracting all file-related route handlers into worker/routes/files.ts. This covers GET, POST for uploads, GET for signed URLs, DELETE, POST for copy/move, and PATCH for renaming files. We'll also handle ZIP downloads here. It’s like creating a dedicated highway for file operations. Again, we'll export it as a route handler function.

2.8 Separating Folder Routes

We're extracting all folder-related route handlers into worker/routes/folders.ts. This includes POST for creating, PATCH for renaming, POST for copy/move, and DELETE. We'll maintain the folder operation logic and export it as a route handler function. This keeps our folder operations neatly organized.

2.9 Handling Static Assets

Let's extract the static asset serving logic into worker/utils/assets.ts. This includes generating the site.webmanifest and handling favicon and manifest.json requests. This is like setting up the storefront for our worker, ensuring all static assets are served correctly.

2.10 Refactoring the Main Worker File

Now, the grand finale for this phase: refactoring the main worker/index.ts file. We'll import all the modules we've extracted and create router logic to dispatch requests to the appropriate handlers. The goal? Keep this file under 300 lines. We'll maintain the request flow: CORS → Auth → Routes → Error handling and update our exports. This is about bringing all the pieces together in a clean, organized way.

2.11 Testing the Worker Checkpoint

Before we move on, we need to make sure everything's still working. We'll verify that all worker imports resolve correctly and check TypeScript compilation using npm run build. Then, we'll run our local worker using npx wrangler dev and manually test each API endpoint. We'll verify that authentication still works and confirm that there are no regressions. Think of this as a comprehensive health check after a major surgery.

Phase 3: Frontend Refactoring (src/filegrid.tsx - 2414 lines)

Alright, guys, let’s shift our focus to the frontend! We're tackling src/filegrid.tsx, another biggie at 2414 lines. Just like with the worker, our goal is to break this down into smaller, more manageable components and hooks. This is all about making our frontend code more maintainable and easier to work with.

3.1 Extracting Utility Functions

Our first step is to extract utility functions like formatFileSize(), getFileExtension(), isImageFile(), and isVideoFile() into src/utils/fileUtils.ts. This is like gathering all our small tools into one handy toolbox. We'll add proper TypeScript types to make sure everything is type-safe.

3.2 Icon Components

We're going to extract getFileTypeIcon() as a component, extract getFolderIcon() as a component, create icon mapping logic, and export them as reusable components in src/components/filegrid/FileTypeIcon.tsx. Icons are crucial for UI, so organizing them makes a huge difference.

3.3 Video Player Component

Let's extract the VideoPlayer component with full functionality into src/components/filegrid/VideoPlayer.tsx. This includes play/pause controls and maintaining all event handlers. We'll add a TypeScript props interface to keep things clear and type-safe. This is about giving our video player its own dedicated space.

3.4 Modals

We're extracting each modal as a separate component: CreateFolderModal.tsx, TransferModal.tsx, and RenameModal.tsx. This helps keep our UI logic modular and testable. Each modal will include its validation logic and maintain proper state management. Exporting them with TypeScript props ensures clear communication between components.

3.5 Breadcrumb

We're extracting the breadcrumb navigation logic into src/components/filegrid/Breadcrumb.tsx. This includes path calculation and maintaining click handlers. Breadcrumbs are important for navigation, so giving them a dedicated component is key.

3.6 Context Menu

Time to extract the context menu logic into src/components/filegrid/ContextMenu.tsx. This includes positioning calculations and maintaining all menu items. We'll handle clicks and keyboard events to make our context menu fully functional.

3.7 File and Folder Items

We’re extracting file and folder rendering logic into src/components/filegrid/FileItem.tsx and src/components/filegrid/FolderItem.tsx. This includes selection handling and maintaining preview logic. This helps us manage how files and folders are displayed in our grid.

3.8 Dropdowns

We're extracting each dropdown as a separate component: SortDropdown.tsx, TransferDropdown.tsx, and BucketDropdown.tsx. This includes positioning logic and maintaining click-outside handling. Dropdowns are crucial for UI interactions, so we want them well-organized.

3.9 Action Bar

We're extracting file action buttons and controls into src/components/filegrid/ActionBar.tsx. This includes all button logic (download, delete, transfer, etc.) and maintaining state-dependent rendering. An organized action bar makes file management a breeze.

3.10 Custom Hooks

Time to get hooked! We're creating several custom hooks to manage state and operations:

  • src/hooks/useFileSelection.ts: This hook extracts file/folder selection logic, including multi-select, shift-click, and ctrl-click. It also maintains the lastSelected reference.
  • src/hooks/useFileOperations.ts: This hook extracts file operations like upload, download, delete, copy, move, and rename. It includes all API calls and maintains progress tracking.
  • src/hooks/useFolderOperations.ts: This hook extracts folder operations, including create, rename, delete, copy, and move. It also includes all API calls.
  • src/hooks/usePagination.ts: This hook extracts pagination and file loading logic, including intersection observer setup and maintaining cursor-based pagination.
  • src/hooks/useFileSort.ts: This hook extracts sorting logic, including all sort fields (name, size, type, uploaded) and maintaining the sort direction state.
  • src/hooks/useContextMenu.ts: This hook extracts context menu state management, including positioning and show/hide logic.

3.11 Refactoring the Main FileGrid Component

Now, we're refactoring the main src/filegrid.tsx component. We'll import all the components and hooks we've extracted and compose them together. Our goal is to keep this file under 400 lines while maintaining all functionality. This is where we bring everything together in a cohesive and organized manner.

3.12 Frontend Testing Checkpoint

Before moving on, we need to make sure the frontend is still working flawlessly. We'll verify that all imports resolve correctly and check TypeScript compilation using npm run build. We'll run the dev server using npm run dev and test all FileGrid functionality. This includes file uploads, selections, sorting, filtering, context menus, modals, pagination, breadcrumb navigation, view mode toggling, video playback, image previews, and all file/folder operations. It’s a full suite of tests to ensure everything's in tip-top shape.

Phase 4: Final Integration and Testing

Alright, guys, we're in the home stretch! This phase is all about making sure our refactored worker and frontend play nicely together. We're going to run comprehensive tests to catch any integration issues. Think of this as the final dress rehearsal before the big show.

4.1 Full Application Test

We'll start by building both the frontend and worker using npm run build. Then, we'll deploy to our local worker using npx wrangler dev and run end-to-end tests. This includes testing bucket management, file uploads, file/folder operations, authentication flows, filtering, sorting, download operations, and transfer operations. We'll also conduct performance testing to ensure there's no degradation and check bundle size changes. Finally, we'll perform security verification to ensure all auth checks and JWT validation are intact.

4.2 Code Quality Review

We’ll run the linter using npm run lint and fix any linting issues. We'll also verify that all TypeScript types are correct and remove any console.log statements. Ensuring consistent error handling is also on our checklist. This is like polishing our code to a mirror finish.

4.3 Documentation Updates

Time to update our documentation. We'll update the README.md with the new file structure, add JSDoc comments to major functions, and document new component props and hook usage patterns. Good documentation is like a treasure map for future developers.

4.4 Final Line Count Comparison

We'll document the final line counts for worker/index.ts (target < 300 lines) and src/filegrid.tsx (target < 400 lines). We'll also compare the total lines across all files to the original count. Finally, we'll create a summary of what was extracted where. This gives us a clear picture of how much we've accomplished.

Phase 5: Git and Completion

5.1 Commit Strategy

We'll create logical commits for each major extraction, use descriptive commit messages, and keep commits focused on a single responsibility. Think of it as creating a clear and easy-to-follow history of our changes.

5.2 Final Verification

We'll run a full build one more time and test locally with the worker and frontend. We'll also verify that our feature branch is clean and document any breaking changes (though there shouldn't be any). This is our final sanity check before we declare victory.

5.3 Ready for Review

Finally, we'll push our feature branch to GitHub and create a summary of changes for the PR description. We'll note that this refactor is ready for testing and emphasize DO NOT merge to main until explicitly instructed. This keeps our main branch safe and sound.

Success Criteria

  1. worker/index.ts reduced from 2067 lines to < 300 lines
  2. src/filegrid.tsx reduced from 2414 lines to < 400 lines
  3. All functionality preserved - zero regressions
  4. All tests passing locally
  5. TypeScript compilation clean with no errors
  6. Linter passing with no warnings
  7. Code is more maintainable and easier to understand
  8. New developers can more easily contribute to specific features
  9. Changes contained in feature branch, not pushed to main

Tools and Methods

  • PowerShell: For any necessary file operations
  • Careful Reading: Of full files before editing
  • Methodical Extraction: One module at a time
  • Immediate Testing: After each extraction
  • Full Context: Maintained throughout (no skipping sections)
  • Complete Implementations: (No TODOs or placeholders)

And that's the plan, guys! Let's get this codebase cleaned up and ready for the future!