Projects

GitHub Backup Observatory Overview The GitHub Backup Observatory is a compact, multi-service architecture designed to clone, archive, and consolidate GitHub repositories into a singular, unified backup repository. Built for reliability and observability, the system integrates a Go-based CLI worker, a Go and PostgreSQL-powered REST API and WebSocket backend, a Next.js frontend dashboard for real-time monitoring, and an AI-driven Agentic Observatory. Live Resources Live Dashboard: github.mishrashardendu22.is-a.dev Video Demonstration: YouTube Architecture Overview This repository operates as a monorepo containing the following core services: Worker (CLI): The root-level Go application responsible for the core logic. It traverses target GitHub accounts, deduplicates repositories, verifies remote HEAD hashes, executes shallow clones, and archives the repositories to a centralized _Repos Git repository. Backend API (backend/): A Go-based web server integrated with PostgreSQL. It serves operational metrics, historical run data, and real-time logs via REST APIs and WebSockets. Frontend Dashboard (frontend/): A Next.js application providing a comprehensive, real-time user interface for monitoring backup executions, performance metrics, and system logs. Agentic Observatory (agentic-observatory/): A Python-based agentic layer utilizing FastAPI and OpenRouter to autonomously analyze and interact with the backup data. Core Engine: Worker (CLI) The primary backup engine resides in the root directory. It leverages a lightweight SQLite database for local state management and metadata tracking. Operational Phases Phase 1: Hash Verification: Concurrently computes the remote HEAD to determine if repository changes have occurred since the last backup. Phase 2: Clone and Archive: Performs shallow clones of modified repositories, removes .git directories to prevent nested repository issues, and generates tar.gz archives. Phase 3: Commit and Push: Commits the generated archives to the central _Repos Git repository and pushes the updates to the remote origin. Configuration Environment variables must be configured prior to execution. Create a .env file in the root directory based on the provided sample.env: ORGACCOUNT / PROJECTACCOUNT: Target GitHub accounts for the backup process. DB_PATH: Absolute or relative path for the SQLite database file (defaults to ./app.db). BACKUPREPOPATH: The remote Git URL for the centralized _Repos directory. GITHUBTOKENPRIVATE / GITHUBTOKENPERSONAL: Authentication tokens for GitHub API access. Execution Ensure the Go toolchain, git, and tar are installed on the host system. Service Documentation For detailed instructions concerning the deployment, configuration, and development of individual services, refer to their respective documentation: Backend Documentation Frontend Documentation Agentic Observatory Documentation Contributing Contributors are expected to adhere to the established coding conventions. Refer to CONTRIBUTING.md for guidelines on submitting improvements and modifications.

Agentic Google Workspace Orchestrator A secure, multi-service orchestration platform for Gmail, Google Calendar, and Google Drive. This repository combines: FastAPI backend with PostgreSQL + pgvector and Redis Next.js chat and orchestration frontend Hybrid retrieval across full-text search, semantic embeddings, and Reciprocal Rank Fusion (RRF) Typed execution plan DAG generation and Python-controlled action gating Google OAuth sync for Gmail, Calendar, and Drive data What this project does The system can: classify user intent and extract structured entities from natural language execute parallel search across Gmail, Calendar, and Drive generate safe workflow plans using a type-driven LLM + Python validator gate mutating actions behind explicit confirmation persist conversations, orchestration state, and audit records provide an interactive frontend + OpenAPI backend docs Architecture Overview Primary components: app/main.py — FastAPI application entrypoint, CORS, middleware, health endpoints app/api/ — REST routers for auth, sync, search, query, actions, orchestration, and conversations app/orchestration/ — intent classification, planning, validation, execution, and confirmation flow app/retrieval/ — hybrid search implementation using PostgreSQL FTS, pgvector embeddings, and RRF fusion app/models/ — SQLAlchemy model definitions for users, workspace items, orchestration runs, and pending actions frontend/ — Next.js UI for conversation-driven orchestration and action confirmation Local runtime topology Prerequisites Python 3.12+ Node.js 20+ and pnpm Docker Engine + Docker Compose Google Cloud OAuth application credentials OpenRouter API key Quick Start Install Python and frontend dependencies: Copy the example environment variables: Update .env with your credentials: Google OAuth Redirect URI must be configured as: http://localhost:8000/api/v1/auth/google/callback Start the app: Common commands API and user-facing endpoints Frontend UI: http://localhost:3000 Backend API: http://localhost:8000 OpenAPI: http://localhost:8000/docs Health probe: GET /health Readiness probe: GET /ready Detailed API reference is available in API.md. Project documentation API.md — API reference and endpoint examples DEMO.md — demo script and evaluation query guidance DESIGN.md — system architecture, ER diagram, and scaling rationale SAMPLE_QUERIES.md — sample orchestrator prompts and expected behavior RETRIEVAL_EVAL.md — retrieval evaluation methodology and metrics Security and responsible use This project uses OAuth tokens and API keys. Keep credentials private and never commit them to source control. For vulnerability reporting, see SECURITY.md. Contribution guide Contributions are welcome. Please read CONTRIBUTING.md before opening issues or pull requests. License This project is licensed under the MIT License. See LICENSE for details.

Git Commit Metadata Rewriter Overview The Git Commit Metadata Rewriter is a specialized utility engineered in Go to retroactively modify Git commit metadata, including author names, email addresses, and timestamps, for cloned GitHub repositories. The system integrates with an AI model to generate realistic, non-uniform commit distributions, subsequently rewriting the repository's history to reflect the updated timelines. Live Resources Video Demonstration: YouTube Commit Testing Sample: Test Repository Core Capabilities Automated Repository Management: Seamlessly clones public GitHub repositories for localized processing. AI-Driven Timestamp Generation: Leverages the OpenRouter API to programmatically generate realistic commit date distributions. History Modification: Utilizes Git filter-branch operations to systematically rewrite commit history with the newly generated metadata. Automated Synchronization: Executes force-pushes to synchronize the rewritten history with the remote repository origin. State Cleanup: Automatically purges temporary files and local repository clones upon process completion to maintain system hygiene. Operational Workflow Initialization: Clones the specified target GitHub repository to the local environment. Extraction: Exports the existing commit log to edited_commits.txt. Generation: Interfaces with the AI model to establish new, realistic commit dates within a defined timeframe. Data Mapping: Compiles the updated commit metadata into updated_commits.txt. Execution: Rewrites the Git history applying the new metadata parameters. Synchronization: Force-pushes the modified commit history to the remote source. Cleanup: Removes all local artifacts and cloned data. Installation and Configuration Repository Setup Clone the repository and resolve dependencies: API Authentication An OpenRouter API key is required for timestamp generation. Create a .env file in the project root directory: Target Configuration Modify main.go to define the target repository and the desired date range parameters: repo: Set to the target repository identifier (e.g., MishraShardendu22). start / end: Define the boundaries for the newly generated commit dates. Execution AI Integration Details The system relies on the OpenRouter API to produce plausible commit timelines. The underlying prompt is strictly constrained to alter timestamps while preserving original author attributions and commit messages. Failure to provide a valid API key will result in immediate process termination. Troubleshooting Missing API_KEY: Verify the .env file is present and properly formatted. Authentication Failures: Ensure SSH keys are correctly configured with GitHub and that the executing environment holds sufficient push permissions for the target repository. Empty AI Responses: Confirm API key validity and network connectivity. An index out of range panic typically indicates a malformed or empty API response. Protected Branches: Force-push operations will fail on protected branches. Branch protection rules on the remote must be temporarily disabled prior to execution. System Architecture Contributing Review CONTRIBUTING.md for detailed contributor guidelines and procedures. Adherence to the CODEOFCONDUCT.md is strictly enforced. Security For instructions on disclosing security vulnerabilities, please refer to SECURITY.md. License Distributed under the MIT License. See LICENSE for further details. This project was developed to observe GitHub's visualization of commit histories and the systemic effects of rewritten repositories.

DFS based repository cleaninge engine Autonomous Repository Maintenance and Dead Component Elimination Engine. GitHub-Cleaner-Go is a production-grade automation tool that systematically traverses GitHub repositories, performs static import analysis on React codebases, identifies and removes unused UI components, validates builds post-cleanup, and commits the results without human intervention. Built for developers managing large React ecosystems where component bloat accumulates across repositories. The tool functions as an autonomous maintenance agent, reducing technical debt through programmatic dead-code elimination. Latest Check Runtime: 16m 1.22s Concurrency: 10 workers Repositories scanned: 148 Architecture The system operates as a three-stage pipeline: repository orchestration, static analysis and dead-component elimination, and build verification and commit. Repositories are processed concurrently (up to 5 at a time) via a goroutine pool with channel-based rate limiting. Core Features Autonomous Repository Discovery - Fetches all repositories from a GitHub account via the REST API (up to 100 repos per request). Concurrent Processing - Processes up to 5 repositories simultaneously using goroutines with channel-based semaphore limiting. Recursive Filesystem Traversal - Walks directory trees (DFS) to locate React projects with components/ui directory structures. Regex-Based Static Import Analysis - Scans .ts, .tsx, .js, .jsx files for import statements referencing components/ui/* components. Dead Component Elimination - Compares used components against filesystem entries; removes orphaned components. Build Validation - Executes npm install --legacy-peer-deps && npm run build to verify post-cleanup integrity. Prometheus Metrics - Exposes real-time metrics including active workers, repos processed, clone/build durations, files deleted, and failure counters. Structured JSON Logging - All operations logged with slog in JSON format with consistent attribute structure. Git Automation - Automatically commits cleanup changes with standardized commit messages. Ephemeral Repository Lifecycle - Clones, processes, and destroys local repository copies leaving no residual artifacts. Technical Workflow Stage 1: Repository Orchestration GitHub API Discovery - Issues GET /users/{username}/repos?per_page=100 to enumerate all repositories (unauthenticated, 60 req/hr limit). Concurrent Clone Pool - Up to 5 repositories cloned simultaneously via goroutine pool with channel-based capacity control. SSH Clone - Clones each repository via git@github.com:{username}/{repo}.git. Absolute Path Resolution - Uses filepath.Abs to resolve the cloned repository root (no os.Chdir state mutation). Recursive Scanner Invocation - Initiates DeepSearchAndClean() on the repository root. Stage 2: Static Analysis and Cleanup Pipeline Filesystem Enumeration - Lists files and directories at current level via Segregator() (separates files from directories). React Project Detection - Checks for package.json containing both react and react-dom dependencies. UI Directory Discovery - DFS walk via FindUIDir() to locate components/ui directories. Source Graph Analysis - Walks every .ts/.tsx/.js/.jsx file, extracting import paths matching the pattern: [./@"]components/ui/([A-Za-z0-9_-]+) Usage Mapping - Builds a lowercase-normalized set of used component names. Dead Component Elimination - Iterates over components/ui entries, deleting any file whose base name (without extension) has zero import references. Build Verification - Runs the project build to confirm no regressions were introduced; build exit code is captured and logged. Stage 3: Commit and Cleanup Git Commit - Stages and commits all changes with message: auto: cleanup ui and build (requires git cm alias for commit -am). Repository Destruction - Recursively removes the cloned repository via deferred os.RemoveAll on absolute path. Example Execution Flow Repository Traversal The traversal is implemented as a depth-first recursive directory walk (DeepSearchAndClean). At each node: The directory is enumerated for files and subdirectories using Segregator(). If a package.json is present, the node is treated as a potential project root and passed to CleanThis. If no package.json exists, the function recurses into each subdirectory. This design allows the system to handle monorepos, nested projects, and repositories with complex directory structures. The traversal terminates at leaf directories with no further subdirectories or upon discovering a valid React project. Static Import Analysis The static analysis subsystem uses regex-based import scanning rather than full AST parsing. The regular expression: Matches import statements in the following common patterns: | Import Pattern | Example | |--------------------|--------------------------------------| | Relative import | ./components/ui/Button | | Absolute import | @/components/ui/Card | | String import | "components/ui/Modal" | | Named import | components/ui/Button | The analysis builds a usage map (boolean, presence-based) by lowercasing the captured component name. Files are then compared against this map; any file in components/ui whose stem does not appear in the usage set is considered dead and scheduled for deletion. Known Limitation: Dynamic imports using template literals or computed strings are not resolved. The analysis also does not handle re-exports or barrel files. Build Validation Post-cleanup, the system executes: This serves dual purposes: Integrity Check - Verifies that no deleted component was actually required at build time (catches false positives). Dependency Resolution - Ensures the project is in a buildable state after modifications. Build results are logged with duration and status. Build failures are tracked via Prometheus metrics but do not halt the pipeline. Observability Prometheus Metrics Metrics are exposed via an HTTP server on :2112/metrics: | Metric | Type | Description | |-----------------------------------|-----------|-----------------------------------| | reposprocessedtotal | Counter | Total repos processed | | reactrepostotal | Counter | React repos found | | filesdeletedtotal | Counter | Total files deleted | | clonefailurestotal | Counter | Clone failures | | buildfailurestotal | Counter | Build failures | | cleanupfailurestotal | Counter | Cleanup failures | | gitcommitfailures_total | Counter | Git commit failures | | active_workers | Gauge | Current active goroutines | | repoprocessingduration_seconds| Histogram | Per-repo processing time | | clonedurationseconds | Histogram | Clone operation duration | | builddurationseconds | Histogram | Build operation duration | Grafana Dashboard A full monitoring stack is available via Docker Compose: Installation Prerequisites | Dependency | Version | Purpose | |----------------|---------|------------------------------------------------------| | Go | 1.24.4+ | Compilation and runtime | | Git | 2.x+ | Repository cloning and automation | | SSH Agent | Any | Authentication for repository cloning | | npm / Node.js | Any | React project build validation | | Docker (opt.) | Any | Prometheus/Grafana monitoring stack | Build and Run Quick Start with Make Usage Fetches repositories from the configured GitHub account, clones each one, performs dead component analysis, deletes unused files, validates builds, commits changes, and cleans up. Project Structure Technical Limitations Regex-based analysis - Cannot resolve dynamic imports, computed paths, or re-exports. May produce false negatives for obfuscated import patterns. React-only scope - Currently limited to React projects with components/ui structures. No support for Vue, Angular, or other frameworks. SSH-only authentication - Requires configured SSH keys for repository cloning. No HTTPS fallback. Single-user mode - Hardcoded to a single GitHub username. No multi-account or organization support. No dry-run mode - Operations are destructive by design. No preview capability for what would be deleted. Git alias dependency - Requires git cm alias for git commit -am. No API pagination - Only fetches the first 100 repos from the GitHub API. No directory exclusion - Traverses .git, node_modules, and hidden directories. Safety Warnings Destructive Operations This tool deletes files and makes Git commits automatically. It is strongly recommended to: Test on a fork or backup repository first. Review the codebase to understand deletion criteria. Ensure all important work is committed and pushed before running. Use a feature branch if possible (modify the commit step to push to a branch). Contributing See CONTRIBUTING.md for detailed contributor guidelines, including setup instructions, code style expectations, and pull request process. License This project is licensed under the MIT License. See the LICENSE file for details. The MIT License was chosen for this project because: It permits commercial and private use without restrictions. It allows other developers to integrate the cleanup engine into their own tooling. It is the most widely understood and accepted license in the open-source ecosystem. It imposes no copyleft obligations, which is appropriate for an automation tool that may be embedded into CI/CD pipelines. It provides appropriate disclaimer of liability (critical for a tool that performs destructive filesystem operations). Author MishraShardendu22 GitHub: @MishraShardendu22 Project: Repository-cleaning-engine Related Links Go Programming Language GitHub REST API Documentation React Documentation Prometheus Documentation

Dragon Ball RESTful API - Keploy Collection Repository: Keploy Public APIs Collection Merged PR: #105 - Dragon Ball API Contribution The only solo developer-built API featured in Keploy’s official public APIs collection, standing among submissions from organizations. 🏗️ Backend Architecture Express.js + TypeScript: Fully typed, production-grade REST API. MongoDB + Mongoose: Structured, scalable schema for characters, series, and trivia data. JWT: Secure role-based authentication and token management. CORS: Proper handling of cross-origin requests for safe frontend access. 📘 API Documentation Swagger/OpenAPI 3.0: Fully interactive, auto-generated API docs. Structured Error Responses: Descriptive, standardized error handling. Rate Limiting: Prevents brute-force and abuse scenarios. 🧪 Testing & QA Keploy AI Testing: Snapshot-based regression testing with AI verification. Jest: Coverage for both unit and integration layers. Supertest: Endpoint behavior validation with real HTTP assertions. GitHub Actions: CI/CD pipeline for automated test, build, and deploy. 🔥 Key Features Comprehensive Trivia Dataset: Covers Dragon Ball, Z, GT, and Super. Dual Access Layer: Public APIs plus gated admin management routes. Dynamic Question System: Admin interface to add/update trivia. Performance Optimization: Indexed queries and lean data responses. "