added subcontractor report generation by RA bill and subcontractor name
This commit is contained in:
4
.env
4
.env
@@ -3,7 +3,7 @@
|
||||
# -----------------------------
|
||||
FLASK_ENV=development
|
||||
FLASK_DEBUG=True
|
||||
FLASK_HOST=127.0.0.1
|
||||
FLASK_HOST=0.0.0.0
|
||||
FLASK_PORT=5001
|
||||
|
||||
# -----------------------------
|
||||
@@ -20,7 +20,7 @@ DB_HOST=127.0.0.1
|
||||
DB_PORT=3306
|
||||
DB_NAME=comparisondb
|
||||
DB_USER=root
|
||||
DB_PASSWORD=root
|
||||
DB_PASSWORD=admin
|
||||
|
||||
# DATABASE_URL=mysql+pymysql://root:root@localhost/comparisondb
|
||||
|
||||
|
||||
571
README.md
571
README.md
@@ -1,3 +1,570 @@
|
||||
# Comparison_Project
|
||||
# Comparison Project - Backend Documentation
|
||||
|
||||
Comparison Project
|
||||
A Flask-based web application for comparing and managing construction project data (excavation and manhole work) between subcontractors and clients. The system imports data from Excel files, stores it in a database, and generates comparison reports.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Project Overview](#project-overview)
|
||||
- [Tech Stack](#tech-stack)
|
||||
- [Project Structure](#project-structure)
|
||||
- [Database Schema](#database-schema)
|
||||
- [Application Flow](#application-flow)
|
||||
- [API Routes & Endpoints](#api-routes--endpoints)
|
||||
- [Setup & Installation](#setup--installation)
|
||||
|
||||
---
|
||||
|
||||
## Project Overview
|
||||
|
||||
The Comparison Project is designed to:
|
||||
- **Manage subcontractors** and their excavation data (Manhole Excavation, Trench Excavation, Domestic Chambers)
|
||||
- **Import client-side project data** for comparison
|
||||
- **Compare subcontractor work** against client specifications
|
||||
- **Generate detailed reports** highlighting differences between client and subcontractor data
|
||||
- **User authentication** with login/registration system
|
||||
|
||||
### Key Features
|
||||
|
||||
✅ User Authentication (Register/Login/Logout)
|
||||
✅ Subcontractor Management (Add/Edit/List/Delete)
|
||||
✅ File Import (Excel/CSV) for both Client and Subcontractor data
|
||||
✅ Data validation and duplicate detection
|
||||
✅ Comparison Reports with Excel export
|
||||
✅ Dashboard with file management
|
||||
|
||||
---
|
||||
|
||||
## Tech Stack
|
||||
|
||||
**Backend Framework**: Flask
|
||||
**Database**: SQL Database (MySQL/PostgreSQL/SQLite configured via environment variables)
|
||||
**ORM**: SQLAlchemy
|
||||
**File Processing**: Pandas, OpenPyXL, XlsxWriter
|
||||
**Authentication**: Werkzeug (Password hashing)
|
||||
**Configuration**: Python Dotenv
|
||||
|
||||
### Dependencies
|
||||
|
||||
```
|
||||
Flask
|
||||
pandas
|
||||
openpyxl
|
||||
xlrd
|
||||
Werkzeug
|
||||
python-dotenv
|
||||
cryptography
|
||||
xlsxwriter
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
Comparison_Project/
|
||||
├── run.py # Application entry point
|
||||
├── requirements.txt # Python dependencies
|
||||
├── .env # Environment variables (DB, secrets)
|
||||
├── README.md # This file
|
||||
│
|
||||
├── app/
|
||||
│ ├── __init__.py # Flask app initialization
|
||||
│ ├── config.py # Configuration settings
|
||||
│ │
|
||||
│ ├── models/ # Database models (SQLAlchemy ORM)
|
||||
│ │ ├── user_model.py # User table schema
|
||||
│ │ ├── subcontractor_model.py # Subcontractor table schema
|
||||
│ │ ├── manhole_excavation_model.py # Subcontractor MH excavation
|
||||
│ │ ├── trench_excavation_model.py # Subcontractor Trench excavation
|
||||
│ │ ├── manhole_domestic_chamber_model.py # Subcontractor Domestic chamber
|
||||
│ │ ├── mh_ex_client_model.py # Client MH excavation
|
||||
│ │ ├── tr_ex_client_model.py # Client Trench excavation
|
||||
│ │ └── mh_dc_client_model.py # Client Domestic chamber
|
||||
│ │
|
||||
│ ├── routes/ # Flask Blueprints (API endpoints)
|
||||
│ │ ├── auth.py # Login, Register, Logout
|
||||
│ │ ├── dashboard.py # Dashboard page
|
||||
│ │ ├── file_import.py # File upload endpoints
|
||||
│ │ ├── file_report.py # Fetch and format data for reports
|
||||
│ │ ├── generate_comparison_report.py # Comparison logic & Excel export
|
||||
│ │ ├── subcontractor_routes.py # CRUD for subcontractors
|
||||
│ │ ├── user.py # User management routes
|
||||
│ │ └── file_format.py # File format validation
|
||||
│ │
|
||||
│ ├── services/ # Business logic layer
|
||||
│ │ ├── db_service.py # Database connection & SQLAlchemy init
|
||||
│ │ ├── file_service.py # File parsing & data insertion
|
||||
│ │ └── user_service.py # User authentication logic
|
||||
│ │
|
||||
│ ├── utils/ # Helper functions
|
||||
│ │ ├── file_utils.py # File path utilities
|
||||
│ │ ├── helpers.py # Decorators, common helpers
|
||||
│ │ └── __pycache__/
|
||||
│ │
|
||||
│ ├── static/ # Static assets
|
||||
│ │ ├── css/ # Stylesheets
|
||||
│ │ ├── images/ # Images
|
||||
│ │ ├── uploads/ # Uploaded files directory
|
||||
│ │ │ ├── Client_Bill_1/
|
||||
│ │ │ └── sub_1/
|
||||
│ │ └── downloads/ # Generated reports
|
||||
│ │
|
||||
│ ├── templates/ # Jinja2 HTML templates
|
||||
│ │ ├── base.html # Base template
|
||||
│ │ ├── login.html
|
||||
│ │ ├── register.html
|
||||
│ │ ├── dashboard.html
|
||||
│ │ ├── file_import.html
|
||||
│ │ ├── file_import_client.html
|
||||
│ │ ├── file_format.html
|
||||
│ │ ├── file_report.html
|
||||
│ │ ├── generate_comparison_report.html
|
||||
│ │ ├── generate_comparison_client_vs_subcont.html
|
||||
│ │ ├── list_user.html
|
||||
│ │ ├── report.html
|
||||
│ │ ├── users.htm
|
||||
│ │ └── subcontractor/
|
||||
│ │ ├── add.html
|
||||
│ │ ├── edit.html
|
||||
│ │ └── list.html
|
||||
│ │
|
||||
│ └── logs/ # Application logs
|
||||
│
|
||||
├── instance/ # Instance folder (DB, temp files)
|
||||
└── logs/ # Root level logs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database Schema
|
||||
|
||||
### 1. **Users Table** (`users`)
|
||||
Stores user authentication data.
|
||||
|
||||
| Column | Type | Constraints | Purpose |
|
||||
|--------|------|-------------|---------|
|
||||
| `id` | Integer | Primary Key | User identifier |
|
||||
| `name` | String(120) | NOT NULL | User's full name |
|
||||
| `email` | String(120) | UNIQUE, NOT NULL | User's email |
|
||||
| `password_hash` | String(255) | NOT NULL | Hashed password (Werkzeug) |
|
||||
|
||||
**Key Methods:**
|
||||
- `set_password(password)` - Hash and store password
|
||||
- `check_password(password)` - Verify password
|
||||
|
||||
---
|
||||
|
||||
### 2. **Subcontractors Table** (`subcontractors`)
|
||||
Stores subcontractor company information.
|
||||
|
||||
| Column | Type | Constraints | Purpose |
|
||||
|--------|------|-------------|---------|
|
||||
| `id` | Integer | Primary Key | Subcontractor ID |
|
||||
| `subcontractor_name` | String(255) | NOT NULL | Company name |
|
||||
| `address` | String(500) | - | Company address |
|
||||
| `gst_no` | String(50) | - | GST number |
|
||||
| `pan_no` | String(50) | - | PAN number |
|
||||
| `mobile_no` | String(20) | - | Contact phone |
|
||||
| `email_id` | String(150) | - | Contact email |
|
||||
| `contact_person` | String(150) | - | Contact person name |
|
||||
| `status` | String(20) | Default: "Active" | Active/Inactive status |
|
||||
| `created_at` | DateTime | Default: today | Record creation date |
|
||||
|
||||
**Relationships:**
|
||||
- One-to-Many with `manhole_excavation`
|
||||
- One-to-Many with `trench_excavation`
|
||||
- One-to-Many with `manhole_domestic_chamber`
|
||||
|
||||
---
|
||||
|
||||
### 3. **Manhole Excavation (Subcontractor)** (`manhole_excavation`)
|
||||
Stores manhole excavation work data from subcontractors.
|
||||
|
||||
| Column Type | Purpose |
|
||||
|-----|---------|
|
||||
| `id` | Primary Key |
|
||||
| `subcontractor_id` | Foreign Key → Subcontractor |
|
||||
| `Location` | Worksite location |
|
||||
| `MH_NO` | Manhole number |
|
||||
| `RA_Bill_No` | RA Bill reference (for grouping) |
|
||||
| `Upto_IL_Depth`, `Cutting_Depth`, `ID_of_MH_m`, `Ex_Dia_of_Manhole`, `Area_of_Manhole` | Dimension fields |
|
||||
| **Excavation Categories** (by depth ranges): | |
|
||||
| Soft_Murum_0_to_1_5, Soft_Murum_1_5_to_3_0, etc. | Material type + depth range |
|
||||
| Hard_Murum_0_to_1_5, Hard_Murum_1_5_to_3_0, etc. | Hard mudstone layers |
|
||||
| Soft_Rock_0_to_1_5, Soft_Rock_1_5_to_3_0, etc. | Soft rock layers |
|
||||
| Hard_Rock_0_to_1_5 through Hard_Rock_6_0_to_7_5 | Hard rock layers |
|
||||
| **Totals** | Sum per category |
|
||||
| `Total` | Grand total excavation |
|
||||
| `Remarks`, `created_at` | Notes and timestamp |
|
||||
|
||||
---
|
||||
|
||||
### 4. **Trench Excavation (Subcontractor)** (`trench_excavation`)
|
||||
Stores trench/sewer line excavation work data from subcontractors.
|
||||
|
||||
**Similar structure to Manhole Excavation with additional fields:**
|
||||
- Width categories: `Width_0_to_2_5`, `Width_2_5_to_3_0`, etc.
|
||||
- `Avg_Depth`, `Actual_Trench_Length`, `Pipe_Dia_mm`
|
||||
|
||||
---
|
||||
|
||||
### 5. **Manhole Domestic Chamber (Subcontractor)** (`manhole_domestic_chamber`)
|
||||
Stores domestic chamber construction data.
|
||||
|
||||
| Key Fields | Purpose |
|
||||
|-----------|---------|
|
||||
| `id`, `subcontractor_id` | Primary & Foreign Key |
|
||||
| `Location`, `MH_NO`, `RA_Bill_No` | Identification |
|
||||
| `Depth_of_MH`, `MH_TOP_LEVEL`, `MH_IL_LEVEL` | Dimensions |
|
||||
| `d_0_to_1_5` through `d_6_0_to_6_5` | Depth-based excavation categories |
|
||||
| `Domestic_Chambers` | Count of chambers |
|
||||
| `DWC_Pipe_Length`, `UPVC_Pipe_Length` | Pipe lengths |
|
||||
|
||||
---
|
||||
|
||||
### 6-8. **Client Data Tables** (Similar structure to subcontractor models)
|
||||
|
||||
- **`mh_ex_client`** - Manhole Excavation (Client specifications)
|
||||
- **`tr_ex_client`** - Trench Excavation (Client specifications)
|
||||
- **`mh_dc_client`** - Manhole Domestic Chamber (Client specifications)
|
||||
|
||||
**Note:** Client tables do NOT have `subcontractor_id` as they represent client baseline data.
|
||||
|
||||
---
|
||||
|
||||
## Application Flow
|
||||
|
||||
### 1. **User Authentication Flow**
|
||||
```
|
||||
User Access (/)
|
||||
↓
|
||||
Redirect to /login
|
||||
↓
|
||||
[Login Page]
|
||||
├─ POST /auth/login
|
||||
│ ↓
|
||||
│ UserService.validate_login(email, password)
|
||||
│ ↓
|
||||
│ Query User table, verify password
|
||||
│ ↓
|
||||
│ Set session["user_id"], session["user_name"]
|
||||
↓
|
||||
Redirect to /dashboard
|
||||
```
|
||||
|
||||
### 2. **Subcontractor File Import Flow**
|
||||
```
|
||||
User → /file/import [GET]
|
||||
↓
|
||||
Display form with subcontractor dropdown
|
||||
↓
|
||||
User selects subcontractor & uploads Excel file
|
||||
↓
|
||||
POST /file/import
|
||||
↓
|
||||
FileService.handle_file_upload()
|
||||
├─ Validate file type (xlsx, xls, csv)
|
||||
├─ Create upload folder: static/uploads/sub_{id}/
|
||||
├─ Save file
|
||||
│
|
||||
├─ Read Excel sheets:
|
||||
│ ├─ "Tr.Ex." (Trench Excavation)
|
||||
│ ├─ "MH Ex." (Manhole Excavation)
|
||||
│ └─ "MH & DC" (Manhole & Domestic Chamber)
|
||||
│
|
||||
├─ Process each sheet:
|
||||
│ ├─ Normalize column names
|
||||
│ ├─ Clean data (handle NaN, empty values)
|
||||
│ ├─ Check for duplicates by (Location, MH_NO, RA_Bill_No, subcontractor_id)
|
||||
│ ├─ Insert records into respective tables
|
||||
│ └─ Commit to database
|
||||
│
|
||||
└─ Return success/error message
|
||||
```
|
||||
|
||||
### 3. **Client File Import Flow**
|
||||
```
|
||||
User → /file/import_client [GET]
|
||||
↓
|
||||
Display form for client file upload
|
||||
↓
|
||||
User uploads Excel file with client specifications
|
||||
↓
|
||||
POST /file/import_client
|
||||
↓
|
||||
FileService.handle_client_file_upload()
|
||||
├─ Validate & save file
|
||||
├─ Read sheets: "Tr.Ex.", "MH Ex.", "MH & DC"
|
||||
├─ Process and insert into:
|
||||
│ ├─ mh_ex_client
|
||||
│ ├─ tr_ex_client
|
||||
│ └─ mh_dc_client
|
||||
└─ Return status
|
||||
```
|
||||
|
||||
### 4. **Comparison Report Generation Flow**
|
||||
```
|
||||
User → /report/generate_comparison [GET]
|
||||
↓
|
||||
Display form to select RA Bill No.
|
||||
↓
|
||||
User selects bill number
|
||||
↓
|
||||
POST /report/generate_comparison
|
||||
↓
|
||||
generate_report_bp routes request
|
||||
├─ Fetch client data (mh_ex_client, tr_ex_client, mh_dc_client)
|
||||
├─ Fetch subcontractor data (by RA_Bill_No)
|
||||
├─ For each record type:
|
||||
│ ├─ Create lookup table by (Location, MH_NO)
|
||||
│ ├─ Match client records with subcontractor records
|
||||
│ ├─ Calculate totals for each category
|
||||
│ ├─ Compute differences/variances
|
||||
│ └─ Format for Excel output
|
||||
│
|
||||
├─ Generate Excel file with comparison results:
|
||||
│ ├─ Summary sheet
|
||||
│ ├─ Detailed Trench Excavation comparison
|
||||
│ ├─ Detailed Manhole Excavation comparison
|
||||
│ └─ Detailed Domestic Chamber comparison
|
||||
│
|
||||
└─ Send file to client as download
|
||||
```
|
||||
|
||||
### 5. **Dashboard & Data Retrieval Flow**
|
||||
```
|
||||
User → /dashboard [GET]
|
||||
↓
|
||||
Check session["user_id"] (authentication)
|
||||
↓
|
||||
Render dashboard.html
|
||||
├─ Display available reports
|
||||
├─ List recent RA Bills
|
||||
├─ Show subcontractor list
|
||||
└─ Provide navigation to:
|
||||
├─ File import
|
||||
├─ Reports
|
||||
├─ Subcontractor management
|
||||
└─ Logout
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Routes & Endpoints
|
||||
|
||||
### Authentication Routes (`/auth`)
|
||||
| Method | Endpoint | Purpose | Auth |
|
||||
|--------|----------|---------|------|
|
||||
| GET/POST | `/login` | Login form & validation | ❌ |
|
||||
| GET/POST | `/register` | User registration | ❌ |
|
||||
| GET | `/logout` | Clear session & logout | ✅ |
|
||||
|
||||
### Dashboard Route (`/dashboard`)
|
||||
| Method | Endpoint | Purpose | Auth |
|
||||
|--------|----------|---------|------|
|
||||
| GET | `/dashboard/` | Main dashboard | ✅ |
|
||||
|
||||
### Subcontractor Routes (`/subcontractor`)
|
||||
| Method | Endpoint | Purpose | Auth |
|
||||
|--------|----------|---------|------|
|
||||
| GET | `/subcontractor/add` | Add form | ✅ |
|
||||
| POST | `/subcontractor/save` | Save new subcontractor | ✅ |
|
||||
| GET | `/subcontractor/list` | List all subcontractors | ✅ |
|
||||
| GET | `/subcontractor/edit/<id>` | Edit form | ✅ |
|
||||
| POST | `/subcontractor/update/<id>` | Update subcontractor | ✅ |
|
||||
| GET | `/subcontractor/delete/<id>` | Delete subcontractor | ✅ |
|
||||
|
||||
### File Import Routes (`/file`)
|
||||
| Method | Endpoint | Purpose | Auth |
|
||||
|--------|----------|---------|------|
|
||||
| GET/POST | `/file/import` | Subcontractor file upload | ✅ |
|
||||
| GET/POST | `/file/import_client` | Client file upload | ✅ |
|
||||
| GET/POST | `/file/report` | View imported data report | ✅ |
|
||||
|
||||
### Report Routes (`/report`)
|
||||
| Method | Endpoint | Purpose | Auth |
|
||||
|--------|----------|---------|------|
|
||||
| GET/POST | `/report/generate_comparison` | Generate comparison report | ✅ |
|
||||
| GET/POST | `/report/generate_comparison_client_vs_subcont` | Alternative comparison | ✅ |
|
||||
|
||||
### User Routes (`/user`)
|
||||
| Method | Endpoint | Purpose | Auth |
|
||||
|--------|----------|---------|------|
|
||||
| GET | `/user/list` | List all users | ✅ |
|
||||
| Other | (Check routes) | User management | ✅ |
|
||||
|
||||
### File Format Route (`/format`)
|
||||
| Method | Endpoint | Purpose | Auth |
|
||||
|--------|----------|---------|------|
|
||||
| GET/POST | `/format/...` | File format reference | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## Setup & Installation
|
||||
|
||||
### Prerequisites
|
||||
- Python 3.7+
|
||||
- MySQL/PostgreSQL (or SQLite for development)
|
||||
- pip (Python package manager)
|
||||
|
||||
### 1. Clone & Install Dependencies
|
||||
```bash
|
||||
cd d:\New folder\Comparison\Comparison_Project
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 2. Configure Environment Variables
|
||||
Create a `.env` file in the project root:
|
||||
```env
|
||||
# Flask Configuration
|
||||
FLASK_HOST=127.0.0.1
|
||||
FLASK_PORT=5000
|
||||
FLASK_DEBUG=True
|
||||
SECRET_KEY=your-secret-key-here
|
||||
|
||||
# Database Configuration
|
||||
DB_DIALECT=mysql # or postgresql, sqlite
|
||||
DB_DRIVER=pymysql # or psycopg2, etc.
|
||||
DB_USER=root
|
||||
DB_PASSWORD=your_password
|
||||
DB_HOST=localhost
|
||||
DB_PORT=3306
|
||||
DB_NAME=comparison_project
|
||||
```
|
||||
|
||||
### 3. Initialize Database
|
||||
```bash
|
||||
python run.py
|
||||
```
|
||||
This will:
|
||||
- Load environment variables
|
||||
- Create Flask app with configuration
|
||||
- Initialize SQLAlchemy
|
||||
- Create all tables (if not exist)
|
||||
- Start Flask development server
|
||||
|
||||
### 4. Access Application
|
||||
Open browser: `http://127.0.0.1:5000/`
|
||||
- First time: Redirect to `/login`
|
||||
- Register a new account
|
||||
- Login and start using the application
|
||||
|
||||
---
|
||||
|
||||
## Key Services & Utilities
|
||||
|
||||
### FileService (`app/services/file_service.py`)
|
||||
**Purpose:** Handles file uploads, parsing, and data validation
|
||||
|
||||
**Key Methods:**
|
||||
- `handle_file_upload(file, subcontractor_id, RA_Bill_No)` - Upload & process subcontractor file
|
||||
- `handle_client_file_upload(file, RA_Bill_No)` - Upload & process client file
|
||||
- `process_trench_excavation(df, subcontractor_id, RA_Bill_No)` - Parse & insert trench data
|
||||
- `process_manhole_excavation(df, subcontractor_id, RA_Bill_No)` - Parse & insert manhole data
|
||||
- `process_manhole_domestic_chamber(df, subcontractor_id, RA_Bill_No)` - Parse & insert chamber data
|
||||
|
||||
### UserService (`app/services/user_service.py`)
|
||||
**Purpose:** User authentication & management
|
||||
|
||||
**Key Methods:**
|
||||
- `register_user(name, email, password)` - Register new user
|
||||
- `validate_login(email, password)` - Authenticate user
|
||||
- `get_all_users()` - Fetch all users
|
||||
|
||||
### DBService (`app/services/db_service.py`)
|
||||
**Purpose:** Database connection & SQLAlchemy initialization
|
||||
|
||||
**Exports:**
|
||||
- `db` - SQLAlchemy instance for ORM operations
|
||||
- `migrate` - Flask-Migrate for schema migrations
|
||||
|
||||
---
|
||||
|
||||
## Authentication & Security
|
||||
|
||||
### Session Management
|
||||
- Uses Flask `session` object
|
||||
- Stores `user_id` and `user_name` after successful login
|
||||
- `@login_required` decorator validates authenticated requests
|
||||
|
||||
### Password Security
|
||||
- Passwords hashed using Werkzeug's `generate_password_hash()`
|
||||
- Verification via `check_password_hash()`
|
||||
- No plaintext passwords stored in database
|
||||
|
||||
### File Security
|
||||
- File uploads sanitized with `secure_filename()`
|
||||
- Only allowed extensions: `.xlsx`, `.xls`, `.csv`
|
||||
- Files stored in user-specific subfolders
|
||||
|
||||
---
|
||||
|
||||
## Error Handling & Validation
|
||||
|
||||
### File Import Validation
|
||||
1. **File Type Check** - Only CSV/XLS/XLSX allowed
|
||||
2. **Sheet Validation** - Required sheets must exist
|
||||
3. **Column Normalization** - Auto-fixes column name inconsistencies
|
||||
4. **Data Type Conversion** - Converts to appropriate types
|
||||
5. **Duplicate Detection** - Prevents duplicate records by (Location, MH_NO, RA_Bill_No)
|
||||
6. **Null Handling** - Converts empty/NaN values to None
|
||||
|
||||
### Database Constraints
|
||||
- Foreign key relationships enforced
|
||||
- Unique email constraint on users
|
||||
- NOT NULL constraints on critical fields
|
||||
|
||||
---
|
||||
|
||||
## Example: Typical User Workflow
|
||||
|
||||
```
|
||||
1. User registers/logs in
|
||||
→ POST /auth/register (new user) or /auth/login
|
||||
|
||||
2. User adds subcontractors
|
||||
→ GET /subcontractor/add
|
||||
→ POST /subcontractor/save
|
||||
|
||||
3. User uploads subcontractor data (Excel file with 3 sheets)
|
||||
→ GET /file/import
|
||||
→ POST /file/import (file, subcontractor_id, RA_Bill_No)
|
||||
→ Database populated with excavation data
|
||||
|
||||
4. User uploads client specifications (Excel file with same 3 sheets)
|
||||
→ GET /file/import_client
|
||||
→ POST /file/import_client (file, RA_Bill_No)
|
||||
→ Database populated with client data
|
||||
|
||||
5. User generates comparison report
|
||||
→ GET /report/generate_comparison
|
||||
→ POST /report/generate_comparison (RA_Bill_No)
|
||||
→ System compares client vs subcontractor data
|
||||
→ Generates Excel file showing differences
|
||||
→ User downloads report
|
||||
|
||||
6. User logs out
|
||||
→ GET /auth/logout
|
||||
→ Session cleared, redirected to login
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- [ ] Email notifications for report generation
|
||||
- [ ] Data visualization dashboards
|
||||
- [ ] Role-based access control (Admin, User, Viewer)
|
||||
- [ ] Bulk report generation
|
||||
- [ ] Data export to PDF format
|
||||
- [ ] Audit logs for data modifications
|
||||
- [ ] API documentation (Swagger/OpenAPI)
|
||||
- [ ] Unit & integration tests
|
||||
|
||||
---
|
||||
|
||||
## Support & Contribution
|
||||
|
||||
For issues, feature requests, or contributions, please contact the development team.
|
||||
|
||||
**Last Updated:** January 2026
|
||||
@@ -12,102 +12,40 @@ import pandas as pd
|
||||
import io
|
||||
from enum import Enum
|
||||
|
||||
# --- 1. DEFINE BLUEPRINT FIRST (Prevents NameError) ---
|
||||
file_report_bp = Blueprint("file_report", __name__, url_prefix="/file")
|
||||
|
||||
class BillType(Enum):
|
||||
Client = 1
|
||||
Subcontractor = 2
|
||||
|
||||
# --- 2. DEFINE CLASSES ---
|
||||
class SubcontractorBill:
|
||||
df_tr = []
|
||||
df_mh = []
|
||||
df_dc =[]
|
||||
def __init__(self):
|
||||
# Initialize as empty DataFrames so .to_excel() always exists
|
||||
self.df_tr = pd.DataFrame()
|
||||
self.df_mh = pd.DataFrame()
|
||||
self.df_dc = pd.DataFrame()
|
||||
|
||||
def Fetch(self, RA_Bill_No):
|
||||
# CLIENT DATA (RA BILL WISE)
|
||||
|
||||
trench = TrenchExcavation.query.filter_by(RA_Bill_No=RA_Bill_No).all()
|
||||
mh = ManholeExcavation.query.filter_by(RA_Bill_No=RA_Bill_No).all()
|
||||
dc = ManholeDomesticChamber.query.filter_by(RA_Bill_No=RA_Bill_No).all()
|
||||
def Fetch(self, RA_Bill_No, subcontractor_id):
|
||||
# Query data filtered by both Bill No and Subcontractor ID
|
||||
trench = TrenchExcavation.query.filter_by(RA_Bill_No=RA_Bill_No, subcontractor_id=subcontractor_id).all()
|
||||
mh = ManholeExcavation.query.filter_by(RA_Bill_No=RA_Bill_No, subcontractor_id=subcontractor_id).all()
|
||||
dc = ManholeDomesticChamber.query.filter_by(RA_Bill_No=RA_Bill_No, subcontractor_id=subcontractor_id).all()
|
||||
|
||||
# Convert SQL objects to DataFrames
|
||||
self.df_tr = pd.DataFrame([c.__dict__ for c in trench])
|
||||
self.df_mh = pd.DataFrame([c.__dict__ for c in mh])
|
||||
self.df_dc = pd.DataFrame([c.__dict__ for c in dc])
|
||||
|
||||
# CLEAN COLUMNS
|
||||
drop_cols = ["id","created_at","Remarks","_sa_instance_state"]
|
||||
|
||||
# Clean Columns (remove SQLAlchemy internal state)
|
||||
drop_cols = ["id", "created_at", "_sa_instance_state"]
|
||||
|
||||
for df in [self.df_tr, self.df_mh, self.df_dc]:
|
||||
if not df.empty:
|
||||
df.drop(columns=drop_cols, errors="ignore", inplace=True)
|
||||
|
||||
mh_exc_columns = [
|
||||
"Location", "MH_NO", "Upto_IL_Depth", "Cutting_Depth", "ID_of_MH_m",
|
||||
"Ex_Dia_of_Manhole", "Area_of_Manhole",
|
||||
|
||||
"Soft_Murum_0_to_1_5", "Soft_Murum_1_5_to_3_0", "Soft_Murum_3_0_to_4_5",
|
||||
"Hard_Murum_0_to_1_5", "Hard_Murum_1_5_to_3_0",
|
||||
"Soft_Rock_0_to_1_5", "Soft_Rock_1_5_to_3_0",
|
||||
"Hard_Rock_0_to_1_5", "Hard_Rock_1_5_to_3_0",
|
||||
"Hard_Rock_3_0_to_4_5", "Hard_Rock_4_5_to_6_0", "Hard_Rock_6_0_to_7_5",
|
||||
|
||||
"Soft_Murum_0_to_1_5_total", "Soft_Murum_1_5_to_3_0_total",
|
||||
"Soft_Murum_3_0_to_4_5_total",
|
||||
"Hard_Murum_0_to_1_5_total", "Hard_Murum_1_5_and_above_total",
|
||||
"Soft_Rock_0_to_1_5_total", "Soft_Rock_1_5_and_above_total",
|
||||
"Hard_Rock_0_to_1_5_total", "Hard_Rock_1_5_and_above_total",
|
||||
"Hard_Rock_3_0_to_4_5_total", "Hard_Rock_4_5_to_6_0_total",
|
||||
"Hard_Rock_6_0_to_7_5_total",
|
||||
|
||||
"Remarks", "Total"
|
||||
]
|
||||
|
||||
trench_columns = [
|
||||
"Location", "MH_NO", "CC_length", "Invert_Level", "MH_Top_Level",
|
||||
"Ground_Level", "ID_of_MH_m", "Actual_Trench_Length", "Pipe_Dia_mm",
|
||||
|
||||
"Width_0_to_2_5", "Width_2_5_to_3_0", "Width_3_0_to_4_5", "Width_4_5_to_6_0",
|
||||
|
||||
"Upto_IL_Depth", "Cutting_Depth", "Avg_Depth",
|
||||
|
||||
"Soft_Murum_0_to_1_5", "Soft_Murum_1_5_to_3_0", "Soft_Murum_3_0_to_4_5",
|
||||
"Hard_Murum_0_to_1_5", "Hard_Murum_1_5_to_3_0",
|
||||
"Soft_Rock_0_to_1_5", "Soft_Rock_1_5_to_3_0",
|
||||
"Hard_Rock_0_to_1_5", "Hard_Rock_1_5_to_3_0",
|
||||
"Hard_Rock_3_0_to_4_5", "Hard_Rock_4_5_to_6_0", "Hard_Rock_6_0_to_7_5",
|
||||
|
||||
"Soft_Murum_0_to_1_5_total", "Soft_Murum_1_5_to_3_0_total",
|
||||
"Soft_Murum_3_0_to_4_5_total",
|
||||
"Hard_Murum_0_to_1_5_total", "Hard_Murum_1_5_and_above_total",
|
||||
"Soft_Rock_0_to_1_5_total", "Soft_Rock_1_5_and_above_total",
|
||||
"Hard_Rock_0_to_1_5_total", "Hard_Rock_1_5_and_above_total",
|
||||
"Hard_Rock_3_0_to_4_5_total", "Hard_Rock_4_5_to_6_0_total",
|
||||
"Hard_Rock_6_0_to_7_5_total",
|
||||
|
||||
"Remarks", "Total"
|
||||
]
|
||||
|
||||
domestic_columns = [
|
||||
"Location", "Node_No", "Depth_of_MH",
|
||||
"d_0_to_0_75", "d_1_06_to_1_65", "d_1_66_to_2_15",
|
||||
"d_2_16_to_2_65", "d_2_66_to_3_15", "d_3_16_to_3_65",
|
||||
"d_3_66_to_4_15", "d_4_16_to_4_65", "d_4_66_to_5_15",
|
||||
"d_5_16_to_5_65", "d_5_66_to_6_15", "d_6_16_to_6_65",
|
||||
"d_6_66_to_7_15", "d_7_16_to_7_65", "d_7_66_to_8_15",
|
||||
"d_8_16_to_8_65", "d_8_66_to_9_15", "d_9_16_to_9_65",
|
||||
"Domestic_Chambers"
|
||||
]
|
||||
|
||||
|
||||
|
||||
# Reorder columns serial wise
|
||||
df_mh_exc = self.df_mh.reindex(columns=mh_exc_columns, fill_value="")
|
||||
df_trench = self.df_tr.reindex(columns=trench_columns, fill_value="")
|
||||
df_domestic = self.df_dc.reindex(columns=domestic_columns, fill_value="")
|
||||
|
||||
|
||||
|
||||
# get report by contractor id and dowanload
|
||||
# --- 3. DEFINE ROUTES ---
|
||||
@file_report_bp.route("/report", methods=["GET", "POST"])
|
||||
@login_required
|
||||
def report_file():
|
||||
@@ -115,27 +53,33 @@ def report_file():
|
||||
|
||||
if request.method == "POST":
|
||||
subcontractor_id = request.form.get("subcontractor_id")
|
||||
ra_bill_no = request.form.get("ra_bill_no") # Collected from the updated HTML
|
||||
|
||||
if not subcontractor_id:
|
||||
flash("Please select a subcontractor.", "danger")
|
||||
if not subcontractor_id or not ra_bill_no:
|
||||
flash("Please select a subcontractor and enter an RA Bill Number.", "danger")
|
||||
return render_template("report.html", subcontractors=subcontractors)
|
||||
|
||||
# Fetch subcontractor for file name
|
||||
subcontractor = Subcontractor.query.get(subcontractor_id)
|
||||
subcontractorBill = SubcontractorBill()
|
||||
|
||||
|
||||
# WRITE EXCEL FILE
|
||||
# Instantiate and Fetch Data
|
||||
bill_gen = SubcontractorBill()
|
||||
bill_gen.Fetch(RA_Bill_No=ra_bill_no, subcontractor_id=subcontractor_id)
|
||||
|
||||
# Check if any data was found
|
||||
if bill_gen.df_tr.empty and bill_gen.df_mh.empty and bill_gen.df_dc.empty:
|
||||
flash(f"No data found for {subcontractor.subcontractor_name} in RA Bill {ra_bill_no}", "warning")
|
||||
return render_template("report.html", subcontractors=subcontractors)
|
||||
|
||||
# WRITE EXCEL FILE
|
||||
output = io.BytesIO()
|
||||
file_name = f"{subcontractor.subcontractor_name}_Report.xlsx"
|
||||
file_name = f"{subcontractor.subcontractor_name}_RA_{ra_bill_no}_Report.xlsx"
|
||||
|
||||
with pd.ExcelWriter(output, engine="xlsxwriter") as writer:
|
||||
subcontractorBill.df_tr.to_excel(writer, index=False, sheet_name="Tr.Ex.")
|
||||
subcontractorBill.df_mh.to_excel(writer, index=False, sheet_name="MH.Ex.")
|
||||
subcontractorBill.df_dc.to_excel(writer, index=False, sheet_name="MH & DC")
|
||||
bill_gen.df_tr.to_excel(writer, index=False, sheet_name="Tr.Ex.")
|
||||
bill_gen.df_mh.to_excel(writer, index=False, sheet_name="MH.Ex.")
|
||||
bill_gen.df_dc.to_excel(writer, index=False, sheet_name="MH & DC")
|
||||
|
||||
output.seek(0)
|
||||
|
||||
return send_file(
|
||||
output,
|
||||
download_name=file_name,
|
||||
@@ -145,31 +89,8 @@ def report_file():
|
||||
|
||||
return render_template("report.html", subcontractors=subcontractors)
|
||||
|
||||
# (ClientBill class and client_vs_all_subcontractor route would follow here...)
|
||||
|
||||
class ClientBill:
|
||||
df_tr = []
|
||||
df_mh = []
|
||||
df_dc =[]
|
||||
|
||||
def Fetch(self, RA_Bill_No):
|
||||
# CLIENT DATA (RA BILL WISE)
|
||||
|
||||
trench = TrenchExcavationClient.query.filter_by(RA_Bill_No=RA_Bill_No).all()
|
||||
mh = ManholeExcavationClient.query.filter_by(RA_Bill_No=RA_Bill_No).all()
|
||||
dc = ManholeDomesticChamberClient.query.filter_by(RA_Bill_No=RA_Bill_No).all()
|
||||
|
||||
self.df_tr = pd.DataFrame([c.__dict__ for c in trench])
|
||||
self.df_mh = pd.DataFrame([c.__dict__ for c in mh])
|
||||
self.df_dc = pd.DataFrame([c.__dict__ for c in dc])
|
||||
|
||||
# CLEAN COLUMNS
|
||||
drop_cols = ["id","created_at","Remarks","_sa_instance_state"]
|
||||
|
||||
for df in [self.df_tr, self.df_mh, self.df_dc]:
|
||||
if not df.empty:
|
||||
df.drop(columns=drop_cols, errors="ignore", inplace=True)
|
||||
|
||||
|
||||
|
||||
@file_report_bp.route("/client_vs_subcont", methods=["GET", "POST"])
|
||||
@login_required
|
||||
@@ -300,4 +221,4 @@ def client_vs_all_subcontractor():
|
||||
mimetype="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
|
||||
)
|
||||
|
||||
return render_template("generate_comparison_client_vs_subcont.html")
|
||||
return render_template("generate_comparison_client_vs_subcont.html")
|
||||
@@ -2,38 +2,39 @@
|
||||
|
||||
{% block content %}
|
||||
<div class="container mt-4">
|
||||
<h2 class="mb-4">Generate Subcontractor Report</h2>
|
||||
|
||||
<h2 class="mb-4">File Report</h2>
|
||||
|
||||
<!-- FLASH MESSAGES -->
|
||||
{% with messages = get_flashed_messages(with_categories=true) %}
|
||||
{% if messages %}
|
||||
{% for category, message in messages %}
|
||||
<div class="alert alert-{{ category }} alert-dismissible fade show" role="alert">
|
||||
{{ message }}
|
||||
<button type="button" class="btn-close" data-bs-dismiss="alert"></button>
|
||||
</div>
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% if messages %}
|
||||
{% for category, message in messages %}
|
||||
<div class="alert alert-{{ category }} alert-dismissible fade show" role="alert">
|
||||
{{ message }}
|
||||
<button type="button" class="btn-close" data-bs-dismiss="alert"></button>
|
||||
</div>
|
||||
)
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
{% endwith %}
|
||||
|
||||
<div class="card p-4 shadow-sm">
|
||||
|
||||
<form method="POST">
|
||||
<div class="mb-3">
|
||||
<label class="form-label fw-semibold">Select Subcontractor</label>
|
||||
<select name="subcontractor_id" class="form-select" required>
|
||||
<option value="">-- Select Subcontractor --</option>
|
||||
{% for sc in subcontractors %}
|
||||
<option value="{{ sc.id }}">{{ sc.subcontractor_name }}</option>
|
||||
{% endfor %}
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<!-- SELECT SUBCONTRACTOR -->
|
||||
<label class="form-label fw-semibold">Select Subcontractor</label>
|
||||
<select name="subcontractor_id" id="subcontractor_id" class="form-select mb-3" required>
|
||||
<option value="">-- Select Subcontractor --</option>
|
||||
<div class="mb-3">
|
||||
<label class="form-label fw-semibold">RA Bill Number</label>
|
||||
<input type="text" name="ra_bill_no" class="form-control" placeholder="e.g. 01" required>
|
||||
</div>
|
||||
|
||||
{% for sc in subcontractors %}
|
||||
<option value="{{ sc.id }}">{{ sc.subcontractor_name }}</option>
|
||||
{% endfor %}
|
||||
</select>
|
||||
|
||||
<button class="btn btn-primary w-100">Generate Report</button>
|
||||
<button type="submit" class="btn btn-primary w-100">Generate Excel Report</button>
|
||||
</form>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
{% endblock %}
|
||||
Reference in New Issue
Block a user