pjpatil12/Comparison_Project
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

git issue solved

Browse Source
This commit is contained in:
anishd100
2026-01-16 12:38:38 +05:30
parent bdc33d2495
commit f73f6524e8
14 changed files with 435 additions and 710 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.env
View File

@@ -3,7 +3,7 @@
# -----------------------------
FLASK_ENV=development
FLASK_DEBUG=True
FLASK_HOST=0.0.0.0
FLASK_HOST='0.0.0.0'
FLASK_PORT=5001
# -----------------------------

571
README.md
View File

@@ -1,570 +1,3 @@
# Comparison Project - Backend Documentation
# Comparison_Project
A Flask-based web application for comparing and managing construction project data (excavation and manhole work) between subcontractors and clients. The system imports data from Excel files, stores it in a database, and generates comparison reports.
## Table of Contents
- [Project Overview](#project-overview)
- [Tech Stack](#tech-stack)
- [Project Structure](#project-structure)
- [Database Schema](#database-schema)
- [Application Flow](#application-flow)
- [API Routes & Endpoints](#api-routes--endpoints)
- [Setup & Installation](#setup--installation)
---
## Project Overview
The Comparison Project is designed to:
- **Manage subcontractors** and their excavation data (Manhole Excavation, Trench Excavation, Domestic Chambers)
- **Import client-side project data** for comparison
- **Compare subcontractor work** against client specifications
- **Generate detailed reports** highlighting differences between client and subcontractor data
- **User authentication** with login/registration system
### Key Features
✅ User Authentication (Register/Login/Logout)
✅ Subcontractor Management (Add/Edit/List/Delete)
✅ File Import (Excel/CSV) for both Client and Subcontractor data
✅ Data validation and duplicate detection
✅ Comparison Reports with Excel export
✅ Dashboard with file management
---
## Tech Stack
**Backend Framework**: Flask
**Database**: SQL Database (MySQL/PostgreSQL/SQLite configured via environment variables)
**ORM**: SQLAlchemy
**File Processing**: Pandas, OpenPyXL, XlsxWriter
**Authentication**: Werkzeug (Password hashing)
**Configuration**: Python Dotenv
### Dependencies
```
Flask
pandas
openpyxl
xlrd
Werkzeug
python-dotenv
cryptography
xlsxwriter
```
---
## Project Structure
```
Comparison_Project/
├── run.py # Application entry point
├── requirements.txt # Python dependencies
├── .env # Environment variables (DB, secrets)
├── README.md # This file
│
├── app/
│ ├── __init__.py # Flask app initialization
│ ├── config.py # Configuration settings
│ │
│ ├── models/ # Database models (SQLAlchemy ORM)
│ │ ├── user_model.py # User table schema
│ │ ├── subcontractor_model.py # Subcontractor table schema
│ │ ├── manhole_excavation_model.py # Subcontractor MH excavation
│ │ ├── trench_excavation_model.py # Subcontractor Trench excavation
│ │ ├── manhole_domestic_chamber_model.py # Subcontractor Domestic chamber
│ │ ├── mh_ex_client_model.py # Client MH excavation
│ │ ├── tr_ex_client_model.py # Client Trench excavation
│ │ └── mh_dc_client_model.py # Client Domestic chamber
│ │
│ ├── routes/ # Flask Blueprints (API endpoints)
│ │ ├── auth.py # Login, Register, Logout
│ │ ├── dashboard.py # Dashboard page
│ │ ├── file_import.py # File upload endpoints
│ │ ├── file_report.py # Fetch and format data for reports
│ │ ├── generate_comparison_report.py # Comparison logic & Excel export
│ │ ├── subcontractor_routes.py # CRUD for subcontractors
│ │ ├── user.py # User management routes
│ │ └── file_format.py # File format validation
│ │
│ ├── services/ # Business logic layer
│ │ ├── db_service.py # Database connection & SQLAlchemy init
│ │ ├── file_service.py # File parsing & data insertion
│ │ └── user_service.py # User authentication logic
│ │
│ ├── utils/ # Helper functions
│ │ ├── file_utils.py # File path utilities
│ │ ├── helpers.py # Decorators, common helpers
│ │ └── __pycache__/
│ │
│ ├── static/ # Static assets
│ │ ├── css/ # Stylesheets
│ │ ├── images/ # Images
│ │ ├── uploads/ # Uploaded files directory
│ │ │ ├── Client_Bill_1/
│ │ │ └── sub_1/
│ │ └── downloads/ # Generated reports
│ │
│ ├── templates/ # Jinja2 HTML templates
│ │ ├── base.html # Base template
│ │ ├── login.html
│ │ ├── register.html
│ │ ├── dashboard.html
│ │ ├── file_import.html
│ │ ├── file_import_client.html
│ │ ├── file_format.html
│ │ ├── file_report.html
│ │ ├── generate_comparison_report.html
│ │ ├── generate_comparison_client_vs_subcont.html
│ │ ├── list_user.html
│ │ ├── report.html
│ │ ├── users.htm
│ │ └── subcontractor/
│ │ ├── add.html
│ │ ├── edit.html
│ │ └── list.html
│ │
│ └── logs/ # Application logs
│
├── instance/ # Instance folder (DB, temp files)
└── logs/ # Root level logs
```
---
## Database Schema
### 1. **Users Table** (`users`)
Stores user authentication data.
| Column | Type | Constraints | Purpose |
|--------|------|-------------|---------|
| `id` | Integer | Primary Key | User identifier |
| `name` | String(120) | NOT NULL | User's full name |
| `email` | String(120) | UNIQUE, NOT NULL | User's email |
| `password_hash` | String(255) | NOT NULL | Hashed password (Werkzeug) |
**Key Methods:**
- `set_password(password)` - Hash and store password
- `check_password(password)` - Verify password
---
### 2. **Subcontractors Table** (`subcontractors`)
Stores subcontractor company information.
| Column | Type | Constraints | Purpose |
|--------|------|-------------|---------|
| `id` | Integer | Primary Key | Subcontractor ID |
| `subcontractor_name` | String(255) | NOT NULL | Company name |
| `address` | String(500) | - | Company address |
| `gst_no` | String(50) | - | GST number |
| `pan_no` | String(50) | - | PAN number |
| `mobile_no` | String(20) | - | Contact phone |
| `email_id` | String(150) | - | Contact email |
| `contact_person` | String(150) | - | Contact person name |
| `status` | String(20) | Default: "Active" | Active/Inactive status |
| `created_at` | DateTime | Default: today | Record creation date |
**Relationships:**
- One-to-Many with `manhole_excavation`
- One-to-Many with `trench_excavation`
- One-to-Many with `manhole_domestic_chamber`
---
### 3. **Manhole Excavation (Subcontractor)** (`manhole_excavation`)
Stores manhole excavation work data from subcontractors.
| Column Type | Purpose |
|-----|---------|
| `id` | Primary Key |
| `subcontractor_id` | Foreign Key → Subcontractor |
| `Location` | Worksite location |
| `MH_NO` | Manhole number |
| `RA_Bill_No` | RA Bill reference (for grouping) |
| `Upto_IL_Depth`, `Cutting_Depth`, `ID_of_MH_m`, `Ex_Dia_of_Manhole`, `Area_of_Manhole` | Dimension fields |
| **Excavation Categories** (by depth ranges): | |
| Soft_Murum_0_to_1_5, Soft_Murum_1_5_to_3_0, etc. | Material type + depth range |
| Hard_Murum_0_to_1_5, Hard_Murum_1_5_to_3_0, etc. | Hard mudstone layers |
| Soft_Rock_0_to_1_5, Soft_Rock_1_5_to_3_0, etc. | Soft rock layers |
| Hard_Rock_0_to_1_5 through Hard_Rock_6_0_to_7_5 | Hard rock layers |
| **Totals** | Sum per category |
| `Total` | Grand total excavation |
| `Remarks`, `created_at` | Notes and timestamp |
---
### 4. **Trench Excavation (Subcontractor)** (`trench_excavation`)
Stores trench/sewer line excavation work data from subcontractors.
**Similar structure to Manhole Excavation with additional fields:**
- Width categories: `Width_0_to_2_5`, `Width_2_5_to_3_0`, etc.
- `Avg_Depth`, `Actual_Trench_Length`, `Pipe_Dia_mm`
---
### 5. **Manhole Domestic Chamber (Subcontractor)** (`manhole_domestic_chamber`)
Stores domestic chamber construction data.
| Key Fields | Purpose |
|-----------|---------|
| `id`, `subcontractor_id` | Primary & Foreign Key |
| `Location`, `MH_NO`, `RA_Bill_No` | Identification |
| `Depth_of_MH`, `MH_TOP_LEVEL`, `MH_IL_LEVEL` | Dimensions |
| `d_0_to_1_5` through `d_6_0_to_6_5` | Depth-based excavation categories |
| `Domestic_Chambers` | Count of chambers |
| `DWC_Pipe_Length`, `UPVC_Pipe_Length` | Pipe lengths |
---
### 6-8. **Client Data Tables** (Similar structure to subcontractor models)
- **`mh_ex_client`** - Manhole Excavation (Client specifications)
- **`tr_ex_client`** - Trench Excavation (Client specifications)
- **`mh_dc_client`** - Manhole Domestic Chamber (Client specifications)
**Note:** Client tables do NOT have `subcontractor_id` as they represent client baseline data.
---
## Application Flow
### 1. **User Authentication Flow**
```
User Access (/)
↓
Redirect to /login
↓
[Login Page]
├─ POST /auth/login
│ ↓
│ UserService.validate_login(email, password)
│ ↓
│ Query User table, verify password
│ ↓
│ Set session["user_id"], session["user_name"]
↓
Redirect to /dashboard
```
### 2. **Subcontractor File Import Flow**
```
User → /file/import [GET]
↓
Display form with subcontractor dropdown
↓
User selects subcontractor & uploads Excel file
↓
POST /file/import
↓
FileService.handle_file_upload()
├─ Validate file type (xlsx, xls, csv)
├─ Create upload folder: static/uploads/sub_{id}/
├─ Save file
│
├─ Read Excel sheets:
│ ├─ "Tr.Ex." (Trench Excavation)
│ ├─ "MH Ex." (Manhole Excavation)
│ └─ "MH & DC" (Manhole & Domestic Chamber)
│
├─ Process each sheet:
│ ├─ Normalize column names
│ ├─ Clean data (handle NaN, empty values)
│ ├─ Check for duplicates by (Location, MH_NO, RA_Bill_No, subcontractor_id)
│ ├─ Insert records into respective tables
│ └─ Commit to database
│
└─ Return success/error message
```
### 3. **Client File Import Flow**
```
User → /file/import_client [GET]
↓
Display form for client file upload
↓
User uploads Excel file with client specifications
↓
POST /file/import_client
↓
FileService.handle_client_file_upload()
├─ Validate & save file
├─ Read sheets: "Tr.Ex.", "MH Ex.", "MH & DC"
├─ Process and insert into:
│ ├─ mh_ex_client
│ ├─ tr_ex_client
│ └─ mh_dc_client
└─ Return status
```
### 4. **Comparison Report Generation Flow**
```
User → /report/generate_comparison [GET]
↓
Display form to select RA Bill No.
↓
User selects bill number
↓
POST /report/generate_comparison
↓
generate_report_bp routes request
├─ Fetch client data (mh_ex_client, tr_ex_client, mh_dc_client)
├─ Fetch subcontractor data (by RA_Bill_No)
├─ For each record type:
│ ├─ Create lookup table by (Location, MH_NO)
│ ├─ Match client records with subcontractor records
│ ├─ Calculate totals for each category
│ ├─ Compute differences/variances
│ └─ Format for Excel output
│
├─ Generate Excel file with comparison results:
│ ├─ Summary sheet
│ ├─ Detailed Trench Excavation comparison
│ ├─ Detailed Manhole Excavation comparison
│ └─ Detailed Domestic Chamber comparison
│
└─ Send file to client as download
```
### 5. **Dashboard & Data Retrieval Flow**
```
User → /dashboard [GET]
↓
Check session["user_id"] (authentication)
↓
Render dashboard.html
├─ Display available reports
├─ List recent RA Bills
├─ Show subcontractor list
└─ Provide navigation to:
├─ File import
├─ Reports
├─ Subcontractor management
└─ Logout
```
---
## API Routes & Endpoints
### Authentication Routes (`/auth`)
| Method | Endpoint | Purpose | Auth |
|--------|----------|---------|------|
| GET/POST | `/login` | Login form & validation | ❌ |
| GET/POST | `/register` | User registration | ❌ |
| GET | `/logout` | Clear session & logout | ✅ |
### Dashboard Route (`/dashboard`)
| Method | Endpoint | Purpose | Auth |
|--------|----------|---------|------|
| GET | `/dashboard/` | Main dashboard | ✅ |
### Subcontractor Routes (`/subcontractor`)
| Method | Endpoint | Purpose | Auth |
|--------|----------|---------|------|
| GET | `/subcontractor/add` | Add form | ✅ |
| POST | `/subcontractor/save` | Save new subcontractor | ✅ |
| GET | `/subcontractor/list` | List all subcontractors | ✅ |
| GET | `/subcontractor/edit/<id>` | Edit form | ✅ |
| POST | `/subcontractor/update/<id>` | Update subcontractor | ✅ |
| GET | `/subcontractor/delete/<id>` | Delete subcontractor | ✅ |
### File Import Routes (`/file`)
| Method | Endpoint | Purpose | Auth |
|--------|----------|---------|------|
| GET/POST | `/file/import` | Subcontractor file upload | ✅ |
| GET/POST | `/file/import_client` | Client file upload | ✅ |
| GET/POST | `/file/report` | View imported data report | ✅ |
### Report Routes (`/report`)
| Method | Endpoint | Purpose | Auth |
|--------|----------|---------|------|
| GET/POST | `/report/generate_comparison` | Generate comparison report | ✅ |
| GET/POST | `/report/generate_comparison_client_vs_subcont` | Alternative comparison | ✅ |
### User Routes (`/user`)
| Method | Endpoint | Purpose | Auth |
|--------|----------|---------|------|
| GET | `/user/list` | List all users | ✅ |
| Other | (Check routes) | User management | ✅ |
### File Format Route (`/format`)
| Method | Endpoint | Purpose | Auth |
|--------|----------|---------|------|
| GET/POST | `/format/...` | File format reference | ✅ |
---
## Setup & Installation
### Prerequisites
- Python 3.7+
- MySQL/PostgreSQL (or SQLite for development)
- pip (Python package manager)
### 1. Clone & Install Dependencies
```bash
cd d:\New folder\Comparison\Comparison_Project
pip install -r requirements.txt
```
### 2. Configure Environment Variables
Create a `.env` file in the project root:
```env
# Flask Configuration
FLASK_HOST=127.0.0.1
FLASK_PORT=5000
FLASK_DEBUG=True
SECRET_KEY=your-secret-key-here
# Database Configuration
DB_DIALECT=mysql # or postgresql, sqlite
DB_DRIVER=pymysql # or psycopg2, etc.
DB_USER=root
DB_PASSWORD=your_password
DB_HOST=localhost
DB_PORT=3306
DB_NAME=comparison_project
```
### 3. Initialize Database
```bash
python run.py
```
This will:
- Load environment variables
- Create Flask app with configuration
- Initialize SQLAlchemy
- Create all tables (if not exist)
- Start Flask development server
### 4. Access Application
Open browser: `http://127.0.0.1:5000/`
- First time: Redirect to `/login`
- Register a new account
- Login and start using the application
---
## Key Services & Utilities
### FileService (`app/services/file_service.py`)
**Purpose:** Handles file uploads, parsing, and data validation
**Key Methods:**
- `handle_file_upload(file, subcontractor_id, RA_Bill_No)` - Upload & process subcontractor file
- `handle_client_file_upload(file, RA_Bill_No)` - Upload & process client file
- `process_trench_excavation(df, subcontractor_id, RA_Bill_No)` - Parse & insert trench data
- `process_manhole_excavation(df, subcontractor_id, RA_Bill_No)` - Parse & insert manhole data
- `process_manhole_domestic_chamber(df, subcontractor_id, RA_Bill_No)` - Parse & insert chamber data
### UserService (`app/services/user_service.py`)
**Purpose:** User authentication & management
**Key Methods:**
- `register_user(name, email, password)` - Register new user
- `validate_login(email, password)` - Authenticate user
- `get_all_users()` - Fetch all users
### DBService (`app/services/db_service.py`)
**Purpose:** Database connection & SQLAlchemy initialization
**Exports:**
- `db` - SQLAlchemy instance for ORM operations
- `migrate` - Flask-Migrate for schema migrations
---
## Authentication & Security
### Session Management
- Uses Flask `session` object
- Stores `user_id` and `user_name` after successful login
- `@login_required` decorator validates authenticated requests
### Password Security
- Passwords hashed using Werkzeug's `generate_password_hash()`
- Verification via `check_password_hash()`
- No plaintext passwords stored in database
### File Security
- File uploads sanitized with `secure_filename()`
- Only allowed extensions: `.xlsx`, `.xls`, `.csv`
- Files stored in user-specific subfolders
---
## Error Handling & Validation
### File Import Validation
1. **File Type Check** - Only CSV/XLS/XLSX allowed
2. **Sheet Validation** - Required sheets must exist
3. **Column Normalization** - Auto-fixes column name inconsistencies
4. **Data Type Conversion** - Converts to appropriate types
5. **Duplicate Detection** - Prevents duplicate records by (Location, MH_NO, RA_Bill_No)
6. **Null Handling** - Converts empty/NaN values to None
### Database Constraints
- Foreign key relationships enforced
- Unique email constraint on users
- NOT NULL constraints on critical fields
---
## Example: Typical User Workflow
```
1. User registers/logs in
→ POST /auth/register (new user) or /auth/login
2. User adds subcontractors
→ GET /subcontractor/add
→ POST /subcontractor/save
3. User uploads subcontractor data (Excel file with 3 sheets)
→ GET /file/import
→ POST /file/import (file, subcontractor_id, RA_Bill_No)
→ Database populated with excavation data
4. User uploads client specifications (Excel file with same 3 sheets)
→ GET /file/import_client
→ POST /file/import_client (file, RA_Bill_No)
→ Database populated with client data
5. User generates comparison report
→ GET /report/generate_comparison
→ POST /report/generate_comparison (RA_Bill_No)
→ System compares client vs subcontractor data
→ Generates Excel file showing differences
→ User downloads report
6. User logs out
→ GET /auth/logout
→ Session cleared, redirected to login
```
---
## Future Enhancements
- [ ] Email notifications for report generation
- [ ] Data visualization dashboards
- [ ] Role-based access control (Admin, User, Viewer)
- [ ] Bulk report generation
- [ ] Data export to PDF format
- [ ] Audit logs for data modifications
- [ ] API documentation (Swagger/OpenAPI)
- [ ] Unit & integration tests
---
## Support & Contribution
For issues, feature requests, or contributions, please contact the development team.
**Last Updated:** January 2026
Comparison Project

47
app/models/laying_model.py Normal file
View File

@@ -0,0 +1,47 @@
from app import db
from datetime import datetime
class Laying(db.Model):
__tablename__ = "laying"
id = db.Column(db.Integer, primary_key=True)
# Foreign Key to Subcontractor table
subcontractor_id = db.Column(db.Integer, db.ForeignKey("subcontractors.id"), nullable=False)
# Relationship for easy access (subcontractor.subcontractor_name)
subcontractor = db.relationship("Subcontractor", backref="laying_records")
# Basic Fields
Location = db.Column(db.String(500))
MH_NO = db.Column(db.String(100))
CC_length = db.Column(db.Float)
Pipe_Dia_mm = db.Column(db.Float)
ID_of_MH_m = db.Column(db.Float)
Laying_Length = db.Column(db.Float)
pipe_150_mm = db.Column(db.Float)
pipe_200_mm = db.Column(db.Float)
pipe_250_mm = db.Column(db.Float)
pipe_300_mm = db.Column(db.Float)
pipe_350_mm = db.Column(db.Float)
pipe_400_mm = db.Column(db.Float)
pipe_450_mm = db.Column(db.Float)
pipe_500_mm = db.Column(db.Float)
pipe_600_mm = db.Column(db.Float)
pipe_700_mm = db.Column(db.Float)
pipe_900_mm = db.Column(db.Float)
pipe_1200_mm = db.Column(db.Float)
Total = db.Column(db.Float)
Remarks = db.Column(db.String(500))
RA_Bill_No=db.Column(db.String(500))
created_at = db.Column(db.DateTime, default=datetime.today)
def __repr__(self):
return f"<LayingModel {self.Location}>"
def serialize(self):
return {c.name: getattr(self, c.name) for c in self.__table__.columns}

1
app/models/manhole_domestic_chamber_model.py
View File

@@ -49,3 +49,4 @@ class ManholeDomesticChamber(db.Model):
def serialize(self):
return {c.name: getattr(self, c.name) for c in self.__table__.columns}

59
app/routes/dashboard.py
View File

@@ -1,11 +1,68 @@
import matplotlib
matplotlib.use("Agg")
from flask import Blueprint, render_template, session, redirect, url_for
import matplotlib.pyplot as plt
import io
import base64
dashboard_bp = Blueprint("dashboard", __name__, url_prefix="/dashboard")
# charts
def plot_to_base64():
img = io.BytesIO()
plt.savefig(img, format="png", bbox_inches="tight")
plt.close()
img.seek(0)
return base64.b64encode(img.getvalue()).decode()
# bar chart
def bar_chart():
categories = ["Trench", "Manhole", "Pipe Laying", "Restoration"]
values = [120, 80, 150, 60]
plt.figure()
plt.bar(categories, values)
plt.title("Work Category Report")
plt.xlabel("Category")
plt.ylabel("Count")
return plot_to_base64()
# Pie chart
def pie_chart():
labels = ["Completed", "In Progress", "Pending"]
sizes = [55, 20, 25]
plt.figure()
plt.pie(sizes, labels=labels, autopct="%1.1f%%", startangle=140)
plt.title("Project Status")
return plot_to_base64()
# Histogram chart
def histogram_chart():
daily_work = [5, 10, 15, 20, 20, 25, 30, 35, 40, 45, 50]
plt.figure()
plt.hist(daily_work, bins=5)
plt.title("Daily Work Distribution")
plt.xlabel("Work Units")
plt.ylabel("Frequency")
return plot_to_base64()
# Dashboaed page
@dashboard_bp.route("/")
def dashboard():
if not session.get("user_id"):
return redirect(url_for("auth.login"))
return render_template("dashboard.html", title="Dashboard")
return render_template(
"dashboard.html",
title="Dashboard",
bar_chart=bar_chart(),
pie_chart=pie_chart(),
histogram=histogram_chart()
)

13
app/routes/file_report.py
View File

@@ -9,6 +9,7 @@ from app.models.mh_ex_client_model import ManholeExcavationClient
from app.models.tr_ex_client_model import TrenchExcavationClient
from app.models.mh_dc_client_model import ManholeDomesticChamberClient
from app.utils.helpers import login_required
from app.models.laying_model import Laying
# --- BLUEPRINT DEFINITION ---
file_report_bp = Blueprint("file_report", __name__, url_prefix="/file")
@@ -43,6 +44,7 @@ class SubcontractorBill:
self.df_tr = pd.DataFrame()
self.df_mh = pd.DataFrame()
self.df_dc = pd.DataFrame()
self.df_lay = pd.DataFrame()
def Fetch(self, RA_Bill_No=None, subcontractor_id=None):
filters = {}
@@ -54,16 +56,20 @@ class SubcontractorBill:
trench = TrenchExcavation.query.filter_by(**filters).all()
mh = ManholeExcavation.query.filter_by(**filters).all()
dc = ManholeDomesticChamber.query.filter_by(**filters).all()
lay= Laying.query.filter_by(**filters).all()
print("Laying datas :",lay)
self.df_tr = pd.DataFrame([c.serialize() for c in trench])
self.df_mh = pd.DataFrame([c.serialize() for c in mh])
self.df_dc = pd.DataFrame([c.serialize() for c in dc])
self.df_lay = pd.DataFrame([c.serialize() for c in lay])
if not self.df_dc.empty and "MH_NO" in self.df_dc.columns:
self.df_dc.rename(columns={"MH_NO": "Node_No"}, inplace=True)
drop_cols = ["id", "created_at", "_sa_instance_state"]
for df in [self.df_tr, self.df_mh, self.df_dc]:
for df in [self.df_tr, self.df_mh, self.df_dc, self.df_lay]:
if not df.empty:
df.drop(columns=drop_cols, errors="ignore", inplace=True)
@@ -151,6 +157,7 @@ def report_file():
bill_gen.df_tr.to_excel(writer, index=False, sheet_name="Tr.Ex.")
bill_gen.df_mh.to_excel(writer, index=False, sheet_name="MH.Ex.")
bill_gen.df_dc.to_excel(writer, index=False, sheet_name="MH & DC")
bill_gen.df_lay.to_excel(writer, index=False, sheet_name="Laying")
output.seek(0)
return send_file(output, download_name=file_name, as_attachment=True)
@@ -160,7 +167,8 @@ def report_file():
tables = {
"tr": bill_gen.df_tr.to_html(classes=table_classes, index=False),
"mh": bill_gen.df_mh.to_html(classes=table_classes, index=False),
"dc": bill_gen.df_dc.to_html(classes=table_classes, index=False)
"dc": bill_gen.df_dc.to_html(classes=table_classes, index=False),
"lay": bill_gen.df_lay.to_html(classes=table_classes, index=False)
}
selected_sc_id = subcontractor_id
@@ -238,3 +246,4 @@ def client_vs_all_subcontractor():
tables["dc"] = df_dc_cmp.to_html(classes='table table-striped table-hover table-sm', index=False)
return render_template("generate_comparison_client_vs_subcont.html", tables=tables, ra_val=ra_val)

133
app/services/file_service.py
View File

@@ -1,19 +1,22 @@
import os
import pandas as pd
from werkzeug.utils import secure_filename
from app.utils.file_utils import ensure_upload_folder
from app.config import Config
from app import db
# Subcontractor models import
from app.models.trench_excavation_model import TrenchExcavation
from app.models.manhole_excavation_model import ManholeExcavation
from app.models.manhole_domestic_chamber_model import ManholeDomesticChamber
from app.models.laying_model import Laying
# Client models import
from app.models.tr_ex_client_model import TrenchExcavationClient
from app.models.mh_ex_client_model import ManholeExcavationClient
from app.models.mh_dc_client_model import ManholeDomesticChamberClient
from app.utils.file_utils import ensure_upload_folder
class FileService:
@@ -60,10 +63,12 @@ class FileService:
df_tr_ex = pd.read_excel(filepath, sheet_name="Tr.Ex.", header=12, dtype={"MH No": str})
df_mh_ex = pd.read_excel(filepath, sheet_name="MH Ex.", header=12, dtype={"MH No": str})
df_mh_dc = pd.read_excel(filepath, sheet_name="MH & DC", header=11, dtype={"MH No": str})
df_laying = pd.read_excel(filepath, sheet_name="Laying", header=11, dtype={"MH No": str})
self.process_trench_excavation(df_tr_ex, subcontractor_id, RA_Bill_No)
self.process_manhole_excavation(df_mh_ex, subcontractor_id, RA_Bill_No)
self.process_manhole_domestic_chamber(df_mh_dc, subcontractor_id, RA_Bill_No)
self.process_laying(df_laying, subcontractor_id, RA_Bill_No)
return True, "SUBCONTRACTOR File uploaded successfully."
@@ -96,18 +101,18 @@ class FileService:
if not location or not mh_no:
continue
exists = TrenchExcavation.query.filter_by(
subcontractor_id=subcontractor_id,
RA_Bill_No=RA_Bill_No,
Location=location,
MH_NO=mh_no,
).first()
# exists = TrenchExcavation.query.filter_by(
# subcontractor_id=subcontractor_id,
# RA_Bill_No=RA_Bill_No,
# Location=location,
# MH_NO=mh_no,
# ).first()
if exists:
errors.append(
f"Model-Tr.Ex. (Row {idx+1}): Duplicate → Location={location}, MH_NO={mh_no}"
)
continue
# if exists:
# errors.append(
# f"Model-Tr.Ex. (Row {idx+1}): Duplicate → Location={location}, MH_NO={mh_no}"
# )
# continue
record_data = {}
for col in df.columns:
@@ -154,18 +159,18 @@ class FileService:
if not location or not mh_no:
continue
exists = ManholeExcavation.query.filter_by(
subcontractor_id=subcontractor_id,
RA_Bill_No=RA_Bill_No,
Location=location,
MH_NO=mh_no,
).first()
# exists = ManholeExcavation.query.filter_by(
# subcontractor_id=subcontractor_id,
# RA_Bill_No=RA_Bill_No,
# Location=location,
# MH_NO=mh_no,
# ).first()
if exists:
errors.append(
f"Model-MH Ex. (Row {idx+1}): Duplicate → Location={location}, MH_NO={mh_no}"
)
continue
# if exists:
# errors.append(
# f"Model-MH Ex. (Row {idx+1}): Duplicate → Location={location}, MH_NO={mh_no}"
# )
# continue
record_data = {}
for col in df.columns:
@@ -212,18 +217,18 @@ class FileService:
if not location or not mh_no:
continue
exists = ManholeDomesticChamber.query.filter_by(
subcontractor_id=subcontractor_id,
RA_Bill_No=RA_Bill_No,
Location=location,
MH_NO=mh_no,
).first()
# exists = ManholeDomesticChamber.query.filter_by(
# subcontractor_id=subcontractor_id,
# RA_Bill_No=RA_Bill_No,
# Location=location,
# MH_NO=mh_no,
# ).first()
if exists:
errors.append(
f"Model-MH & DC (Row {idx+1}): Duplicate → Location={location}, MH_NO={mh_no}"
)
continue
# if exists:
# errors.append(
# f"Model-MH & DC (Row {idx+1}): Duplicate → Location={location}, MH_NO={mh_no}"
# )
# continue
record_data = {}
for col in df.columns:
@@ -245,6 +250,66 @@ class FileService:
db.session.commit()
# ---------------- Laying (Subcontractor) ----------------
def process_laying(self, df, subcontractor_id, RA_Bill_No):
df.columns = (
df.columns.astype(str)
.str.strip()
.str.replace(r"[^\w]", "_", regex=True)
.str.replace("__+", "_", regex=True)
.str.strip("_")
)
df = df.dropna(how="all")
if "Location" in df.columns:
df["Location"] = df["Location"].ffill()
errors = []
for idx, row in df.iterrows():
location = self.normalize(row.get("Location"))
mh_no = self.normalize(row.get("MH_NO"))
if not location or not mh_no:
continue
# exists = ManholeDomesticChamber.query.filter_by(
# subcontractor_id=subcontractor_id,
# RA_Bill_No=RA_Bill_No,
# Location=location,
# MH_NO=mh_no,
# ).first()
# if exists:
# errors.append(
# f"Model-MH & DC (Row {idx+1}): Duplicate → Location={location}, MH_NO={mh_no}"
# )
# continue
record_data = {}
for col in df.columns:
if hasattr(Laying, col):
val = row[col]
if pd.isna(val) or str(val).strip() in ["", "-", "—", "nan"]:
val = None
record_data[col] = val
record = Laying(
subcontractor_id=subcontractor_id,
RA_Bill_No=RA_Bill_No,
**record_data,
)
db.session.add(record)
if errors:
raise Exception(" | ".join(errors))
db.session.commit()
# ---------------- CLIENT FILE UPLOAD ----------------
def handle_client_file_upload(self, file, RA_Bill_No):

36
app/templates/base.html
View File

@@ -40,10 +40,10 @@
</a>
</li>
<!-- Subcontractor -->
<!-- Subcontractor Model -->
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" data-bs-toggle="dropdown" href="#">
<i class="bi bi-people-fill me-1"></i> Subcontractor
<i class="bi bi-people-fill me-1"></i> Subcontractor Model
</a>
<ul class="dropdown-menu dropdown-menu-dark">
<li>
@@ -59,10 +59,10 @@
</ul>
</li>
<!-- File System -->
<!-- Subcontractor File System -->
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" data-bs-toggle="dropdown" href="#">
<i class="bi bi-folder-fill me-1"></i> File System
<i class="bi bi-folder-fill me-1"></i>Subcontractor File System
</a>
<ul class="dropdown-menu dropdown-menu-dark">
<li>
@@ -81,7 +81,7 @@
<!-- Client System -->
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" data-bs-toggle="dropdown" href="#">
<i class="bi bi-building me-1"></i> Client System
<i class="bi bi-building me-1"></i> Client File System
</a>
<ul class="dropdown-menu dropdown-menu-dark">
<li>
@@ -90,11 +90,6 @@
</a>
</li>
<li>
<a class="dropdown-item" href="/report/comparison_report">
<i class="bi bi-arrow-left-right me-2"></i> client vs sub-cont. Comparison Report
</a>
</li>
<li>
<a class="dropdown-item" href="/file/client_vs_subcont">
<i class="bi bi-arrow-left-right me-2"></i> Comparison Report
@@ -103,6 +98,27 @@
</ul>
</li>
<!-- Reports -->
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" data-bs-toggle="dropdown" href="#">
<i class="bi bi-building me-1"></i> Reports
</a>
<ul class="dropdown-menu dropdown-menu-dark">
<li>
<a class="dropdown-item" href="/report/comparison_report">
<i class="bi bi-arrow-left-right me-2"></i> client vs sub-cont. Comparison Report
</a>
</li>
<!-- <li>
<a class="dropdown-item" href="/file/client_vs_subcont">
<i class="bi bi-arrow-left-right me-2"></i> Comparison Report
</a>
</li> -->
</ul>
</li>
<!-- Formats -->
<li class="nav-item">
<a class="nav-link" href="/file_format">

85
app/templates/dashboard.html
View File

@@ -1,10 +1,87 @@
{% extends "base.html" %}
{% block content %}
<h2 class="mb-4">Dashboard</h2>
<div class="card p-4 shadow-sm">
<h5>Welcome to Comparison Project</h5>
<p>This is dashboard panel.</p>
<div class="container-fluid px-2 px-md-4">
<h4 class="mb-3 text-center text-md-start">Comparison Software Solapur(UGD) </h4>
<!-- Summary Cards -->
<div class="row g-3 mb-4">
<!-- Total Work -->
<div class="col-12 col-md-4">
<div class="card text-white bg-primary shadow h-100">
<div class="card-body text-center text-md-start">
<h6>Total Work</h6>
<h3 class="fw-bold">410</h3>
</div>
</div>
</div>
<!-- Completed -->
<div class="col-12 col-md-4">
<div class="card text-white bg-success shadow h-100">
<div class="card-body text-center text-md-start">
<h6>Completed</h6>
<h3 class="fw-bold">265</h3>
</div>
</div>
</div>
<!-- Pending -->
<div class="col-12 col-md-4">
<div class="card text-dark bg-warning shadow h-100">
<div class="card-body text-center text-md-start">
<h6>Pending</h6>
<h3 class="fw-bold">145</h3>
</div>
</div>
</div>
</div>
<!-- Charts -->
<div class="row g-3">
<!-- Bar Chart -->
<div class="col-12 col-md-6">
<div class="card shadow-sm h-100">
<div class="card-header bg-dark text-white text-center text-md-start">
Work Category Bar Chart
</div>
<div class="card-body text-center">
<img src="data:image/png;base64,{{ bar_chart }}" class="img-fluid" style="max-height:300px;">
</div>
</div>
</div>
<!-- Pie Chart -->
<div class="col-12 col-md-6">
<div class="card shadow-sm h-100">
<div class="card-header bg-dark text-white text-center text-md-start">
Project Status Pie Chart
</div>
<div class="card-body text-center">
<img src="data:image/png;base64,{{ pie_chart }}" class="img-fluid" style="max-height:300px;">
</div>
</div>
</div>
<!-- Histogram -->
<div class="col-12">
<div class="card shadow-sm">
<div class="card-header bg-dark text-white text-center text-md-start">
Daily Work Histogram
</div>
<div class="card-body text-center">
<img src="data:image/png;base64,{{ histogram }}" class="img-fluid" style="max-height:350px;">
</div>
</div>
</div>
</div>
</div>
{% endblock %}

18
app/templates/generate_comparison_client_vs_subcont.html
View File

@@ -1,7 +1,8 @@
{% extends "base.html" %}
{% block content %}
<div class="container-fluid mt-4"> <h2 class="mb-4">File Comparison</h2>
<div class="container-fluid mt-4">
<h2 class="mb-4">File Comparison</h2>
{% with messages = get_flashed_messages(with_categories=true) %}
{% if messages %}
@@ -21,10 +22,12 @@
<div class="row">
<div class="col-md-6">
<button type="submit" name="action" value="preview" class="btn btn-secondary w-100">Preview Data</button>
<button type="submit" name="action" value="preview" class="btn btn-secondary w-100">Preview
Data</button>
</div>
<div class="col-md-6">
<button type="submit" name="action" value="download" class="btn btn-primary w-100">Download Excel Report</button>
<button type="submit" name="action" value="download" class="btn btn-primary w-100">Download Excel
Report</button>
</div>
</div>
</form>
@@ -36,13 +39,16 @@
<ul class="nav nav-tabs" id="reportTabs" role="tablist">
<li class="nav-item">
<button class="nav-link active" id="tr-tab" data-bs-toggle="tab" data-bs-target="#tr" type="button">Tr.Ex Comparison</button>
<button class="nav-link active" id="tr-tab" data-bs-toggle="tab" data-bs-target="#tr"
type="button">Tr.Ex Comparison</button>
</li>
<li class="nav-item">
<button class="nav-link" id="mh-tab" data-bs-toggle="tab" data-bs-target="#mh" type="button">Mh.Ex Comparison</button>
<button class="nav-link" id="mh-tab" data-bs-toggle="tab" data-bs-target="#mh" type="button">Mh.Ex
Comparison</button>
</li>
<li class="nav-item">
<button class="nav-link" id="dc-tab" data-bs-toggle="tab" data-bs-target="#dc" type="button">MH & DC Comparison</button>
<button class="nav-link" id="dc-tab" data-bs-toggle="tab" data-bs-target="#dc" type="button">MH & DC
Comparison</button>
</li>
</ul>

2
app/templates/login.html
View File

@@ -31,7 +31,7 @@
Laxmi Civil Engineering Services Pvt Ltd
</h4>
<p class="text-muted mb-0">
Data Comparison System
Data Comparison Software Solapur(UGD)
</p>
</div>

23
app/templates/report.html
View File

@@ -30,7 +30,8 @@
</div>
<div class="form-check form-switch mb-3">
<input class="form-check-input" type="checkbox" name="download_all" value="true" id="downloadAllSwitch" onchange="toggleRAInput(this)" {% if download_all %}checked{% endif %}>
<input class="form-check-input" type="checkbox" name="download_all" value="true" id="downloadAllSwitch"
onchange="toggleRAInput(this)" {% if download_all %}checked{% endif %}>
<label class="form-check-label fw-bold text-primary" for="downloadAllSwitch">
Download All RA Bills for this Contractor
</label>
@@ -38,7 +39,8 @@
<div class="mb-4" id="ra_bill_container">
<label class="form-label fw-semibold">RA Bill Number</label>
<input type="text" name="ra_bill_no" id="ra_bill_input" class="form-control" placeholder="e.g. 01" value="{{ ra_bill_no or '' }}">
<input type="text" name="ra_bill_no" id="ra_bill_input" class="form-control" placeholder="e.g. 01"
value="{{ ra_bill_no or '' }}">
<small class="text-muted">Required if "Download All" is off.</small>
</div>
@@ -66,13 +68,19 @@
<div class="card-header bg-light">
<ul class="nav nav-tabs card-header-tabs" id="reportTabs" role="tablist">
<li class="nav-item">
<button class="nav-link active" id="tr-tab" data-bs-toggle="tab" data-bs-target="#tr" type="button">Transport (Tr.Ex.)</button>
<button class="nav-link active" id="tr-tab" data-bs-toggle="tab" data-bs-target="#tr"
type="button">Transport (Tr.Ex.)</button>
</li>
<li class="nav-item">
<button class="nav-link" id="mh-tab" data-bs-toggle="tab" data-bs-target="#mh" type="button">Manpower (MH.Ex.)</button>
<button class="nav-link" id="mh-tab" data-bs-toggle="tab" data-bs-target="#mh"
type="button">Manpower (MH.Ex.)</button>
</li>
<li class="nav-item">
<button class="nav-link" id="dc-tab" data-bs-toggle="tab" data-bs-target="#dc" type="button">MH & DC</button>
<button class="nav-link" id="dc-tab" data-bs-toggle="tab" data-bs-target="#dc" type="button">MH
& DC</button>
</li>
<li class="nav-item">
<button class="nav-link" id="lay-tab" data-bs-toggle="tab" data-bs-target="#lay" type="button">Laying</button>
</li>
</ul>
</div>
@@ -92,6 +100,10 @@
{{ tables.dc | safe }}
</div>
</div>
<div class="tab-pane fade" id="lay" role="tabpanel">
<div class="table-responsive">
{{ tables.lay | safe }}
</div>
</div>
</div>
</div>
@@ -104,6 +116,7 @@
font-size: 0.9rem;
white-space: nowrap;
}
.table-responsive thead {
background-color: #f8f9fa;
}

4
app/templates/subcontractor/list.html
View File

@@ -11,7 +11,7 @@
</h4>
</div>
<div class="col-12 col-md-6 text-center text-md-end">
<div class="col-12 col-md-3 text-center text-md-end">
<a href="/subcontractor/add" class="btn btn-success w-100 w-md-auto">
➕ Add Subcontractor
</a>
@@ -65,7 +65,7 @@
</a>
</div>
</td>
/tr>
</tr>
{% endfor %}
</tbody>

1
requirements.txt
View File

@@ -6,3 +6,4 @@ Werkzeug
python-dotenv
cryptography
xlsxwriter
matplotlib