Back to Projects
GWU MSBA

Flight Big Data Analysis

Processed and transformed a 30M-row flight dataset on Azure Virtual Machines with optimized SQL queries.

Role: Data EngineerDate: 2024
Flight Big Data Analysis - GWU MSBA project

Situation

The Data Management course final project required processing a massive flight dataset to demonstrate cloud-based big data skills.

Task

Design and implement a scalable data processing pipeline on Azure that could handle 30 million rows efficiently.

Actions

  • Set up Azure Virtual Machines for distributed processing
  • Optimized SQL queries and data transformation logic
  • Designed aggregation and summarization pipelines
  • Produced analysis-ready datasets for downstream reporting

Results

30M

Rows processed

What I'd Do Next

Apply similar patterns to real-time streaming data scenarios.