Overview
Overarching Research Question: Analyze if the ongoing renovation project improved the reliability and performance of the MBTA system. Are there early indicators of significant improvement to the usage and performance of the MBTA system?
Goal: Gather and compile datasets relevant to our research question and its sub-questions. We identified reputable data sources and defined our collection methods by choosing an dataset accessed through an API, while finding datasets that have features which fit into requirements of future binary, multivariate, and regression analysis.
Motivation:
Centralize Data Access: This is a huge project and a tough question to answer! Consolidating datasets to one source not only makes our analysis easier, but also helps Jen and I collaborate.Contextual Insights: We want to collect as much datasets to inform our question. The more, the better. We want to gain a holistic understanding of MBTA from its performance to public perceptions
Objectives:
Identify and Gather Relevant and Reliable DatastsEnsure Data Coverage (2020-2024)Document data sourcesValidate data accessibility so recreation is possible
Dataset Info:
Data source 1
Datasource: MBTA Bluebook Open Data PortalType: Tabular (.csv)Structure: 2,736,828 rows, 28 features (combined)Years: 2020-2024Purpose:- Analyze causes and effects of service disruptions.
- Identify routes and stations prone to frequent alerts.
- Measure severity and frequency of alerts over time.
Data source 2
Datasource: Serpapi: Google News FeedType: Tabular (.json), APIStructure: 100 entriesYears: 2024Purpose:- Understand current public perception through text analysis (sentimen analysis, topic modeling, keyword extraction)
- Technical use of APIs
Data source 3
Datasource: MBTA Bluebook Open Data PortalType: Tabular (.csv)Structure: 3682 rows, 7 featuresYears: 2014-2024Purpose:- Evlauate long-term trends in service reliability
- Identify improvements or challenges in accessibility services.
Data source 4
Datasource: MBTA Bluebook Open Data PortalType: Tabular (.json)Structure: 7185 rows, 10 featuresYears: 2020-2024Purpose:- Find differences between predicted and actual service performance
- Evalute prediction accuracy trends
Data source 5
Datasource: Financial StatementsType: PDF (.json)Structure: 5 pdfs (transformed to 8 rows, 5 features during cleaning)Years: 2019-2024Purpose:- Understand macro-level trends in financial trends and whether it correlates with system performance (i.e. reliability dataset)
- examine important money values such as the net positions, expenses, revenues, and debts throughout the years from 2019 to 2023